Rpi 5 with Pimoroni NVMe Base poor performance

Support
1

Hi all,

Recently I’ve bought a NVMe Base for my newly arrived Rpi 5.
I’ve assemled it with an ADATA 512GB NVME SSD.

I wanted to test the speed with hdparm, and I’ve got this results:

sudo hdparm -t --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT disk reads: 2500 MB in 3.00 seconds = 832.83 MB/sec I’ve got it once

sudo hdparm -t --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT disk reads: 2 MB in 3.34 seconds = 613.04 kB/sec usually got this result

the actual config.txt:
dtparam=nvme
dtparam=pciex1
dtparam=pciex1_gen=3

but if I delete _gen=3, the result is the same.

I use the Rpi5 headless mode, and when I connect to it with ssh/VNC sometimes I feel it sluggish, and no, not because the wifi.

I left USB current limit in enabled mode.

Any idea?
What am I doing wrong?

sudo smartctl -i /dev/nvme0n1
smartctl 7.3 2022-02-28 r5338 [aarch64-linux-6.1.0-rpi7-rpi-2712] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number: ADATA SX6000PNP
Serial Number: 2N0829Q2K5KJ
Firmware Version: VC0S036M
PCI Vendor/Subsystem ID: 0x1cc1
IEEE OUI Identifier: 0x707c18
Controller ID: 1
NVMe Version: 1.4
Number of Namespaces: 1
Namespace 1 Size/Capacity: 512,110,190,592 [512 GB]
Namespace 1 Formatted LBA Size: 512
Namespace 1 IEEE EUI-64: 707c18 60f248eea6
Local Time is: Thu Jan 11 18:06:48 2024 CET

2

You could try to run sudo journalctl -f in a second terminal to see if you have any errors in the system log.

3

sudo journalctl -f
Jan 11 21:12:13 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 11 21:12:13 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 11 21:12:13 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 11 21:12:46 raspberrypi5 kernel: nvme nvme0: I/O 81 (I/O Cmd) QID 1 timeout, aborting
Jan 11 21:12:46 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0

Jan 11 21:15:22 raspberrypi5 kernel: nvme nvme0: I/O 578 (I/O Cmd) QID 2 timeout, aborting
Jan 11 21:15:22 raspberrypi5 kernel: nvme nvme0: I/O 579 (I/O Cmd) QID 2 timeout, aborting
Jan 11 21:15:22 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 11 21:15:22 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0

4

SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 41 Celsius
Available Spare: 100%
Available Spare Threshold: 32%
Percentage Used: 0%
Data Units Read: 72,453 [37.0 GB]
Data Units Written: 87,157 [44.6 GB]
Host Read Commands: 610,462
Host Write Commands: 461,070
Controller Busy Time: 0
Power Cycles: 35
Power On Hours: 7
Unsafe Shutdowns: 9
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0

Error Information (NVMe Log 0x01, 8 of 8 entries)
No Errors Logged

5

Ok, with all these errors I would not give anything for the output of smartctl. You obviously have problems with the communication between Pi an the SSD. First thing is to check the cable and cable-connections.

Also, I would remove and reinsert the SSD (probably vodoo, but you never know). Also, keep away from gen3.

To really find out the root cause of the problem, it would be helpful to have an external USB-enclosure for the SSD. With such an enclosure you could rule out the SSD as the problem.

6

Since I wrote my last message I recable it and disable current limit. I’ve also commented out gen3.
At first, in the journal, I couldn’t seen the abort status message, but after a while they appeared again.

I have an USB exclosure, I’ve checked the ssd with it under windows an I could copy/delete a PDF to the bootfs quick. So I could check it later today.

7

The usb current limit is for usb-devices and normally does not influence the SSD. Unless you disable the limit and have multiple current-hungry USB-devices attached.

But this raises an interesting question: are you using the official power-supply? I am asking this because maybe there is some issue with the voltage. Do you see any messages in the log pointing at this direction? sudo journalctl -b will give you all the messages since last boot.

Another thing to test: does your system always work after some time of rest (i.e. after everything has cooled down)?

8

I didn’t attached (yet) anything else.
I’ve bought the Rpi5 with the official power supply.
I’ll check sudo journalctl -b.
When everything settle down I’ve got a lots of kernel: nvme nvme0: Abort status: 0x0, kernel: nvme nvme0: I/O 81 (I/O Cmd) QID 1 timeout, aborting messages.

9

I’ve reinstalled the OS to the ssd and assemble it again.

Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: I/O 261 (I/O Cmd) QID 4 timeout, aborting
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: I/O 262 (I/O Cmd) QID 4 timeout, aborting
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: I/O 263 (I/O Cmd) QID 4 timeout, aborting
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: I/O 264 (I/O Cmd) QID 4 timeout, aborting
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
Jan 12 16:38:19 raspberrypi5 kernel: nvme nvme0: Abort status: 0x0
^C
babusag@raspberrypi5:~ $ sudo journalctl -b
Jan 12 16:31:14 raspberrypi5 kernel: Booting Linux on physical CPU 0x0000000000 [0x414fd0b1]
Jan 12 16:31:14 raspberrypi5 kernel: Linux version 6.1.0-rpi7-rpi-2712 (debian-kernel@lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0, GNU ld (GNU B>
Jan 12 16:31:14 raspberrypi5 kernel: random: crng init done
Jan 12 16:31:14 raspberrypi5 kernel: Machine model: Raspberry Pi 5 Model B Rev 1.0
Jan 12 16:31:14 raspberrypi5 kernel: efi: UEFI not found.
Jan 12 16:31:14 raspberrypi5 kernel: Reserved memory: created CMA memory pool at 0x0000000018000000, size 320 MiB
Jan 12 16:31:14 raspberrypi5 kernel: OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
Jan 12 16:31:14 raspberrypi5 kernel: Zone ranges:
Jan 12 16:31:14 raspberrypi5 kernel: DMA [mem 0x0000000000000000-0x00000000ffffffff]
Jan 12 16:31:14 raspberrypi5 kernel: DMA32 empty
Jan 12 16:31:14 raspberrypi5 kernel: Normal [mem 0x0000000100000000-0x00000001ffffffff]
Jan 12 16:31:14 raspberrypi5 kernel: Movable zone start for each node
Jan 12 16:31:14 raspberrypi5 kernel: Early memory node ranges
Jan 12 16:31:14 raspberrypi5 kernel: node 0: [mem 0x0000000000000000-0x000000000007ffff]
Jan 12 16:31:14 raspberrypi5 kernel: node 0: [mem 0x0000000000080000-0x000000003fbfffff]
Jan 12 16:31:14 raspberrypi5 kernel: node 0: [mem 0x0000000040000000-0x00000001ffffffff]
Jan 12 16:31:14 raspberrypi5 kernel: Initmem setup node 0 [mem 0x0000000000000000-0x00000001ffffffff]
Jan 12 16:31:14 raspberrypi5 kernel: On node 0, zone DMA: 256 pages in unavailable ranges
Jan 12 16:31:14 raspberrypi5 kernel: psci: probing for conduit method from DT.
Jan 12 16:31:14 raspberrypi5 kernel: psci: PSCIv1.1 detected in firmware.
Jan 12 16:31:14 raspberrypi5 kernel: psci: Using standard PSCI v0.2 function IDs
Jan 12 16:31:14 raspberrypi5 kernel: psci: MIGRATE_INFO_TYPE not supported.
Jan 12 16:31:14 raspberrypi5 kernel: psci: SMC Calling Convention v1.2
Jan 12 16:31:14 raspberrypi5 kernel: percpu: Embedded 14 pages/cpu s177448 r8192 d43736 u229376
Jan 12 16:31:14 raspberrypi5 kernel: pcpu-alloc: s177448 r8192 d43736 u229376 alloc=14*16384
Jan 12 16:31:14 raspberrypi5 kernel: pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
Jan 12 16:31:14 raspberrypi5 kernel: Detected PIPT I-cache on CPU0
Jan 12 16:31:14 raspberrypi5 kernel: CPU features: detected: Virtualization Host Extensions
Jan 12 16:31:14 raspberrypi5 kernel: CPU features: detected: Hardware dirty bit management
Jan 12 16:31:14 raspberrypi5 kernel: CPU features: detected: Spectre-v4
Jan 12 16:31:14 raspberrypi5 kernel: CPU features: detected: Spectre-BHB
Jan 12 16:31:14 raspberrypi5 kernel: CPU features: kernel page table isolation forced ON by KASLR
Jan 12 16:31:14 raspberrypi5 kernel: CPU features: detected: Kernel page table isolation (KPTI)
Jan 12 16:31:14 raspberrypi5 kernel: alternatives: applying boot alternatives
Jan 12 16:31:14 raspberrypi5 kernel: Built 1 zonelists, mobility grouping on. Total pages: 521984
Jan 12 16:31:14 raspberrypi5 kernel: Kernel command line: coherent_pool=1M 8250.nr_uarts=1 pci=pcie_bus_safe snd_bcm2835.enable_compat_alsa=0 snd_bcm283>
Jan 12 16:31:14 raspberrypi5 kernel: Unknown kernel command line parameters “splash”, will be passed to user space.
Jan 12 16:31:14 raspberrypi5 kernel: Dentry cache hash table entries: 1048576 (order: 9, 8388608 bytes, linear)
Jan 12 16:31:14 raspberrypi5 kernel: Inode-cache hash table entries: 524288 (order: 8, 4194304 bytes, linear)
Jan 12 16:31:14 raspberrypi5 kernel: mem auto-init: stack:all(zero), heap alloc:off, heap free:off
Jan 12 16:31:14 raspberrypi5 kernel: software IO TLB: area num 4.

sudo hdparm -t --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT disk reads: 2 MB in 3.19 seconds = 641.69 kB/sec

I’ve started to install the OS to another SSD

10

I’ve changed the ssd.
It seems good now:
sudo hdparm -t --direct /dev/nvme0n1

/dev/nvme0n1:
Timing O_DIRECT disk reads: 1162 MB in 3.00 seconds = 387.02 MB/sec
Unfortunatelly it is only 128GB.

but now, a strange side effect occured.
I couldn’t connect to it with ssh/vnc, so I had to connect a monitor and keyboard/mouse to it.
When OS loaded, it seemed, that the wifi couldn’t connect to my 5GHz network.
So I had to add an 2,4GHz network.
With the previous ssd it wasn’t an issue.
I’m planning to reset my mikrotik dualband wifi router when my family go rest.
I’ll see would it help the connection.

11

New ssd:

sudo smartctl -i /dev/nvme0n1
smartctl 7.3 2022-02-28 r5338 [aarch64-linux-6.1.0-rpi7-rpi-2712] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number: LITEON CL1-4D128
Serial Number: SS0V15982L1TH9AN04LC
Firmware Version: 1S262
PCI Vendor ID: 0x14a4
PCI Vendor Subsystem ID: 0x126f
IEEE OUI Identifier: 0x002303
Total NVM Capacity: 128,035,676,160 [128 GB]
Unallocated NVM Capacity: 0
Controller ID: 1
NVMe Version: 1.3
Number of Namespaces: 1
Namespace 1 Size/Capacity: 128,035,676,160 [128 GB]
Namespace 1 Formatted LBA Size: 512
Namespace 1 IEEE EUI-64: 002303 5630221d65
Local Time is: Fri Jan 12 21:10:10 2024 CET

12

Wifi router reset helped, I could switch to the 5GHz network.