The Raspberry Pi 4 is the fastest of all Pi. It brings new interfaces to the Pi family: The USB 3.0 port, and gigabit ethernet. Now, people won't have to be constrained by the crappy USB 2.0 interface of the previous Raspberry Pi generations. The Raspberry Pi Compute Module 4 is even better. A whole PCIe 2.0 port? That's like 5Gbps of transfer speed! That is enough to enjoy the lightning speed of NVMe SSDs.
"Oh my goodness, I can't wait to abuse its 5GT/s data link!"I thought.
No! Of course there is a catch. Raspberry Pi 4's CPU does not have hardware-acceleration of common encryption ciphers (wink AES wink). The Raspberry Pi foundation somehow decided that hardware-based encryption is of no importance, even though its competitors have that years ago!
So anything remotely related to encryption will be slow. To name a few:
- Downloading a file from the Internet (Most websites use HTTPS)
- Anything related to torrents
- Encrypting hard drives
- ...and yes, SSH and SFTP
Fortunately, hope is not all lost. It is still possible to saturate the 1Gbps ethernet link while downloading your favorite movies. Similarly, you can reduce the CPU load on your puny Raspberry Pi while transferring large files.
Use a 64-bit OS for your Pi
Yes, a 64-bit OS actually speeds up your Raspberry Pi, by a lot. I mean, Raspberry Pi 4's
bcm2711 processor supports 64-bit, might as well use it.
If you have enough RAM, that is. It is true that 64-bit systems consume more RAM than their 32-bit counterparts. RAM won't be a problem for most Pi users, as most Pi have 2GB RAM. So if your Pi 4 is the 1GB RAM model, why did you buy it?2GB model costs like the same price as the 1GB model. Moreover, if you have a 4GB or 8GB model, using a 64-bit system enables an application to allocate more than 3GB RAM. So, quite a win-win situation I believe.
Official 64-bit Raspberry Pi OS
64-bit OS historically had very bad support from the Raspberry Pi Foundation. The good news is: Albeit in beta, Raspberry Pi Foundation finally released an official 64-bit Raspberry Pi OS Lite! It is better than 32-bit Raspberry Pi OS in almost every way. Even the system itself takes up less spaces! It can be downloaded here.
Basically, if you are most people, use the 64-bit Raspberry Pi OS. It automagically makes all programs faster, which includes cryptographic libraries.
Testing SFTP speed
Some people might wonder: "I know 64-bit makes applications faster, but how do you know it improves SFTP specifically?" It is actually quite easy to figure it out yourself. Just connect to the Pi itself and try to upload a file!
The procedure is simple:
- Create a temporary file on the Pi in a ramdisk.
- On the Pi, connect to itself via SFTP.
- Upload the temporary file to a ramdisk, basically copying the file through SFTP.
This eliminates factors such as SD card performance, ethernet stability, and USB performance.
And the testing can be done by executing the commands below on your Pi:
dd if=/dev/zero of=/tmp/testfile bs=1M count=512 status=progress; sync sftp pi@localhost sftp> put /tmp/testfile /tmp/out.tmp
From my tests, SFTP's transfer speed averages about
55.9 MB/s on a 32-bit system, while averaging about
76.6 MB/s on the 64-bit one.
That is a stunning 37% improvement in upload speed, by switching Pi OS to 64-bit alone!
Yes. Overclocking. If the program is CPU-bound, and you need to squeeze out some extra performance, the obvious choice is to overclock the CPU and make some smoke.
And yes, encryption ciphers are CPU-bound. So let's try overclocking!
Overclocking might damage your Raspberry Pi!
Basically, you only need to insert these two lines into your
This config will overclock your Raspberry Pi's CPU from
1.5 GHz to
1.75 GHz. About ~15% increase in frequency.
And the results are pretty good:
Our overclocked Pi yields
84.4 MB/s SFTP transfer speed. A nice 10% increase. If you want to be more overkill, you can overclock the CPU to
2.0 GHz for a theoretical
102.13 MB/s upload speed. That is pretty close to saturating the 1Gbps ethernet!
But what if we want something else?
Use an alternative file transfer protocol
It is known that SFTP is horribly slow.
It is true! For example, SFTP uses
chacha20-poly1305 cipher on Raspberry Pi, which is actually really fast. If you benchmark its speed on the Pi with this command
openssl speed -evp chacha20-poly1305
That is some
300 MB/s of raw
chacha20-poly1305 speed! So, if there are alternative means such as HTTP download, use it! SFTP is simply way too inefficient for file transfer.
Use an alternative encryption method
We can simply ask SSH/SFTP to use a more Raspberry Pi-friendly cipher.
aes-gcm is horribly slow and (somewhat) insecure on devices without hardware
chacha20-poly1305 is the de-facto cipher for mobile devices without
AES hardware acceleration.
chacha20-poly1305 should already be the default cipher for Raspberry Pi. It provides enough security and speed.
But if you want more speed, you can use a weaker and potentially insecure cipher like arcfour. Just don't do it.
So yeah, considering Raspberry Pi's raw CPU power, saturating a 1Gbps link is a piece of cake even with encryption (unless you are using SFTP like me). Using a 64-bit system and a moderate overclocking results in 40% faster file transfer in SFTP. This really shows the power of a modern instruction set (aarch64).
Thus, if there are other means to upload/download files, please use them. Even HTTPS download is faster than SFTP.
Hey! It might even be possible to use a PCIe-to-ethernet adapter and saturate a 5Gbps ethernet link! Unfortunately, that is for another time.