📅 February 14, 2017
Do you have some extra RAM in your system?
Want better read speeds from slower mechanical hard drives?
How about adding a RAM drive to your system?
RapidDisk is an open source tool for Linux that enables you to create RAM disks for general purpose usage or use them as caching systems for existing hard drives.
I have been using RapidDisk myself on desktop systems, and I can say that it greatly improves the read performance of slow hard drives whether they be single drives or RAID arrays. You can even set up a RapidDisk RAM cache for a USB device. RAM disks are surpringly useful despite being volatile. Best of all, RapidDisk is free to obtain and simple to use after a little reading.
Here is how to use RapidDisk in Linux Mint 18.1 in a simple configuration so you can experience faster read speeds from your hard drives.
So, what is a RAM disk and what does RapidDisk do?
A RAM disk is just what it sounds like: A drive that exists in RAM. You can format it, read and write data, and perform any function with it that you can with any other USB or hard drive.
It is volatile, so all contents of the RAM disk are lost when the computer is turned off. However, it operates at the speed of RAM, which can be 7-15 GB/s depending upon the hardware.
This makes RAM disks great for fast, temporary storage, such as caches or web browser temp locations. Your imagination is the limit.
RapidDisk is not in the Ubuntu repository, so you must download the source from the RapidDisk web site (Look for Downloads and Installation). The site also contains a good description.
Prebuilt packages exist, but I had no success installing them. I had to compile the source.
Compiling was simple. Uncompress rapiddisk-master.zip and follow the directions in README.md.
Make sure that you have the development libraries
- libcryptsetup (libcryptsetup-dev)
- libjansson (libjansson-dev)
installed, which you can install from Synaptic.
sudo make sudo make install sudo make tools-install
sudo make dkms
DKMS (Dynamic Kernel Module Support) is not required, but it is recommended since it eliminates the need to recompile rapiddisk in case you ever upgrade your kernel.
To view the instructions, open the man page.
RapidDisk and RapidDisk-Cache Modules
There are two modules: rapiddisk and rapiddisk-cache. rapiddisk creates a RAM drive, and rapiddisk-cache allows the RAM drive to be used as a device cache.
Use modprobe to load a module temporarily.
sudo modprobe rapiddisk sudo modprobe rapiddisk-cache
To remove the modules,
sudo modprobe -r rapiddisk sudo modprobe -r rapiddisk-cache
Load Modules at Boot
modprobe is temporary. To tell Linux to use the rapiddisk modules upon boot, edit /etc/modules.
sudo gedit /etc/modules
Add two lines if they are not already present.
Note: During one installation, RapidDisk automatically added an entry
rapiddisk max_sectors=2048 nr_requests=1024
to /etc/modules, but this never worked for me (the RapidDisk module would not load). I commented it out and manually added a plain rapiddisk entry without any arguments. RapidDisk then worked.
Also, if you see other modules, such as dm_mod and dm_crypt, it is fine to leave them.
Is RapidDisk Running?
To check enter this command:
sudo rapiddisk --list
If the RapidDisk module is loaded, then you will see something like this:
If not, and error message is shown.
Keep in mind that rapiddisk is a sudo command.
With the preliminary steps complete, how to do we create a RAM drive? Easy. rapiddisk is the tool that handles RAM disks. Simple use rapiddisk with the –attach option followed by the size of the RAM drive in megabytes (MB).
sudo rapiddisk --attach 4096
This creates a 4GB RAM drive. (4096MB = 4GB) If successful, you will see a message with the device name, usually something like rd0, rd1, rd2 depending upon how many RAM drives you already have.
Note that even though rapiddisk will allow you to create a RAM disk greater than your RAM’s capacity, it would defeat the purpose. For best results, ensure that you stay within your RAM limits. For example, if you have 16GB of RAM, do not create a 128GB RAM disk. 4-8GB is a reasonable range if you have at least that much RAM.
sudo rapiddisk –list shows all current RAM drives and their capacities.
sudo rapiddisk --list
(sudo rxadm –list also works.)
This tells us that we have two RAM drives named rd0 and rd1 and no RAM caching. We will add a cache later.
We can also use
to view pretty much the same thing.
Removing a RAM Disk
In the above example, a 32G RAM disk plus a 4GB RAM disk requires 36GB of free RAM. This is more than the available system RAM. Let’s remove the 32GB RAM disk with the –detach option.
sudo rapiddisk --detach rd1
Viewing the details again shows that rd1 has been removed.
sudo rapiddisk --list
If you open System Monitor to view memory usage as you create the drives, you will notice that there is no RAM usage. This is because RapidDisk fills a RAM disk as it is needed. So, even though we create a 4GB RAM drive, no RAM is actually used until we write files to it. Then, System Monitor shows an increase in RAM usage.
Formatting the RAM Drive
The newly created RAM drive, rd0, does not appear in sudo fdisk -l or in GParted, but it still exists. The RapidDisk RAM drive, rd0, exists in /dev/rd0. If you have more RAM drives, such as rd1 and rd2, the you can access them at /dev/rd1 and /dev/rd2.
This is important because we must use /dev/rd0 (for rd0) in order to format the drive before mounting.
sudo mkfs.ext4 /dev/rd0
This formats the RAM drive as ext4, but you can format it using any supported file system that you wish.
By default, ext4 reserves 5% of the block space following a format. RAM is scarce, so let us disable this by setting the reserved blocks to 0. This RAM disk will be used for data, after all, and the reserved space is a waste in this case.
sudo tune2fs -m 0 /dev/rd0
Mounting the RAM Disk
With the RAM disk formatted, let’s mount it.
sudo mkdir /media/ramdrive sudo mount /dev/rd0 /media/ramdrive
Any mount point you like is allowed just like any other device. I created a folder in /media/ramdrive since anything mounted in /media will automount in Nemo for easy access. As far as Linux is concerned, the RAM drive is a hard drive, so you can treat it like you would a hard drive.
At this point, the RAM drive is ready for use like any other hard drive. You can create directories, change ownership and permissions, and write and read files.
How Fast is It?
Using the Disks Benchmark utility, let’s check the speed of this RAM disk.
After running a 100x100M test three times, 3.1 GB/s read/write throughput remained consistent. This speed is correct for this test system that uses slow RAM.
Faster performance is expected on systems with faster RAM.
Note that benchmarks will cause RapidDisk to use the RAM disk capacity, so you will see an increase in RAM usage in System Monitor. This is normal. The RAM disk will be limited to the size specified during creation, but if you find that you are needed more space, you can dynamically grow the size of the RAM drive with the –resize option.
Quick Review: RAM Disk Creation
sudo rapiddisk --attach 4096 sudo mkfs.ext4 /dev/rd0 sudo tune2fs -m 0 /dev/rd0 sudo mount /dev/rd0 /media/ramdrive
Bye, Bye RAM Disk Data
The RAM disk is as fast as your RAM will allow, but it will be destroyed and all contents lost when the computer is turned off. Bash scripts are your friend if you want to create a RAM drive upon system startup.
Archiving and Encrypting RAM Disk Contents
RapidDisk supports archiving and retrieval. You can save the current contents of the RAM disk to a file using the –archive option.
When the computer started up again, you can recover a RAM disk’s contents with the –restore option.
“Is it Worth Archiving and Restoring?”
In my opinion, no. It takes too long. From my tests, it took over 10 minutes to archive a 4GB RAM disk because the contents are also compressed to save hard drive space. If archiving before shutdown, then this increases the shutdown time. For larger RAM disks, the wait time will be longer.
Hard Drive Cache
Extending the RAM disk idea, we can map a RAM disk “between” a slow hard drive and the system to improve read performance. The RAM drive caches the most frequently used data. Once the data is in cache, by reading it for the first time, subsequent reads for the same data will be read from the much faster RAM.
With a RAM drive working, how do we use it as a cache for a slower hard drive?
Easy. The –cache-map option maps a RAM disk to a hard drive.
We will reuse rd0, the 4G RAM disk.
sudo umount /media/ramdrive (Should already be unmounted if benchmarked.) sudo rapiddisk --list (To verify rd0.)
In this example, we will cache a slow 5400 RPM hard drive /dev/sdc. Here is the drive’s benchmark:
With the RAM disk created as rd0, we use the –cache-map option to map the RAM disk to the hard drive.
sudo rapiddisk --cache-map rd0 /dev/sdc1
If successful, you should see a message like this:
Note the new name of the cached hard drive: rc-wt_sdc1. This name might vary based upon the device name of your hard drive. sudo fdisk -l now shows this device as /dev/mapper/rc-wt_sdc1. We must use this in order to mount the hard drive.
This particular drive already contains a partition, /dev/sdc1. You can map partitions or entire drives.
Mounting the Cache
We do not mount the hard drive /dev/sdc. Instead, we mount its RAM disk cache /dev/mapper/rc-wt_sdc-part1 in the mount point /media/ramdrive.
sudo mount /dev/mapper/rc-wt_sdc1 /media/ramdrive
DO NOT FORMAT!
You can use a hard drive or USB device that is already formatted and already contains data, so there is no need to format. If you do, you will lose your data, so double-check your actions.
Of course, if the hard drive is unformatted, then you should format it before mapping it to a RAM drive.
We can use sudo rapiddisk –list to view the current status and see if the cache was mapped.
Benchmarking the RAM Cache
Does RapidDisk work? We use the Disks Benchmark again, but this time we benchmark /dev/mapper/rc-wt_sdc1 (the RAM cache for the hard drive), not the original drive /dev/sdc. Benchmarks should show a slight improvement upon second and third runs.
Data must first be cached into the RAM cache before we see any speed improvements. This is why the first run for both is lower. However, the 2nd and 3rd benchmark runs show an increase. The smaller file test shows the most improvement, but this is probably because of a combination of the RAM cache and the hard drive’s internal cache.
Also, the computer was not restarted for the 200x1M cache, so the 1st run results seem higher than normal. However, the 2nd and 3rd test runs are consistent with cached data.
Why is the Write Speed the Same?
Did you notice something about the benchmark results above? Read speeds improved, but the writes remained mostly the same.
This is normal.
RapidDisk only caches reads, not writes. if we look at the sudo rapiddisk –list result, we see that the RAM cache was set to WRITE THROUGH mode. Write through is one of the more common caching modes (the other is write back) that writes to the hard drive at the speed of the drive. This ensures that data is written to the physical media and helps prevent data loss.
For a RAM cache, this is what we want for reliability. Write see no improvement, but reads improve. When data is read from the hard drive, it is cached into the RAM cache. Upon subsequent reading of the same data, files for example, the data is retrieved from the faster RAM cache instead of the slower hard drive.
This offers an excellent read boost for files that you or multiple users access often. In real life, this translates to less waiting and makes your computer feel “snappier.” Of course, new data must be read at the speed of the hard drive.
If RAM speeds are 3.1 GB/s, why are these speeds so low?
In the RAM drive created earlier, we benchmarked ~3.1 GB/s for reads and writes. If the cache is using the same RAM on the same system, then why isn’t the cache operating at 3.1 GB/s? Why the ~190-700MB/s read speeds? This is slower than an SSD.
At this moment, I am not sure. For some reason, RapidDisk RAM caching always results in read speeds lower than what is possible with a plain RAM drive. The results are still higher – much higher – than the drive’s speed limits, but the 3-8GB/s speeds never materialize. My guess for right now is that overhead or a misconfiguration somewhere on my part is limiting the full speed potential.
Access times are almost nonexistent with a RAM cache, so this helps reduce wear and tear on a mechanical hard drive.
What About a Real File Test?
Using time, I tested how long it would take to transfer two Linux Mint ISO images totaling 3.7GB. Would caching improve the read speeds?
First, I copied them TO the drive mounted at /media/ramdrive.
Then, I benchmarked the RAM drive again using Disks in order to replace any cached data. A –flush option exists, but the man page warns against using it.
Next, I performed the actual test, which took ~40 seconds for the first read. The first read will usually be limited to the read speed of the hard drive, so its numbers should be the lowest.
Finally, I performed a second read and then a third.
Caching an External USB Drive?
Is it possible to set up a RAM cache to improve the read speeds of a USB device or an external USB drive?
The setup is the same. First, we will unmount and detach the existing RAM cache so we can
sudo umount /media/ramdrive sudo rapiddisk --cache-unmap rc-wt_sdc1
Note that to unmap a RAM cache, we use the –cache-unmap option and the name of the cache device, which is rc-wt_sdc1 in this case, not /dev/mapper/rc-wt_sdc1.
Using sudo fdisk -l, find the device of the USB drive. In this example, it is /dev/sdd1, another partition on a USB2 drive.
Next, we map the rd0 RAM cache to the hard drive like we did before. (If the USB drive automounts, be sure to unmount it first. You cannot map a cache to a mounted drive.)
sudo rapiddisk --cache-map rd0 /dev/sdd1
Now, mount the RAM cache.
sudo mount /dev/mapper/rc-wt_sdd1 /media/ramdrive
This drive already had two linux ISO images on it totaling 3.2GB, so I timed the copy to home.
This is a 5400 RPM USB2 drive limited by a USB2 interface. Let’s run the Disks benchmark for testing.
Original 100x10M Result. No RAM Cache.
We must benchmark /dev/mapper/rc-wt_sdd1, which is the mapped RAM cache, not the actual drive /dev/sdd.
The system was not restarted nor the cache cleared for the 200x1M benchmark. In all cases, subsequent reads were faster than the first read benchmark while writes remained the same. Again, RapidDisk does not cache writes. Only reads. This is normal operation.
For USB2, the read performance is an improvement. This can be applied to any USB device including USB sticks.
Can I Cache a RAID Array?
Yes. Simply set up the RAM cache in the same way. RapidDisk setup and operation is the same whether you use a USB stick, a hard drive, or a RAID array.
After setting up a test RAID array on a different system using faster RAM and a faster CPU, read speeds increased to 1.3GB/s over the native 140MB/s. Definitely an improvement.
If you have spare RAM and slow mechanical hard drives, then RapidDisk helps. Find yourself accessing the same files repeatedly? Are other users accessing 20% of the file 80% of the time? RapidDisk will improve Linux performance.
I like it. RapidDisk has proven its worth, and it is one of the extra tricks you can implement to give the illusion of a faster system without upgrading to expensive M.2 NVMe or high-capacity SSDs.
While I was hoping for cached read speeds equivalent to RAM speeds, it could be that my test system was at fault by using slower RAM, a slower CPU, and older parts. A quick test on a system with a faster CPU yielded faster cached read speeds, but they were still not up to native RAM speeds.
On the other hand, pure RAM disks do operate at the full RAM speed, so I am not quite certain where the bottleneck lies.
Overall, RapidDisk is a worthy improvement to any Linux system seeking improved read speeds from mechanical hard drives. And yes, RapidDisk works with SSDs too.