RapidDisk – Improved Hard Drive Caching

📅 February 14, 2017
coverDo you have some extra RAM in your system?

Want better read speeds from slower mechanical hard drives?

How about adding a RAM drive to your system?

 

RapidDisk is an open source tool for Linux that enables you to create RAM disks for general purpose usage or use them as caching systems for existing hard drives.

I have been using RapidDisk myself, and I can say that it greatly improves the read performance of slow hard drives whether they be single drives or RAID arrays. You can even set up a RapidDisk RAM cache for a USB device. RAM disks are surpringly useful despite being volatile. Best of all, RapidDisk is free to obtain and simple to use after a little reading.

Here is how to use RapidDisk in Linux Mint 18.1 in a simple configuration so you can experience faster read speeds from your hard drives.

Overview

So, what is a RAM disk and what does RapidDisk do?

A RAM disk is just what it sounds like: A drive that exists in RAM. You can format it, read and write data, and perform any function with it that you can with any other USB or hard drive.

a

Hard drive/RAM drive comparison.

It is volatile, so all contents of the RAM disk are lost when the computer is turned off. However, it operates at the speed of RAM, which can be 7-15 GB/s depending upon the hardware.

This makes RAM disks great for fast, temporary storage, such as caches or web browser temp locations. Your imagination is the limit.

 

Installation

RapidDisk is not in the Ubuntu repository, so you must download the source from the RapidDisk web site  (Look for Downloads and Installation). The site also contains a good description.

Prebuilt packages exist, but I had no success installing them. I had to compile the source.
Compiling was simple. Uncompress rapiddisk-master.zip and follow the directions in README.md.
Make sure that you have the development libraries

  • zlib
  • libcryptsetup (libcryptsetup-dev)
  • libjansson (libjansson-dev)

installed, which you can install from Synaptic.

After that,

sudo make
sudo make install
sudo make tools-install

Optional

sudo make dkms

DKMS (Dynamic Kernel Module Support) is not required, but it is recommended since it eliminates the need to recompile rapiddisk in case you ever upgrade your kernel.

Man Page

To view the instructions, open the man page.

man rapiddisk
a

RapidDisk man page.

RapidDisk and RapidDisk-Cache Modules

There are two modules: rapiddisk and rapiddisk-cache. rapiddisk creates a RAM drive, and rapiddisk-cache allows the RAM drive to be used as a device cache.
Use modprobe to load a module temporarily.

sudo modprobe rapiddisk
sudo modprobe rapiddisk-cache

To remove the modules,

sudo modprobe -r rapiddisk
sudo modprobe -r rapiddisk-cache

Load Modules at Boot

modprobe is temporary. To tell Linux to use the rapiddisk modules upon boot, edit /etc/modules.

sudo gedit /etc/modules

Add two lines if they are not already present.

rapiddisk
rapiddisk-cache

Save.

Note: During one installation, RapidDisk automatically added an entry

rapiddisk max_sectors=2048 nr_requests=1024

to /etc/modules, but this never worked for me (the RapidDisk module would not load). I commented it out and manually added a plain rapiddisk entry without any arguments. RapidDisk then worked.

Also, if you see other modules, such as dm_mod and dm_crypt, it is fine to leave them.

a

/etc/modules with the rapiddisk modules. These will be loaded at system startup. Note the automatic rapiddisk line is commented out since I had no success with it.

Is RapidDisk Running?

To check enter this command:

sudo rapiddisk --list

If the RapidDisk module is loaded, then you will see something like this:

a

RapidDisk module loaded and active, but no RAM disks yet.

If not, and error message is shown.

a

Oops! RapidDisk module not loaded. RapidDisk will not work.

Keep in mind that rapiddisk is a sudo command.

Usage

With the preliminary steps complete, how to do we create a RAM drive? Easy. rapiddisk is the tool that handles RAM disks. Simple use rapiddisk with the –attach option followed by the size of the RAM drive in megabytes (MB).

sudo rapiddisk --attach 4096

This creates a 4GB RAM drive. (4096MB = 4GB) If successful, you will see a message with the device name, usually something like rd0, rd1, rd2 depending upon how many RAM drives you already have.

Here, the first RAM drive exists as device name rd0.

Here, the first RAM drive exists as device name rd0.

Note that even though rapiddisk will allow you to create a RAM disk greater than your RAM’s capacity, it would defeat the purpose. For best results, ensure that you stay within your RAM limits. For example, if you have 16GB of RAM, do not create a 128GB RAM disk. 4-8GB is a reasonable range if you have at least that much RAM.

a

A second RAM disk was created. RapidDisk named it rd1, and it is 32GB. This exceeds the system RAM, but RapidDisk still created it. Would it work? Most likely, but it would require swap space on the hard drive, and this defeats the reason for creating a RAM drive — speed.

Viewing Details

sudo rapiddisk –list shows all current RAM drives and their capacities.

sudo rapiddisk --list
a

Listing RapidDisk RAM drives.

(sudo rxadm –list also works.)

This tells us that we have two RAM drives named rd0 and rd1 and no RAM caching. We will add a cache later.

We can also use

cat /sys/kernel/rapiddisk/mgmt

to view pretty much the same thing.

a

More RapidDisk info regarding the current configuration.

Removing a RAM Disk

In the above example, a 32G RAM disk plus a 4GB RAM disk requires 36GB of free RAM. This is more than the available system RAM. Let’s remove the 32GB RAM disk with the –detach option.

sudo rapiddisk --detach rd1

Viewing the details again shows that rd1 has been removed.

sudo rapiddisk --list
a

Bye, bye rd1.

Dynamic Sizing

If you open System Monitor to view memory usage as you create the drives, you will notice that there is no RAM usage. This is because RapidDisk fills a RAM disk as it is needed. So, even though we create a 4GB RAM drive, no RAM is actually used until we write files to it. Then, System Monitor shows an increase in RAM usage.

Formatting the RAM Drive

The newly created RAM drive, rd0, does not appear in sudo fdisk -l or in GParted, but it still exists. The RapidDisk RAM drive, rd0, exists in /dev/rd0. If you have more RAM drives, such as rd1 and rd2, the you can access them at /dev/rd1 and /dev/rd2.

This is important because we must use /dev/rd0 (for rd0) in order to format the drive before mounting.

sudo mkfs.ext4 /dev/rd0

This formats the RAM drive as ext4, but you can format it using any supported file system that you wish.
By default, ext4 reserves 5% of the block space following a format. RAM is scarce, so let us disable this by setting the reserved blocks to 0. This RAM disk will be used for data, after all, and the reserved space is a waste in this case.

sudo tune2fs -m 0 /dev/rd0
a

Formatting a RAM disk and setting its reserved blocks to 0 for maximum available space.

Mounting the RAM Disk

With the RAM disk formatted, let’s mount it.

sudo mkdir /media/ramdrive
sudo mount /dev/rd0 /media/ramdrive

Any mount point you like is allowed just like any other device. I created a folder in /media/ramdrive since anything mounted in /media will automount in Nemo for easy access. As far as Linux is concerned, the RAM drive is a hard drive, so you can treat it like you would a hard drive.

a

4G ext4 RAM drive in Nemo.

At this point, the RAM drive is ready for use like any other hard drive. You can create directories, change ownership and permissions, and write and read files.

How Fast is It?

Using the Disks Benchmark utility, let’s check the speed of this RAM disk.

a

The 4GB RAM disk appears in Disks as /dev/rd0.

After running a 100x100M test three times, 3.1 GB/s read/write throughput remained consistent. This speed is correct for this test system that uses slow RAM.

a

3.2 GB/s reads and writes using this 4GB RAM disk. 3.2GB/s is the limit for the RAM in this test system.

Faster performance is expected on systems with faster RAM.

a

On another system with faster RAM and a faster CPU, RAM drive performance is greater. Here, we max out at 8.1-8.3 GB/s for RAM drive reads and writes.

Note that benchmarks will cause RapidDisk to use the RAM disk capacity, so you will see an increase in RAM usage in System Monitor. This is normal. The RAM disk will be limited to the size specified during creation, but if you find that you are needed more space, you can dynamically grow the size of the RAM drive with the –resize option.

Quick Review: RAM Disk Creation

sudo rapiddisk --attach 4096

sudo mkfs.ext4 /dev/rd0
sudo tune2fs -m 0 /dev/rd0
sudo mount /dev/rd0 /media/ramdrive

 

Bye, Bye RAM Disk Data

The RAM disk is as fast as your RAM will allow, but it will be destroyed and all contents lost when the computer is turned off. Bash scripts are your friend if you want to create a RAM drive upon system startup.

Archiving and Encrypting RAM Disk Contents

RapidDisk supports archiving and retrieval. You can save the current contents of the RAM disk to a file using the –archive option.

When the computer started up again, you can recover a RAM disk’s contents with the –restore option.
“Is it Worth Archiving and Restoring?”

In my opinion, no. It takes too long. From my tests, it took over 10 minutes to archive a 4GB RAM disk because the contents are also compressed to save hard drive space. If archiving before shutdown, then this increases the shutdown time. For larger RAM disks, the wait time will be longer.

Hard Drive Cache

Extending the RAM disk idea, we can map a RAM disk “between” a slow hard drive and the system to improve read performance. The RAM drive caches the most frequently used data. Once the data is in cache, by reading it for the first time, subsequent reads for the same data will be read from the much faster RAM.

a

RAM cache diagram.

With a RAM drive working, how do we use it as a cache for a slower hard drive?

Easy. The –cache-map option maps a RAM disk to a hard drive.

We will reuse rd0, the 4G RAM disk.

sudo umount /media/ramdrive (Should already be unmounted if benchmarked.)
sudo rapiddisk --list (To verify rd0.)

In this example, we will cache a slow 5400 RPM hard drive /dev/sdc. Here is the drive’s benchmark:

a

58.3 MB/s read, 49.8 MB/s write. Let’s see if a RapidDisk cache will improve anything.

Mapping

With the RAM disk created as rd0, we use the –cache-map option to map the RAM disk to the hard drive.

sudo rapiddisk --cache-map rd0 /dev/sdc1

If successful, you should see a message like this:

a

RAM disk mapped to /dev/sdc1 – a partition on /dev/sdc.

Note the new name of the cached hard drive: rc-wt_sdc1. This name might vary based upon the device name of your hard drive. sudo fdisk -l now shows this device as /dev/mapper/rc-wt_sdc1. We must use this in order to mount the hard drive.

This particular drive already contains a partition, /dev/sdc1. You can map partitions or entire drives.

Mounting the Cache

We do not mount the hard drive /dev/sdc. Instead, we mount its RAM disk cache /dev/mapper/rc-wt_sdc-part1 in the mount point /media/ramdrive.

sudo mount /dev/mapper/rc-wt_sdc1 /media/ramdrive

DO NOT FORMAT!

You can use a hard drive or USB device that is already formatted and already contains data, so there is no need to format. If you do, you will lose your data, so double-check your actions.

Of course, if the hard drive is unformatted, then you should format it before mapping it to a RAM drive.
We can use sudo rapiddisk –list to view the current status and see if the cache was mapped.

a

To access the hard drive via the cache, we use rc-wt_sdc1, not /dev/sdc1.

Benchmarking the RAM Cache

Does RapidDisk work? We use the Disks Benchmark again, but this time we benchmark /dev/mapper/rc-wt_sdc1 (the RAM cache for the hard drive), not the original drive /dev/sdc. Benchmarks should show a slight improvement upon second and third runs.

a

320 GB Hard Disk is the hard drive itself. /dev/rd0 is the RAM cache, and /dev/mapper/rc-wt_sdc1 is the RAM cache mapped to the hard drive. /dev/mapper/rc-wt_sdc1 is what we must benchmark.

100x10M Benchmark

a

Disks 100x10M benchmark. 1st, 2nd, 3rd comparisons.

200x1M Benchmark

a

Disks 200x1M benchmark. 1st, 2nd, 3rd comparisons. Smaller file sizes seem to result in faster read performance. Note: Same RAM cache was used, so first run is higher than real drive.

Data must first be cached into the RAM cache before we see any speed improvements. This is why the first run for both is lower. However, the 2nd and 3rd benchmark runs show an increase. The smaller file test shows the most improvement, but this is probably because of a combination of the RAM cache and the hard drive’s internal cache.

Also, the computer was not restarted for the 200x1M cache, so the 1st run results seem higher than normal. However, the 2nd and 3rd test runs are consistent with cached data.

Why is the Write Speed the Same?

Did you notice something about the benchmark results above? Read speeds improved, but the writes remained mostly the same.

This is normal.

RapidDisk only caches reads, not writes. if we look at the sudo rapiddisk –list result, we see that the RAM cache was set to WRITE THROUGH mode. Write through is one of the more common caching modes (the other is write back) that writes to the hard drive at the speed of the drive. This ensures that data is written to the physical media and helps prevent data loss.

For a RAM cache, this is what we want for reliability. Write see no improvement, but reads improve. When data is read from the hard drive, it is cached into the RAM cache. Upon subsequent reading of the same data, files for example, the data is retrieved from the faster RAM cache instead of the slower hard drive.

This offers an excellent read boost for files that you or multiple users access often. In real life, this translates to less waiting and makes your computer feel “snappier.” Of course, new data must be read at the speed of the hard drive.

If RAM speeds are 3.1 GB/s, why are these speeds so low?

In the RAM drive created earlier, we benchmarked ~3.1 GB/s for reads and writes. If the cache is using the same RAM on the same system, then why isn’t the cache operating at 3.1 GB/s? Why the ~190-700MB/s read speeds? This is slower than an SSD.

At this moment, I am not sure. For some reason, RapidDisk RAM caching always results in read speeds lower than what is possible with RAM. The results are still higher – much higher – than the drive’s speed limits, but the 3-8GB/s speeds never materialize. My guess for right now is that overhead or a misconfiguration somewhere is limiting the full speed potential.

Access times are almost nonexistent with a RAM cache, so this helps reduce wear and tear on a mechanical hard drive.

What About a Real File Test?

Using time, I tested how long it would take to transfer two Linux Mint ISO images totaling 3.7GB. Would caching improve the read speeds?

First, I copied them TO the drive mounted at /media/ramdrive.

a

~26 seconds to write the files to the drive. RAM cache is mapped.

Then, I benchmarked the RAM drive again using Disks in order to replace any cached data. A –flush option exists, but the man page warns against using it.

Next, I performed the actual test, which took ~40 seconds for the first read. The first read will usually be limited to the read speed of the hard drive, so its numbers should be the lowest.

Finally, I performed a second read and then a third.

a

Subsequent reads of the same files were faster and completed in less time than the first (top).

Caching an External USB Drive?

Is it possible to set up a RAM cache to improve the read speeds of a USB device or an external USB drive?

Yes.

The setup is the same. First, we will unmount and detach the existing RAM cache so we can

sudo umount /media/ramdrive
sudo rapiddisk --cache-unmap rc-wt_sdc1

Note that to unmap a RAM cache, we use the –cache-unmap option and the name of the cache device, which is rc-wt_sdc1 in this case, not /dev/mapper/rc-wt_sdc1.

a

Unmapping a RAM disk. “None” means no RAM disks are mapped to hard drives.

Using sudo fdisk -l, find the device of the USB drive. In this example, it is /dev/sdd1, another partition on a USB2 drive.

Next, we map the rd0 RAM cache to the hard drive like we did before. (If the USB drive automounts, be sure to unmount it first. You cannot map a cache to a mounted drive.)

sudo rapiddisk --cache-map rd0 /dev/sdd1

 

a

The RAM cache is mapped as /dev/mapper/rc-wt_sdd1

Now, mount the RAM cache.

sudo mount /dev/mapper/rc-wt_sdd1 /media/ramdrive

This drive already had two linux ISO images on it totaling 3.2GB, so I timed the copy to home.

a

~1m29s to ~14s is a good read speed improvement.

This is a 5400 RPM USB2 drive limited by a USB2 interface. Let’s run the Disks benchmark for testing.

Original 100x10M Result. No RAM Cache.

a

Original USB2 external drive. ~35MB/s read and write. This is the limit for a USB2 interface.

We must benchmark /dev/mapper/rc-wt_sdd1, which is the mapped RAM cache, not the actual drive /dev/sdd.

100x10M USB2

a

RAM drive effect on USB2 drive.

200x1M USB2

a

200x1M test on the same RAM cache.

The system was not restarted nor the cache cleared for the 200x1M benchmark. In all cases, subsequent reads were faster than the first read benchmark while writes remained the same. Again, RapidDisk does not cache writes. Only reads. This is normal operation.

For USB2, the read performance is an improvement. This can be applied to any USB device including USB sticks.

Can I Cache a RAID Array?

Yes. Simply set up the RAM cache in the same way. RapidDisk setup and operation is the same whether you use a USB stick, a hard drive, or a RAID array.

After setting up a test RAID array on a different system using faster RAM and a faster CPU, read speeds increased to 1.3GB/s over the native 140MB/s. Definitely an improvement.

Conclusion

If you have spare RAM and slow mechanical hard drives, then RapidDisk helps. Find yourself accessing the same files repeatedly? Are other users accessing 20% of the file 80% of the time? RapidDisk will improve Linux performance.

I like it. RapidDisk has proven its worth, and it is one of the extra tricks you can implement to give the illusion of a faster system without upgrading to expensive M.2 NVMe or high-capacity SSDs.

While I was hoping for cached read speeds equivalent to RAM speeds, it could be that my test system was at fault by using slower RAM, a slower CPU, and older parts. A quick test on a system with a faster CPU yielded faster cached read speeds, but they were still not up to native RAM speeds.

On the other hand, pure RAM disks do operate the the full RAM speed, so I am not quite certain where the bottleneck lies.

Overall, RapidDisk is a worthy improvement to any Linux system seeking improved read speeds from mechanical hard drives. And yes, RapidDisk works with SSDs too.

Have fun!

Advertisements

, ,

  1. Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: