Timing the file transfers for the Patriot Supersonic Boost XT USB stick in the previous article was performed by hand using two commands: one to measure writes, and another to measure reads.
Why not use a script instead to automate the entire process? Scripts sound good in theory, but they must be handled carefully or else the time results will be incorrect.
Automated scripts were originally used, but they made the times results trickier to understand. The core of the script (to measure writing time) relied upon a command like this:
(time cp -R pictures /media/PATRIOT) 2>&1 | grep 'real' >> write_time.txt
And like this:
(time cp -R pictures /media/PATRIOT) 2>&1 >&- | grep 'real' >> write_time.txt
(The second statement closes standard output after redirection. Both statements produced the same timing results, so closing standard output had no apparent effect.)
We are measuring the time it takes for the copy (cp) command to complete. The command within the parentheses does the actual writing to the Supersonic. The result from the time command is redirected to standard output using 2>&1 so it can be piped to grep for filtering.
The >&- closes the regular standard output as a precaution. We do not want any extra output to skew the timing by consuming extra processing. The >&- is not required, but if using it, make sure to place it after the 2>&1 so it looks like 2>&1 >&-. Using >&- 2>&1 closes standard output (1) first so there is no way to redirect standard error (2) to it.
Grep is important because the time command produces three results for time: real, user, and sys. We only want the real time, so grep extracts that line and appends it to an output file for later reference.
The problem is that when this command is placed in loop, all three iterations are performed and recorded within a single time result. It’s not a major issue. Just remember to convert the real time into seconds and divide by three to get the time for a single iteration. For example, in one of the original tests, the result for all three iterations (writes) was “real 6m17.265s” and the rest “real 0m0s.” Still a speedy USB device. To find the time for a single write, round off and convert to seconds (6m * 60 + 17s = 377s), divide by 3 (377s / 3 = ~127s per iteration), and convert back to minutes (127 / 60 = ~2m7s per iteration).
However, this only divides a single number and does not let us see the individual write times with their variations for averaging. This is important because, for reasons unexplained, benchmarking with a script produced wildly varied and inaccurate results. Some writes took only 1m to complete, while the same write would take over 6 minutes the next time.
Even removing the loop and running the statements one by one in a script produced drastically different times that were practically unusable.
(time cp * "$dest") 2>&1 >&- | grep 'real' >> "$out" (time cp * "$dest") 2>&1 >&- | grep 'real' >> "$out" (time cp * "$dest") 2>&1 >&- | grep 'real' >> "$out"
First run result:
real 1m46.248s real 6m33.719s real 8m25.399s
Second run result:
real 5m45.629s real 1m48.527s real 1m51.772s
These numbers were practically useless due to their wide differences.
There may have been background processes running unaware, but who knows? Maybe unexpected caching was affecting the process somewhere? The end result is that using a script became more hassle than it was worth. More time was being spent resolving these and similar issues and maintaining the script than actually measuring the read and write times of the Supersonic.
In the end, the only command that accurately measured the write time to the Supersonic was this:
time cp -R pictures /media/PATRIOT
And for measuring read times:
time cp -R /media/PATRIOT/pictures ~/test
Running both statements from a terminal produced the most accurate timings.
Here is an example of one of many scripts tried. It only measures write times to the Supersonic by copying all files to the USB stick. Loops are deliberately avoided in this script.
#!/bin/bash dest=/media/PATRIOT/temp out=~/write_time.txt # Remove write file if it exists if [ -f "$out" ] then rm "$out" fi # Create temp dir on USB if [ ! -d "$dest" ] then mkdir "$dest" fi # Script is located in dir above the picture dir to copy cd pictures # Main. Three times. echo 'Write test 1' (time cp * "$dest") 2>&1 >&- | grep 'real' >> "$out" rm -rf "$dest" && mkdir "$dest" echo 'Write test 2' (time cp * "$dest") 2>&1 >&- | grep 'real' >> "$out" rm -rf "$dest" && mkdir "$dest" echo 'Write test 3' (time cp * "$dest") 2>&1 >&- | grep 'real' >> "$out"
The timing results varied as widely (1m to 8m per write in random order) as in running the individual redirection commands with grep shown earlier, and each script execution produced greatly different results totaling anywhere from ~7m to ~16m or more for three consecutive writes. While this could most likely be resolved, the point was to measure read and write times in the least amount of time possible, and too much time was being spent on script maintenance.
Conclusion and Extra Notes
When using scripts to record timings such as this, always remember to record the time of the command performing the work and not the entire script. Place the time command inside the script where it needs to be. Do not run time ./script.sh. This might seem obvious, but it can be easy to overlook when attention is on something else.
Parentheses and redirection are needed in order to redirect the output of the time command for processing.
Benchmark scripts are more convenient but less accurate than timing the commands manually in a terminal. Less can go wrong when timing a single command, and less time is spent debugging a script. In all, about three times the effort was placed in writing, debugging, and eliminating timing discrepancies involving the script than in timing the read and write speeds of the Supersonic.
As a final check to make sure the single commands were approximately correct, the same files were copied to and from the Supersonic with a drag and drop operation and recorded with a stopwatch along with observing the speed reported in the file transfer dialog. This produced the most real-life timing, which was the goal, and these timings remained consistent with the single, one-by-one commands executed from the terminal.
Overall, a fun experiment!
Related article on which this article is based on: