Tag Archives: script

Simple Linux Performance Benchmarking

Recently I did some very simple benchmarking of the CPU and disk drives on a few of my Linux PC’s and a Raspberry Pi. This was a quick test to see how all of the machines compare.

CPU: I went for some very simple tests of the CPU performance under a load, calculating Pi. The code is not the best at calculating pi, it is just there to exercise the processor(s) and provide some standard piece of executable code that can be tried on multiple machines. When the pi code runs it will use the processor at close to 100% when viewed using top.

13502 erick     20   0  2784 1964 1112 R  98.8  0.4   0:36.13 arm_pi

There are programs available that are better at calculating a lot of digits of pi, fast. An example is Hyper Pi for Windows that will certainly calculate Pi to many more digits in a given period of time than the pi shown in this post program can.

DISKS: To test disk speeds. I used a simple writing and reading out of the bitbucket ( /dev/null) to disk and noting the speeds displayed.


CPU Benchmarking

CPU benchmarking was done by running an executable that calculates Pi to an arbitrary number of digits. Below is the link to the C file which is easily compiled for the target machine. I downloaded this from the net, from this page… http://www.overclockers.com/forums/archive/index.php/t-402437.html

pi.c

For a Linux machine or the Raspberry Pi, downloading the file and executing…

gcc -o pi pi.c

or

gcc -o pi pi.c -lm

As the original post states. The -l means link a library. The m means the math library. I don’t think it is necessary???

…will create the executable pi.

To run in the directory it was create just call pi with an argument of the number of digits to calculate, for example Pi to 1000 digits is calculated via…

./pi 1000

The ./ in front of the program name is needed if you are executing the program from the directory that you are in, or else Linux will go off searching through the path for programs named pi. If you have created a bin directory under your home folder you can put pi, or any executable code there and run it from other directories on the machine by just calling the program without the ./

Run of Pi program for 1000 digits

Approximation of PI to 1000 digits

3.1415926535  8979323846  2643383279  5028841971  6939937510  5820974944
5923078164  0628620899  8628034825  3421170679  8214808651  3282306647
0938446095  5058223172  5359408128  4811174502  8410270193  8521105559
6446229489  5493038196  4428810975  6659334461  2847564823  3786783165
2712019091  4564856692  3460348610  4543266482  1339360726  0249141273
7245870066  0631558817  4881520920  9628292540  9171536436  7892590360
0113305305  4882046652  1384146951  9415116094  3305727036  5759591953
0921861173  8193261179  3105118548  0744623799  6274956735  1885752724
8912279381  8301194912  9833673362  4406566430  8602139494  6395224737
1907021798  6094370277  0539217176  2931767523  8467481846  7669405132
0005681271  4526356082  7785771342  7577896091  7363717872  1468440901
2249534301  4654958537  1050792279  6892589235  4201995611  2129021960
8640344181  5981362977  4771309960  5187072113  4999999837  2978049951
0597317328  1609631859  5024459455  3469083026  4252230825  3344685035
2619311881  7101000313  7838752886  5875332083  8142061717  7669147303
5982534904  2875546873  1159562863  8823537875  9375195778  1857780532
1712268066  1300192787  6611195909  2164201989

Calculations Completed!
Time: 1 seconds


It is possible to run this program to see how the various machines that you own and compare the CPU performance by using the pi program.

Minimal Boot

More advanced benchmarking of a machine would involve trying to boot using a disk that would allow only the command line and a minimal amount of background stuff to load. I might try this at some point to see what difference it makes. I would try to boot from a Ultimate Boot CD (UBCD) and then go into the mode that loads the minimal Linux boot and somehow get pi.c loaded via USB stick maybe? It would be an experiment!

For more advanced testing that runs outside of the OS it is possible to run the code included on the UBCD, for example the  for a any machine that will boot from CD. The CD contains a suite of benchmarking, testing and stress testing tools, in addition to other tools for working with a hard drive and unlocking machines.

Pi Benchmarking Script

For benchmarking I close out of all applications that are running and open one terminal to execute the pi program.

Code

Below is code for a script file that will run the pi program,looping it multiple times, calculating the results for 1000,2000,4000,8000,16000,32000,64000,128000 and 256000 digits. I wanted to see if there would be any noticeable variations in the time to calculate various amounts of digits among the machines. I didn’t notice much of a deviation when I lined up plots of the various machines. They were all nominally nearly a constant multiple of speeds across the multiple levels of pi calculation

The script creates two temporary files cols.txt that lists the number of digits it has run up to. Additionally a file called results.txt captures the amount of time that it took to calculate the corresponding number of digits.

#! /bin/bash
rm cols.txt
rm results.txt

DIR=/home/erick/pi
x=1000


while [ $x -le 300000 ]; do

echo $x
echo $x >> cols.txt

$DIR/pi $x > temp.txt
tail -n 1 temp.txt >> results.txt

x=$(( $x * 2 ))

done;

Results of the Pi Benchmarking Script

Raspberry Pi Model B, single core CPU at 700MHz
Time: 1 seconds
Time: 1 seconds
Time: 5 seconds
Time: 21 seconds
Time: 86 seconds
Time: 352 seconds
Time: 1438 seconds
Time: 5835 seconds
Time: 23507 seconds
Dell Dimension 530-mt Xeon dual core processor at 2.4GHz

The machine has 1GB RAM and I don’t think that comes into play running this program. The pi program is only running on one of the cores in this example.

Time: 0 seconds
Time: 0 seconds
Time: 1 seconds
Time: 3 seconds
Time: 14 seconds
Time: 58 seconds
Time: 232 seconds
Time: 932 seconds
Time: 3741 seconds
Dell Dimension 2400: Pentium 4 single core at 2.4GHz

Nearly identical performance as the Dell Dimension. This machine has 1.5GB RAM.

Time: 0 seconds
Time: 0 seconds
Time: 1 seconds
Time: 4 seconds
Time: 13 seconds
Time: 56 seconds
Time: 223 seconds
Time: 898 seconds
Time: 3596 seconds

Suprise

I benchmarked two older laptops, an old Dell Inspiron (2003) running Ubuntu 10.04 and a not so old Toshiba Satellite A135 (2009), Mint 17 xfce, both 1.6GHz processors.

Pentium M on the Dell and Celeron M on the Toshiba. 333 and 533 MHz busses respectively. I thought that they would be fairly similar in performance but was pleasantly surprised to find the Toshiba was a decent amount faster than all of the other machines! This was a machine that had only 512MB RAM and ran Vista poorly and I almost scrapped it. Until I bought another stick of RAM and loaded Linux Mint 17 on it! It is like a miracle how much better it runs. It is a good test machine to try out Mint as I might consider it for a future desktop machine.

Dell Inspiron

Time: 0 seconds
Time: 0 seconds
Time: 1 seconds
Time: 3 seconds
Time: 13 seconds
Time: 54 seconds
Time: 276 seconds
Time: 1134 seconds
Time: 4526 seconds

Toshiba Satellite

Time: 0 seconds
Time: 0 seconds
Time: 0 seconds
Time: 2 seconds
Time: 7 seconds
Time: 30 seconds
Time: 164 seconds
Time: 692 seconds
Time: 2747 seconds

A ratio of 1.82 times faster, not bad at all.

 


 

Disk Benchmarking

Disk benchmarking was done by writing from the /dev/null bitbucket to disk, flushing caches and then reading back a 1GB file, discarding it into /dev/null.

Write Script

The follow code is copied into a script file ending in .sh and made executable using chmod +x filename, will write 1GB of zeros to a file named after the of=.

#!/bin/bash
dd if=/dev/zero of=/home/erick/testfile-1024x1M bs=1M count=1024

Read Script

The following code is copied into a script file ending in .sh and made executable using chmod +x filename. This script will read back the file created by the write script, dumping it into the null device. It will read it back in 8k blocks.

#!/bin/bash
dd if=/home/erick/testfile-1024x1M of=/dev/null bs=8k

Flush Caches

In order to test the disk if the RAM on the the machine is sufficiently large and the write operation leaves the written data it in the cache, you need to flush the cache in order to have the machine actually read it from disk. Linux is pretty clever about using RAM that is not doing anything, not already in use for programs and OS, for a disk read/write cache. I remember in the old days of DOS that there were a number of these utility programs that could be loaded that would use some RAM as a cache, effectively Linux does the same thing natively.

Take the following code, copy into a file ending in .sh, such as flush-caches.sh and make executable via the chmod +x filename command and then run it between writing and reading the disk.

#!/bin/bash
sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"

Example Results

These disk write and read utilities can be used to test harddrives, USB sticks, SD cards, RAM Drives (effectively tests RAM speed) and so forth. It even can be used to test the network speed when a drive is mounted using NFS ( or using rsync, scp or sftp ) as this will usually be the speed constraint and not the drive R/W speed.

Dell Dimension 530-mt Primary Hard Drive

Write

erick@Precision-WorkStation-530-MT:~/bin$ ./harddrive-write-test.sh
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 24.0374 s, 44.7 MB/s

Read

erick@Precision-WorkStation-530-MT:~/bin$ ./harddrive-read-test-8k.sh

131072+0 records in

131072+0 records out

1073741824 bytes (1.1 GB) copied, 25.3427 s, 42.4 MB/s

Toshiba Satellite A135

Write

erick@erick-Satellite-A135 ~/bin $ ./harddrive-write-test.sh
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 36.6714 s, 29.3 MB/s

Read

erick@erick-Satellite-A135 ~/bin $ ./harddrive-read-test-8k.sh
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 35.093 s, 30.6 MB/s

 

Old, circa 1998 machine, 4GB primary Hard Drive (/dev/sda) and 8GB Compact Flash card (/dev/sdb) Secondary drive

The interesting thing here besides just how slow the speeds are, is the fact that the CF card is actually faster than the hard drive. The drive is actually pretty loud on this machine as well. When CRON runs at the top of the hour, there is definitely a small burst of sound, enough to serve as a reminder of the time if you are in the same room as the machine. When you SSH into it, you can hear it grind away for about 2 seconds as it reconciles the password! I used this machine for remote monitoring before I had a Raspberry Pi, works good enough for that with it’s limited RAM and HD space. Now it is just a backup in case the Pi is down.

Disk /dev/sda: 4303 MB, 4303272960 bytes

Write: 1073741824 bytes (1.1 GB) copied, 144.733 s, 7.4 MB/s

Read: 1073741824 bytes (1.1 GB) copied, 138.498 s, 7.8 MB/s

Disk /dev/sdb: 8195 MB, 8195604480 bytes

Write: 1073741824 bytes (1.1 GB) copied, 91.4438 s, 11.7 MB/s

Read: 1073741824 bytes (1.1 GB) copied, 73.3132 s, 14.6 MB/s

Raspberry Pi SD Card

Using a shorter test via the following script to write…

#!/bin/bash
dd if=/dev/zero of=/home/erick/testfile-10000x8k bs=8k count=10000

and to read…

#!/bin/bash
dd if=/home/erick/testfile-10000x8k of=/dev/null bs=8k
Write To SD Card
erick@raspberrypi ~/bin $ ./sd-write-test.sh
10000+0 records in
10000+0 records out
81920000 bytes (82 MB) copied, 8.45663 s, 9.7 MB/s
Read Back from SD Card
erick@raspberrypi ~/bin $ ./sd-read-test-8k.sh
 10000+0 records in
 10000+0 records out
 81920000 bytes (82 MB) copied, 4.38749 s, 18.7 MB/s
RAM Drive Performance

tmp on my Raspberry Pi is set up as a RAM Drive (RAMFS) on my Raspberry Pi. So this gives some indication of how fast the RAM can be wrote and read to.

Using the following write script…

#!/bin/bash
dd if=/dev/zero of=/tmp/testfile-1000x8k bs=8k count=1000

and read script…

#!/bin/bash
dd if=/tmp/testfile-1000x8k of=/dev/null bs=8k
Write to RAMFS
erick@raspberrypi ~/bin $ ./ram-write-test.sh
1000+0 records in
1000+0 records out
8192000 bytes (8.2 MB) copied, 0.0458338 s, 179 MB/s
Read from RAMFS
erick@raspberrypi ~/bin $ ./ram-read-test-8k.sh
1000+0 records in
1000+0 records out
8192000 bytes (8.2 MB) copied, 0.0339184 s, 242 MB/s

 

 

end

Automating PHPList Process Queue

Frequently when using the non command line version of PHPList which I have loaded on my service providers server, I have to hit Process Queue multiple times for a message. I have read a lot of posts online complaining this issue. I have done some troubleshooting of it myself by turning on the verbose feature in PHPList and I am still at a loss as to what it gets stuck on and then stops processing the queue.

PHPList works great, I love it for all that it does and does well. My one complaint is the Process Queue issue and from reading a bunch of posts online I have successfully patched together a solution.

This code can be executed on the server where PHPList lives if they will let you run a wget command from a CRON entry, some won’t. In my case I opted to simply run the command from a Raspberry Pi server that I have that runs 24/7. It is simple a looping script that does the equivalent of hitting the “Process Queue” button on PHPList at regularly timed intervals and detected when the queue has been processed. I have seen several examples of using wget to be fired off every 5 minutes to do this, but that means it gets run every 5 minutes, all day, every day. Why not run something that can be scheduled for a time and then runs to completion which this script does.
This script also produces a log file that can be copied to a location viewable via the server so you can double check to see if all ran OK. I see this mostly as a debug feature and it could potentially be turned off along with some of the “echos” of text. Except for the one in the else-if, it is an error to have an empty else-if in a bash script.

PHPList Automatic Process Queue Script

#!/bin/bash
 # Controller Script for PHPlist Process Queue
 # Erick Clasen blog.oils-of-life.com 05/16/2015
# User defined variables
 # No trailing / on DIR!
 DIR=/home/erick/php-proc
 LOG=php-pq.log
 # Time to sleep between loop iterations.
 TIME=3m
 # Max number of loops, so the script breaks out if for some reason it fails to complete.
 LOOPS=100
 # Publish a copy of the log to the following directory
 PUBDIR=/home/erick/public_html
# wget having trouble with some options so simply switch to the dir we want it to work in.
 cd $DIR
x=1
 echo Start------------- >> $LOG
# While loop, breaks out if done or at LOOPS as a guard against #infinite loop.
 while [ $x -le $LOOPS ]; do
# Timestamp in DDMMYYYY_HHMM-SS (UTC)
 timestamp=$(date -u +"%d%m%Y_%H%M-%S")
echo $x $timestamp
 echo $x $timestamp >> $LOG
x=$(( $x + 1 ))
# Use wget to run process queue with login and password provided
 wget 'your_url_here.com/phpList/admin/?page=processqueue&login=phplist&password=your_password_here'
# Test to see if PHPlist process queue has completed.
 if grep -q "Finished, Nothing to do" $DIR/index.*
 then
 # Exit from Loop with a break, mark log,remove index files that wget fetched, break out.
 echo All Done! Breaking Out
 echo All Done!--------- >> $LOG
 echo ------------------ >> $LOG
 # Publish Log, Optional
 cp $LOG $PUBDIR/$LOG
 rm $DIR/index.*
 break
 else
 # Nothing Happens
 echo Keep Running...
 fi
# Sleep for TIME
 echo ------- Sleeping Process --------
sleep $TIME;
done;

PHPList Automated Process Bounces Script

Process Bounces can be run from a script as well…

#!/bin/bash
 # Controller Script for PHPlist Process Bounces
 # Erick Clasen blog.oils-of-life.com 05/16/2015
# User defined variables
 # No trailing / on DIR!
 DIR=/home/erick/php-proc
 LOG=php-pq.log
# Publish a copy of the log to the following directory
 PUBDIR=/home/erick/public_html
# wget having trouble with some options so simply switch to the dir we want it $
 cd $DIR
# Use wget to run process queue with login and password provided
 wget 'your_url_here.com/phpList/admin/?page=processbounces&login=phplist&password=your_password_here'
 echo ---------------------------- >> $LOG
 echo BOUNCES HAVE BEEN PROCESSED! >> $LOG
 echo ---------------------------- >> $LOG
 # Publish Log, Optional
 cp $LOG $PUBDIR/$LOG
 rm $DIR/index.*

CRON entry to run these scripts

These entries run Process Queue every Monday, Wednesday and Friday (1,3,5) at 4:15AM and Process Bounces at 12:15AM on the first day of the month. Nothing magic about those settings, you can set them any way you wish. I figure starting the process in the wee hours of the morning will allow people to receive the email by the time that they wake and first check their mail. I set the time to 15 after so it won’t run at the same time as some processes that I have set to run on the hour. Sending mail on weekdays keeps people from having to deal with more email than they might like on the weekends.

15 04 * * 1,3,5 /home/erick/php-proc/php-pq.sh
 15 00 1 * * /home/erick/php-proc/php-pb.sh

PHPList Automatic Process Queue Override

If you must make PHPList process the queue now and don’t want to wait for the CRON task. Two options are available, do it the via the normal way from the PHPList admin website or you can use something as simple as a CGI script to force the php-pq.sh script to execute. Below is an example of a dirt simple CGI script. It prints out the tail of the php-pq.log and runs the php-pq.sh script. Php-pq.log is the log of the previous automatic processing event and then goes ahead and fires off the Process Queue Override by running the script. Basically the act of loading the page makes it run. I keep my cgi-bin directory locked down so no one outside of a few IP addresses can get to the files there. But it would be fairly harmless if someone did, the worst that happens is that the queue gets processed.

CGI Code for Automatic Process Queue Override

#!/bin/bash
 echo "Content-type: text/html"
 echo ""
 echo "PHPList Automatic Process Queue Override"
 echo ""
 echo "
PHPList Automatic Process Queue Override from host $(hostname -s &)
"
 echo ""
 echo "
 $(tail /home/erick/php-proc/php-pq.log &)
"
 echo ""
 echo "
Information generated on $(date &)
"
 echo ""
 echo "Processing..."
echo "
 $(/home/erick/php-proc/php-pq.sh &)
"
echo "
"

 

PHP Config File Tweaks

I am working on taking some notes on some changes I have made to the PHPList config.php file and will be posting on this in August 2015.