Tag Archives: Raspberry Pi

CPU Temperature Monitoring on the Raspberry Pi

One of my thoughts for using the Raspberry Pi is to monitor ambient temperature in my house and send me warnings if it is too low. This would be helpful when away from home in the winter and there is an issue with the heat that I might need to know about fairly quickly.

Eventually I might consider building in a weather station capability into the Raspberry Pi by using both an indoor temperature, humidity and barometric sensor and an outdoor temperature and humidity sensor. So with that in mind I also want to make some simple graphs of data every hour or so. Something that can be just saved into a text file and requires no extra graphing code on the client or the server.

CPU Temperature Experiment

I started out by experimenting with taking reading of the CPU temperature of the Raspberry Pi in order to get a little practice with some live data while I was awaiting the arrival of a Bosch BME280 ambient temp., humidity and barometric pressure sensor.

sSMTP

I plan on using sSMTP to send me warnings, such as when my ambient temperature falls below a critical value. I have already worked up some code to monitor the CPU temperature and email me if it goes out of a specific range, to get a bit of practice with coding this into a script. Plus it was a good way to try out sSMTP. sSMTP will be covered in more detail in a separate post.

Snippet of script that sends email if the upper and lower CPU temp limits are exceeded

The temperature read by the cat /sys/class/thermal/thermal_zone0/temp is in milliCelsius, so using cut on it will grab whole degree values. The rest of the code is pretty straightforward. The variables minimum and maximum are setup to be the appropriate values in whole degrees Celsius. mailaddr is exactly what it sounds like. What is nice is that it just gives you a very simple method of sending a basic email warning when the temperatures get out of bounds.

# Read the temp and cut it to grab leftmost 2 characters, integer Temp
temp="`cat /sys/class/thermal/thermal_zone0/temp | cut -c1-2`"

#echo $temp

# Mail if about or below the limits
if (( $temp > $maximum )); then
   #echo "above"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr
elif (( $temp < $minimum )); then
   #echo "below"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr

fi

 


Simple Graphical Temperature Logging

To just monitor the CPU temperature on a hourly basis and have a simple graphical representation of it I had to do some digging online.

I dug around the web and found a bit of Perl code to make a simple ASCII graph of the temperatures that I was measuring for the CPU. I wanted to be able to simply post to the web a running capture of temperatures, grabbed hourly by using CRON. I did not want to have to rely on graphing tools either on the Raspberry Pi or the client computer to interpret data. I spent a while searching for something simple and this was about as easy as it can get. I found a one line, long line! Method for creating a simple graph using Perl. I looked around a lot on line and wanted to avoid anything that would have to be installed on either the Raspberry Pi or the client machine to be able to plot data. So after a long search this piece of Perl code is about as simple as it gets and will plot and ASCII based graph across one line each time it executes.

The key variables in the code are min, max and w. The variable w controls the width of the plot in characters that it will be allowed to reach when the input value is at max. The variables min and max control the minimum and maximum input values expected and scale the graph accordingly. PLOTTABLE_FILE is the path/filename of the file that the output will be sent to.

 PLOTTABLE_FILE=/var/www/tmp/cputemp-milli-celsius.txt

Perl Code Snippet

The code resides in a script in the /etc/cron.hourly directory and produces a file that is readable via the web, with a new entry every hour. The filename is cputemp, not the lack of a .sh extension. This is important when you create scripts that will reside in the “CRON” folders. It also has to be made executable by using sudo chmod +x filename.

(cat /sys/class/thermal/thermal_zone0/temp ) | perl -ne '$min=20000; $max=65000; $w=79; use POSIX; $d=ceil(log($max)/log(10)); $w-=$d; $v=$_<$min?0:$_>$max?$max:$_; $s=$w*$v/($max-$min); $bar=join("", ("*")x$s); $bar=~s/.$/|/ if $v==$max; print sprintf("%${d}d ",$_)."$bar\n";' >> $PLOTTABLE_FILE

Sample Output

Below of a capture of a few lines of output from the text file that is generated by the Perl code. It does not look as clean as it would by looking at the text file. So I have a capture of that here…. cputemp-milli-celsius

42236 *********************************************************************
41160 *******************************************************************
41160 *******************************************************************
41160 *******************************************************************
40084 *****************************************************************
41160 *******************************************************************
42774 **********************************************************************
44388 ************************************************************************
45464 **************************************************************************
45464 **************************************************************************
46002 ***************************************************************************

Alternate Method to Read CPU Temperture for the Raspberry Pi

There is another way to read the CPU in a more human readable form. Executing the line below…

/opt/vc/bin/vcgencmd measure_temp

…will show a result like this…

temp=45.5'C

Resources

Source of Perl Script

 

Raspberry Pi

Reduce writes to the Raspberry Pi SD card

After 5 months of solid up-time for my Raspberry Pi server, which has been running great. It has been taking a picture every hour and from them creating a creating a timelapse video every day. Also it is being used as a place to drop files to periodically from other place on the network, a little bit of file storage. Eventually I will add more storage space to it to use it even more for network storage.

Recently, I started to think about the potential wear of the SD card as I came across several articles online dealing with the topic. I decided to make a few changes to the Raspberry Pi configuration to reduce the amount of writing to the SD card.

Write Saving #1: Using a tmpfs

I editted /etc/default/tmpfs. In it the comments state  that /run, /run/lock and /run/shm are already mounted as tmpfs on the Pi by default. Which I have observed. This was a change made a while ago for the Pi according to the buzz online. I additionally set RAMTMP=Yes to add /tmp to the directories put on the tmpfs. This sets up access to /tmp with rwx-rwx-rwx permissions. There was a suggestion that I saw online to limit the sizes of the various directories, I added that as well.

# These were recommended by http://raspberrypi.stackexchange.com/questions/169/how-can-i-extend-the-life-of-my-sd-card
# 07262015, mods for using less of the SD card, RAM optimization.
TMPFS_SIZE=10%VM
RUN_SIZE=10M
LOCK_SIZE=5M
SHM_SIZE=10M
TMP_SIZE=25M

The OS and some programs will use /tmp. But so do I. I created a /tmp/web folder under it when the Raspberry Pi boots. Into this folder files go such as the hourly photo and the daily video that scripts create for the webcam that is attached. I have reduced 3 hourly writes to just one photo. I keep only one on the SD card as I don’t want to risk losing a bunch of them taken during the day if I totally relied on the tmpfs. If I was using a UPS, I would have no problem saving all of them on the tmpfs and occasionally backing up to the SD or another device. The big saver is the daily timelapse.avi for the web that is created daily from all of the hourly captured photos. It is many megs in size gets written daily and it doesn’t matter if I lose it. It can be recreated from the photos at will. So it is the perfect kind of file to throw on a RAM file system.

I also store the hourly and daily logs that I create using the cron driven logcreate script that I run. The logcreate script creates an hourly log that is concatenated into a daily log on the tmpfs then every day the daily log will concatenate into a full log, that is rotated, on the SD card, so I have a permanent record. Need to put the link for this here!

What is a tmpfs?

It is a RAM Disk, a.k.a. RAM Drive that allows RAM to be used as a hard drive. Obviously when the power goes out, it goes away. So we don’t want anything important to go there. But for things like files that I make such as the hourly photos that my Web Cam takes and the video it makes daily and logs, it is perfectly fine for usage. It is not a really big deal if the power went out and I lost this information as it will be recreated shortly anyways.

Caution

The only issue that I see with having logs on a tmpfs would be a situation where the Pi got in a state of weirdness where it started rebooting itself and then you had no logs to track down the problem. Then I suppose, it would be just a matter of changing the /etc/fstab file to revert to putting the logs back onto the SD card for a while to track down the problem. But, for a Raspberry Pi like mine that is running stable and I am not doing many experiments with right now, having the logs in volatile memory is not something I worry about. Plus it is easy to make a script to backup the logs to the SD card or another computer, if you manually reboot it, so you can save them if you like when you have control of the reboots.

Write Saving #2: Turning off swap

If the Raspberry Pi runs out of RAM, not likely if it is a server set up for light duty usage, it will start to use swap which is on the SD card, causing writes to the swap file. Mine rarely touches swap. I would rather tune the thing for better memory use than have it use swap.

It is possible to turn off swap usage using the command…

sudo dphys-swapfile swapoff

This is not persistant and needs to be done on every boot. It could be put into the root crontab by editting it using sudo crontab -e and adding the line. Or creating a script for it along with other items that are to be run at startup.

@boot dphys-swapfile swapoff

Online, people said that there was another way to turn it off by reducing the swap file size to zero, a config file for swap, can’t remember the name. But it is claimed that when it reboots it just overrides that and makes a default 100M swap file.

Write Saving #3: Moving /var/log to a tmpfs

One of the biggest offenders as far as writing to files periodically is the logs that live under /var/log and it’s sub-directories. You can create an entry in /etc/fstab that will create a tmpfs for /var/log. The only caution here is daemons, like Apache that require a directory to exist under /var/log or else they will not start. Apt also has a directory under /var/log, but it creates itself when apt runs for the first time so that is no problem. The apt directory has logs that keep track of what apt installs or uninstalls, good info to know about. News seems to work fine creating a directory for itself too. So for me only Apache is a problem.

  1.  Put an entry in /etc/fstab…
     tmpfs /var/log tmpfs defaults,noatime,mode=0755 0 0
  2.  Found out that news and apt folders create themselves when these things run.
  3. Apache is the one thing that does not like a missing folder so made a Kludge for now using ~/bin/setup-tmp.sh where I create /var/log/apache2 and chmod it 750. Then I restart apache using apachehup.sh, which just restarts it. Apache was failing to load when I pointed the log dir to /tmp in /etc/apache2/envvars under the export APACHE_LOG_DIR directive.

Write Saving #4: noatime

As you can see above one of the options used in the /etc/fstab file is the noatime option. By default the Raspberry Pi uses this option for the mount of the SD card. If you add mount points of your own to the card, make sure noatime is used. Without it Linux makes a small write each time a file is read to keep track of when it was last accessed, this obviously causes writes. It is possible to use it for the writes to the tmpfs as I am doing above. It saves a bit of time as the system does not have to do a write when a file is just being read.

Another good use of noatime is for drives connected across the network. For example on NFS mounts noatime is a really good choice. The network is generally slower than devices attached to a PC and having to send a write across every time a file is read, slows things down a bit when moving many files.

 


Been running this setup with the RAM savings for a few months now with no problems. I hardly ever see the ACT light blinking on the Pi anymore.

 

The LEDs have the following meanings :

  • ACT – D5 (Green) – SD Card Access
  • PWR – D6 (Red) – 3.3 V Power is present
  • FDX – D7 (Green) – Full Duplex (LAN) connected
  • LNK – D8(Green) – Link/Activity (LAN)
  • 100 – D9(Yellow) – 100Mbit (LAN) connected

From http://www.raspberrypi-spy.co.uk/2013/02/raspberry-pi-status-leds-explained/

FTP on Raspberry Pi. An easy way to make shared folders

The idea with FTP is to have folders that can be reachable between Linux and Windows, locally and remotely and easily. FTP is not secure, but it can be made secure, that info can be found on the web. For now I am covering the basics of FTP here.

For most things that I need to do, I don’t need the files to be secure anyways, 90% of the time nothing critical is going back and forth across remotely. If it is I would use a secure method of sending files via SSH via SFTP or an SSHFS.

FTP is an old protocol but it just plain works and is compatible with Windows, Linux and Mac. I have tried WebDAV in the past but it is compatible to only a degree with various Windows operating systems. I have had a hard time getting it working correctly on versions of Windows beyond XP, resorting in installing patches to Windows and etc. Generally not easy to implement.

I was also looking at FTP as a native tool typical of server installs. I have experimented with cloud setups such as OwnCloud and Sparkleshare, but with FTP I was looking for something simple and quick to setup, no special software, no mySQL databases running on the Raspberry Pi, no special software on client PCs, that sort of thing.

vsFTP

sudo apt-get install vsftpd

Edit the configuration file

Back it up first then do an edit.

sudo cp /etc/vsftpd.conf /etc/vsftpd.orig
sudo nano /etc/vsftpd.conf

uncomment local_enable = YES

uncomment write_enable = YES

Find this and check that it is set this way…

local_umask=022

Enabling PASV

I have read online that enabling the PASV capability for FTP is a good idea. Frequently when I have FTP’d to various ISP’s sites I have seen them operate in PASV mode. So it stands to reason that if the pro’s are have it set up that way it may have it’s advantages.

Add the following lines to the /etc/vsftp.conf file.

pasv_enable= Yes
pasv_min_port=40000
pasv_max_port=40100

There is nothing magic about the numbers of the port range other than they should be unused by anything else that your setup might require and generally I have seen high numbers used commonly. To work out side of your local network you must enable port forwarding of the range of port numbers through your router configuration.

Changes to vsFTP

With the newer versions of vsFTP there is a change that has occurred since I wrote my previous post about vsFTP (  http://oils-of-life.com/blog/linux/server/additional-utilities-for-a-linux-server/ )

The change has to do with the fact that the root directory of the user has to be non-writable and I have read online that it is best to make it owned by root as well. This is covered below, after the section on adding a user. You need to have a user first before modifying their permissions!

FTP User

To create an FTP user, create it in a way that it does not have a login shell. So that someone who can log in to the FTP account can’t execute shell commands. The line /sbin/nologin may not be in the /etc/shell file and in that case it needs to be added in there. The user basically has to be jailed in their directory and has to have no login shell.

sudo useradd -m -s /sbin/nologin -d /home/user user

I added Documents, public_html directories to the /home/user as well. Then made the users root folder /home/user, owned by root and nonwritable.

cd /home/user
chown user:user Documents
chown user:user public_html

chown root:root /home/user
Make Root of user non writable
sudo chmod a-w /home/user



FTPing on the PC

Now that ftp is set up on the server you will want to be able to connect to it!

Options for connecting…

Command Line, WIndows and Linux

ftp yoursite.com

That gets you into FTP via the command line. The command prompt will now start with ftp> ,that is how you know that you are within the ftp command shell.

It is archaic, but worth knowing when you have to stick a file up or pull it down right at the command line. The commands the ftp prompt accepts are basic, but good enough to get most work done. Type help at the prompt to get a list of commands.

Via Folders

Linux

Just enter the location of the ftp server right into the top of the directory folder and you will be prompted for a password and taken there.

Windows
Windows7/Vista:
  1. Open Computer by clicking the “Start” button, and then clicking Computer.
  2. Right-click anywhere in the folder, and then click Add a Network Location.
  3. In the wizard, select Choose a custom network location, and then click Next.
  4. To use a name and password, clear the Log on anonymously check box.

From: https://www.google.com/search?q=connect+to+ftp+windows+7&ie=utf-8&oe=utf-8

 

 

Kombucha with SCOBY

Kombucha SCOBY Timelapse Video

I ran a webcam pointed at a fermentation of Kombucha capturing frames using fswebcam running on a Raspberry Pi server. It ran for a few weeks.

You can see the Kombucha SCOBY’s forming in the video. The setup is behind the PC monitor with a desk lamp and camera pointing towards the back of the desk. This gives fairly steady ambient lighting. I grew a dozen SCOBY’s for the Summer Fermentation Class that was held on June 29th 2015.

SCOBY Timelapse Video AVI Format 8 frames/sec

The video should be viewable most of the time unless I am servicing the Raspberry Pi or I pulled the plug on it during a nasty storm.

Tux Favicons

While I was setting up and testing one of my servers I noticed that when I access it using Opera on a Kindle, Opera tries to grab a favicon.ico file, which was missing. I noticed the file was missing when I was browsing through the Apache error log on the server. In the log there is a missing file complaint about the favicon.ico whenever I browsed it using Opera.

So I found some graphics of Tux the Penguin and the Raspberry Pi raspberry and made up some quick favicons for the web pages on the Raspberry Pi Server and my main server.

I have loaded them up here in large format and the favicons as well. All the Tux pics come from Linux 2.0 Penguins, see this page for the original art. I used GIMP to edit them which worked well.

Two 300 x 300 pixel examples of the Favicons

Tux Server Favicon
Tux Server Favicon
Tux Raspberry Pi Favicon
Tux Raspberry Pi Favicon

 

Favicon ICO Files

http://oils-of-life.com/blog/wp-content/uploads/2015/07/tux-raspberry.ico

http://oils-of-life.com/blog/wp-content/uploads/2015/07/tux-favicon-32.ico

http://oils-of-life.com/blog/wp-content/uploads/2015/07/tux-favicon-16.ico

http://oils-of-life.com/blog/wp-content/uploads/2015/07/tux-shine-server-big-S-32.ico

Automatic Server Status Page Creation Update

In January 2015 I created a post about automatically creating a status page for a Linux server that I have. Typically this is put under a restricted directory and allows you to see a snapshot of what is happening with the server. I run it by putting the scripts in the /etc/cron.hourly directory on a Linux PC and a Raspberry Pi running Linux.

It serves as a simple way to check up on the server without having to use a tool such as Webmin that requires a login. It also keeps a trail of log files that get rotated on a monthly basis, so there is always a few old ones around to track down any problems and patterns in the operation.

I have found this information useful when I have traced down malfunctions that can occur when setting up a server and also when I was trying to get a webcam up and running and had the USB bus hang up a few times when the cam was overloaded with too much light.

In the new script file I fixed a bug by adding parenthesis around a line that I was trying to echo and I added code to run the w command to show a quick picture on who is logged in, how long the server has been up and running and the values for the average load on the server at the 1, 5 and 15 minute marks.

Logcreate Script

#!/bin/dash
# Remove old log
rm /var/www/status/log.txt
# Print logged outputs into new log.txt
date >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
tail /var/log/syslog >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
free >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
df -h >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> /var/www/status/log.txt
ps aux | sort -nrk 4 | head >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Logged in User info using w command
w >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Copy log.txt into the full log that is collected
cat /var/www/status/log.txt >> /var/www/status/fulllog.txt
# Create a free standind copy of the process tree
pstree > /var/www/status/pstree.txt

Alternate Version

I also created a version of the script for a desktop Linux PC that does not have Apache installed.  In it I use a DIR variable to contain the directory that I want the log.txt file stored.

 #!/bin/dash

# User defined variables
# No trailing / on DIR!
DIR=/home/erick/status

# Remove old log
rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt
ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/fulllog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

Rotation of Log

In the /etc/cron.monthly directory I have created a file that is called status-log-rotate and it will save backup copies of 2 months worth of the full concatenated server status logs.

#! /bin/bash
DIR=/home/erick/status
mv $DIR/fulllog.txt.1 $DIR/fulllog.txt.2
mv $DIR/fulllog.txt $DIR/fulllog.txt.1

Tweaks for Raspberry Pi

For the Raspberry Pi which has an SD card that I am trying to be conscious of writing to often. I have recently made some modifications to put the /tmp folder onto RAM using tmpfs. I create the hourly log underneath a folder there. Daily via a script it cron.hourly it gets concatenated into a daily log which is under a status folder that has restricted access. This gets appended once per day to the fulllog which actually lives on the SD card. The end result, no multiple hourly writes to the log file, just one append to the full log per day. The only downside is if the power drops and then some log entries will be lost for the day.

Logcreate runs from /etc/cron.hourly for Raspberry Pi

#!/bin/dash
# Set DIR, on Pi this is a temp location for log
DIR=/tmp/web

# Set fixed DIR FIXDIR for files that have to be stored on SD card
# Nevermind, just make a daily log and then copy that to the full log daily.
#FIXDIR=/var/www/status

# Remove old log

rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt

ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/dailylog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

dailylog-to-fulllog script, runs from /etc/cron.daily

#! /bin/bash

DIR=/tmp/web
FIXDIR=/var/www/status

echo "----------------------------------------------" >> $DIR/dailylog.txt
date >> $DIR/dailylog.txt
echo "----------------------------------------------" >> $DIR/dailylog.txt
cat $DIR/dailylog.txt >> $FIXDIR/fulllog.txt
rm $DIR/dailylog.txt

Logcreate Output from Raspberry Pi

Below is what the logcreate script will output to the log.txt file on a Raspberry Pi that I have running as a web server.

Sun Jul 12 14:17:01 EDT 2015

Jul 12 13:47:51 raspberrypi dhclient: DHCPACK from 192.168.1.1
Jul 12 13:47:52 raspberrypi dhclient: bound to 192.168.1.17 -- renewal in 40673 seconds.
Jul 12 13:59:01 raspberrypi /USR/SBIN/CRON[28010]: (erick) CMD (aplay /opt/sonic-pi/etc/samples/guit_e_fifths.wav)
Jul 12 13:59:07 raspberrypi /USR/SBIN/CRON[28009]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:00:01 raspberrypi /USR/SBIN/CRON[28013]: (erick) CMD (/home/erick/fswebcam/cron-timelapse.sh >> timelapse.log)
Jul 12 14:00:23 raspberrypi /USR/SBIN/CRON[28012]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:01:01 raspberrypi /USR/SBIN/CRON[28022]: (root) CMD (/home/erick/bin/usbreset /dev/bus/usb/001/004)
Jul 12 14:01:02 raspberrypi /USR/SBIN/CRON[28021]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:09:01 raspberrypi /USR/SBIN/CRON[28053]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Jul 12 14:17:01 raspberrypi /USR/SBIN/CRON[28064]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

             total       used       free     shared    buffers     cached
Mem:        445804     424488      21316          0     106768     260516
-/+ buffers/cache:      57204     388600
Swap:       102396          0     102396

Filesystem      Size  Used Avail Use% Mounted on
rootfs          6.3G  3.1G  3.0G  51% /
/dev/root       6.3G  3.1G  3.0G  51% /
devtmpfs        214M     0  214M   0% /dev
tmpfs            44M  240K   44M   1% /run
tmpfs           5.0M  8.0K  5.0M   1% /run/lock
tmpfs            88M     0   88M   0% /run/shm
/dev/mmcblk0p5   60M   19M   41M  32% /boot

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      2071  0.0  3.0  24896 13652 ?        Ss   Jun28   2:24 /usr/sbin/apache2 -k start
www-data 27745  0.0  1.5  25412  7084 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27744  0.0  1.5  24960  6760 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27743  0.0  1.5  25428  7116 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27742  0.0  1.5  25396  7036 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27538  0.0  1.5  25396  7032 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27502  0.0  1.5  25404  7036 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27501  0.0  1.5  25396  7044 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27747  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27746  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start

 14:17:02 up 14 days, 12:56,  1 user,  load average: 0.00, 0.01, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
erick    pts/0    192.168.1.5      14:04   10:39   1.70s  1.70s -bash

Wake on LAN for PC’s via Raspberry Pi

I found a nice PHP script for Wake on LAN. I loaded it on the Raspberry Pi that I have and configured it for my system. The Raspberry Pi runs 24/7, so I can just navigate to a web page that it serves, hit a button and start up one of my machines at home from anywhere. Mostly this is useful for the starting my Linux file server remotely but I do use it to fire off the desktop too.

Right from the read me file for the code…

REMOTE WAKE/SLEEP-ON-LAN SERVER
=========================
This is simple webapp that runs on your Raspberry Pi to turn it into a remotely accessible Wake/Sleep-On-LAN Server. [Follow the detailed tutorial](http://www.jeremyblum.com/2013/07/14/rpi-wol-server/) on my website for instructions on how to get this working, and forwarded through a router. This is very useful when you have high-powered machine that you don’t want to keep on all the time, but that you want to keep remotely accessible for Remote Desktop, SSH, FTP, etc.

http://www.jeremyblum.com/2013/07/14/rpi-wol-server/

Results

It is rare when something works right out of the box. But, this did, I followed Jeremy Blum’s instructions and within a few minutes I had this working. It has a nice drop-down menu where you can select a computer. It pings it to see if it is awake, then you can wake it from anywhere in the world. Once the WOL packet is sent, the application keeps pinging the PC at a defined interval and you can see when it wakes. I have not tried the sleep functionality as I am using it with Linux PC’s and his outline covers Windows machines. I am sure the code could me modified to shut down a Linux PC somehow. Perhaps it automatically SSH’s in and sends a shutdown command, something like that. I have my Linux server set to shutdown automatically so I don’t need this functionality myself.

It is configurable through an easily understood config.php file as well. You can set the computers name and IP address, MAC address, timing between pings, amount of times to ping the machine and etc.

Also see on this site…

Original Wake on LAN via Ubuntu Linux Post

Windows Wake on LAN Post

Raspberry Pi

Network File System (NFS)

For a while I have been using Samba to remotely connect Windows computers to my Linux computers and one Linux file server. I can even connect my Linux laptop to my Linux server via Samba. But, recently I bought a Raspberry Pi and I got interested in using NFS for three reasons.

  1. The Raspberry Pi will be running 24/7 and I would like the option to automount the home folder and others from it on my Linux computers when I start them up.
  2. I would like to mount some folders on a server (“big” file storage Linux server) that I can start remotely to the Raspberry Pi so that it will act like expanded storage on the Raspberry Pi. Then I can start the “big” server remotely and mount the folders on the Pi and use the Pi as a proxy. So it is connected to the web and I can navigate to folders that are on the big server via a connection to the Pi with NFS mounted directories.
  3. It would make it easy to backup Linux machines, including the Pi to the big server periodically. Years ago I thought about NFS for mounting folders for backup but I was pretty happy using a scripted FTP system for backups, so I shelved implementing NFS mounts back then.

Implementing NFS was a lot easier than I thought it would be. It was actually much easier than getting Samba to work the way I wanted it to.

The first step ( in my opinion) is to have the machines that you will mount directories from and to on static IP address. On a home network it really does make it easier to have all the machines other than guest machines on static IP’s. This can be done either by setting the machine to have a static IP. Or it may be possible depending on the router, for the router to be configured to hand out the same IP address to a machine with a specific MAC ID. Effectively the results are the same.

Static IP’s are useful as the actual IP addresses will be listed in the export file. It may be possible to use names, however this depends on how DNS is handled on your network. Using the actual IP addresses will make initial setup nearly foolproof. Also an easy way to use names on any machine is to add the static IP’s and names of the machines on the network to the /etc/hosts file.

Install NFS Support

To install support for NFS on the machines run….

sudo apt-get install nfs-kernel-server

Exports File on Server

Modify the servers /etc/exports file to suit your needs. Below is an example from my system on the Raspberry Pi. Remember to restart the NFS server when you have made changes to the file….

sudo service nfs-kernel-server restart
/etc/exports
# /etc/exports: the access control list for filesystems which may be exported
 #        to NFS clients.  See exports(5).
 #
 # Example for NFSv2 and NFSv3:
 # /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
 #
 # Example for NFSv4:
 # /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
 # /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
 #
 /srv/homes       192.168.1.9(rw,sync,no_subtree_check)
 /home/erick      192.168.1.0/24(rw,all_squash,anonuid=1001,anongid=1004,no_subtree_check)
 /         192.168.1.9(rw,no_root_squash,anonuid=1001,anongid=1004,no_subtree_check)

My initial try at this was to semi follow an example and create a /srv/homes directory and export it to one machine at 192.168.1.9. rw = read write access, sync = change both folders on server and client to keep sync’d, no_subtree_check = keeps the machine from having to check consistency of file names, prevents problems if a file is open and the file is renamed.

Then I decided to export my own home directory to all of the machines on the LAN, using 192.168.1.0/24 which allows access from 192.168.1.1-192.168.1.255. This time I am using all_squash which maps all UID’s and GID’s to nobody and nogroup, then setting anonuid=1001, my UID on the Rasp Pi and anongid=1004 my group on the Rasp Pi, they will map over to the correct UID and GID for myself on the other machines. Therefore I have no problem with read write access as the same user on the other machine to the NFS drive.

The next line exports the entire file system to one machine, but has no_root_squash set, which allows the root to access and create files as root on the server. This is one to be careful with, I use it only when I need to mount the entire file system and usually I have to move things around or something as root anyways, but as always be cautious.

Restart Required

After modifying the /etc/exports file the NFS server needs to be restarted using…

service nfs-kernel-server restart

Client Machine

You have to install the common code for NFS on the client machine.

sudo apt-get install nfs-common

Mount Commands

My Home Directory on the Raspberry Pi

In this case I am mounting my home directory from the Pi under /home/erick-pi. I had to use the nolock option because I was getting an error without it, other than that it works fine.

Example of the mount command from the command line…

sudo mount -o nolock 192.168.1.17:/home/erick /home/erick-pi
Entire Root Directory

Occasionally I want to mount the entire file system of the Raspberry Pi at a location on one PC.

sudo mount -o nolock 192.168.1.17:/ /mnt/nfs/srv/
Mount Scripts

For situations that only apply occasionally, such as the above example of mounting the entire directory structure, I have created some scripts and placed them in the bin folder under my home folder and made them executable by using chmod +x filename. Then I can run them as needed by running a script with a filename that makes sense to me. Like the one for the code below is, rasp-pi-mount-root.sh.

#! /bin/bash
sudo mount -o nolock 192.168.1.17:/ /mnt/nfs/srv/
intr option

Note that the intr option for nfs mounts is a good one if the computer loses it’s connection with the server. With the Raspberry Pi this is never an issue for me but it is with other servers. It allows an interupt to stop NFS requests if the server goes down or the connection is lost. If intr is not used NFS will keep trying and the process will hang, requireing a reboot. I have had this occur mostly with servers that are on only part time. Most likely, I started the server and then put my computer on standby. When I start it and the server is off, NFS will hang looking for the mount points that have disappeared. This will hang not only the X window folders, but it will hang any command in a terminal that has to touch NFS, such as df -h which tries to look for something that is not there anymore.

Hard and Soft Options

The hard and soft options are like what they sound like. Soft mounts give up after a timeout and don’t keep trying to write or read from an NFS mount point if it flakes out or goes down. Soft mounts should only be used for read only mounts as data being written can be corrupted if a soft mount gives up where the hard mount will just keep trying until the mount comes back. If there is an issue with a write mount point flaking or dropping it is best to mount hard, the default and use the intr option.

Mounting on Startup

It is possible to set up the /etc/fstab file to mount the NFS drive on start up.  It is as simple as adding the following line to the file…

192.168.1.17:/home/erick    /home/erick-pi    nolock    0    0

On my laptop this did not work, I remembered that Wireless LAN is handled at the user level and not on during bootup when mountpoints are handed out via the /etc/fstab file. So I got an error about the mount point not being found.

On my desktop running Lubuntu 14.04, connected via Ethernet cable, the line above did not work either but I modified it as follows and it worked. It might be that I left out the part with nfs, although I thought that the OS could tell it was a NFS mount from the format, oh well. I also decided to mount it right under my home folder on the desktop…

192.168.1.17:/home/erick    /home/erick/erick-pi nfs   auto,nolock    0    0

On the desktop once the drive is mounted it will stay mounted even if I reboot the Rasp Pi while the desktop PC is running.

Delayed Mount on Startup

To mount an NFS drive on a machine that has wireless, you have to mount it after it connects to the router and by then it is already running at the user level. You have to trick the system into waiting. There are multiple ways of doing this. I chose putting a line into to root crontab and used sleep 60 for the delay. After all most mounting has to be done as root anyways.

So I put a 60 second delay in before the mount command executes in the root crontab using the @reboot directive…

@reboot bash -c "sleep 60; mount -o nolock 192.168.1.17:/home/erick /home/erick-pi"

To edit the root crontab, simple do…

sudo nano crontab -e

To simply list what is in the root crontab, which is how I cut and pasted the code above, simply do…

sudo crontab -l

Using Names

It is entirely possible to use names instead of IP addresses when you mount NFS drives and even in the /etc/exports file. One caution, if DNS is down or flaky on you LAN, it could present a problem with reliably mounting drives.  Therefore I recommend adding the server names to your /etc/hosts file. On my LAN I take it a step further, the servers are all set as static IP and my router has the ability to always hand out the same IP to a machine at a specific MAC address so I use that for laptops & etc that normally connect to the network. So in effect every normally used device has a static IP. Therefore I can put them all in /etc/hosts and I don’t even have to care about DNS on the LAN for the machines on it 99% of the time.

/etc/hosts comes with the top two entries, just add what you want to it. As you can see commenting them out works too. The erick-MS-6183 server is down, probably for good at this point!

 127.0.0.1    localhost
 127.0.1.1    erick-laptop
 #192.168.1.11    erick-MS-6183
 192.168.1.2    renee-pc
 192.168.1.9      erick-laptop
 192.168.1.10     ubuntuserver
 192.168.1.17     raspberrypi

 Gotcha

One time I was backing up my laptop to a laptop-backup directory under my home folder on the big file server. The problem was that I had my home folder on the big file server set as an NFS mount as a folder under my home folder on the laptop. It copied in circles until the harddrive filled up. Oh well, learned the hard way on that one! Be careful of NFS mounts and even symlinks to places when running backups.

NFS and Users

With users there is the notion of the name and then there is the numerical UID. NFS uses the numerical UID to map across machines. If you plan on using NFS on multiple machines, it pays to keep the UID’s lined up between them. For example, if you set up 2 Linux machines from scratch, there will be a user at UID 1000, that would be you, whatever you called it by name. The first user is at 1000. If you use NFS to mount a directory from one machine to another, no problem it all lines up. The user at UID 1000 is the same on both machines, permissions work out, files can be moved back and forth, no problems.

 Resources

Used this one to get started with NFS…

https://help.ubuntu.com/community/SettingUpNFSHowTo

Helped to figure out the whole user and group ID mapping

NFS: Overview and Gotchas

 exports(5) – Linux man page

Easy to follow, I hink I might have started with this one,

Setting up an NFS Server and Client on Debian Wheezy

I need to look at this one for a sanity check on the errors when I launch NFS server on Raspberrry Pi,

Problem with NFS network

 

SSH Keys

On one of my servers once I got it set up right and working smooth there was rarely a need to log into via SSH remotely. So I left the SSH port 22 closed down in the router. When I really needed to log into it, I would log into the router and hit the DMZ button and open up all the ports to the server briefly. Then I would SSH into it using the normal username and password combo do my business and lock down the ports again.

I learned about SSH Keys a few years back while I was doing some volunteer work on a site. The owner of the server had SSH Keys setup on it so that I could use WinSCP to move files up to it. He believed, rightly so in keeping the security beefed up and didn’t bother with FTP at all. Recently (February 2015) I purchased a Raspberry Pi. Eventually it will replace one of the servers I run. For now, it is a test bed and I would like to be able to log right in, no fiddling with the router! Plus why not make it more secure, that is where SSH Keys come in.

I hunted down the method to set up SSH Keys online. Not hard at all. I followed one article that helped set up the keys and it logged in great. But, I still could also still login via username and password, so I had to apply another step beyond what the article explained.

Finally, once you open up port 22, many attempts to login will occur on the port and you can see this in your router log. Mine is setup to email me the router log and I quickly noticed that I was being emailed logs one after the other. I decided to change the default port 22 to map to an obscure number higher than 1024 by adjusting the port forwarding in the router.

Installing SSH Server

In case you haven’t installed the server part of SSH on your machine here is the command line directive…

sudo apt-get install openssh-server

Setting up SSH Keys (Public Key Authentication)

These are examples of the commands that I used to set up the keys while on the client machine. It is best to try this while you are not too far from you machine physically, just in case something goes wrong and you need to get on the machine physically.

Create the RSA Public/Private Key on the client machine

You will be asked where you want the key stored the default is the .ssh directory under your home folder with a filename of id_rsa. Then you will be asked to provide a passphrase, hit enter if you do not want a passphrase. A passphrase provides an extra level of security. If someone gets a hold of your machine or private key, they still need the passphrase to get anything going.

ssh-keygen -t rsa

You will get the following message. Depending on the machine, it may take a few seconds after the first line, while the machine is doing the calculation before you see the second line. The Raspberry Pi took about 3-4 seconds to spit out the second line.

Generating public/private rsa key pair.
Enter file in which to save the key (/home/pi/.ssh/id_rsa):

The file is alright by default, hit enter, unless you have another place that you need it and know what you are doing. I assume some config file in the system expects the key in the .ssh folder.

Next comes the passphrase question…

Enter passphrase (empty for no passphrase):

…and again….

Enter same passphrase again:

Finally the key is generated and a randomart image is generated, interesting looking but nothing we need for this operation…

Your identification has been saved in /home/pi/.ssh/id_rsa.
Your public key has been saved in /home/pi/.ssh/id_rsa.pub.
The key fingerprint is:
d7:33:ed:91:ab:00:a7:bd:15:8d:15:21:fe:ed:6b:df pi@raspberrypi
The key's randomart image is:
+--[ RSA 2048]----+
|            . o. |
|           . . . |
|            . .  |
|           . * o |
|        S o * * .|
|         *   = + |
|        . o . o .|
|           + . .o|
|          . . ..E|
+-----------------+

Copy the Public Key to the Server

Next you will copy the key up to the server using the ssh-copy-id command. It will log you in and you will use your normal password that you have for your login and then it will copy the key to the server. The example shows that the user is pi and the ip=192.168.1.17. Change these to your id and server IP.

ssh-copy-id pi@192.168.1.17

In this example I am installing it on the same machine that I created it. So this is what I see…

pi@raspberrypi ~ $ ssh-copy-id pi@192.168.1.17
The authenticity of host '192.168.1.17 (192.168.1.17)' can't be established.
ECDSA key fingerprint is 7e:f0:94:8a:bd:f2:95:44:f3:a5:36:ff:e3:64:48:a3.
Are you sure you want to continue connecting (yes/no)?
Warning: Permanently added '192.168.1.17' (ECDSA) to the list of known hosts.

…And the key is added.

Test It

Now you can login to your server with the newly created keys. But you still can also login via the username and password combo.

Making it SSH Key Only login

You need to set the sshd_config to explicity allow Public Key Authentication. This step requires editing the sshd_config file. Which I didn’t remember the location of so I used…

sudo find / -name sshd_config

Edit it …

sudo nano /etc/ssh/sshd_config

Find the line that reads PasswordAuthentication which is set to yes by default, as in commented out = yes.

Set it to no and make sure it is uncommented…

PasswordAuthentication no

Check to see that this is set also…

ChallengeResponseAuthentication no

Restart the SSH server…

sudo service ssh restart

Remapping the SSH Port 22 to something less obvious

If you don’t remap to port, lots of hits happen to it. Attempts to login that will fill up your router logs. In theory someone can still find the new port, but they would have to get lucky or scan the ports. So this does cut down on bogus login attempts significantly.

There are two ways of doing this, in the configuration file sshd_config or by setting up the port forwarding in yur router. I left sshd_config set at port 22 and made the change in the router. I care about the port being mapped to something else for the outside world on my LAN it can stay 22. So I can simply use SSH servername and not specify a port.

sshd_config mod method

There is a line near the beginning of the file, change the 22 to something else and restart the sshd server…

# What ports, IPs and protocols we listen for
Port 22

Router Port Forward Mapping Change Method

Or go into your router and find the port forwarding. In my Netgear router it was under Advanced–> Port Forwarding/Port triggering. You will see a list that allows there to be changed…

# Service Name External Start Port External End Port Internal Start Port Internal End Port Internal IP address
External End Port Internal Start Port Internal End Port

Set it up for a port other than 22 for External Start and end Port, 5678 in this example…

 

SSH 5678 5678 22 22 192.168.1.170

More Tightening of SSH Security

I have not done any of this yet on my machine but for FYI. Under the spot in sshd_config where the port is set there is a place where you can place a whitelist of IP addresses that the sshd will listen for. This can restrict the IP space that can connect to the machine..

# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0

It is also possible to further restrict the actual users that are allowed or denied access to the machine via SSH. This is accomplished by using the AllowUsers, AllowGroups,DenyUsers,DenyGroups directives.

Example…

AllowUsers joe bob naomi
AllowGroups workinggroup powerusergroup
DenyUsers tempuser1 tempuser2
DenyGroups gaming

You can also block the ability to login as root, so that users will have to su to root once logged in.

PermitRootLogin on

Encryption and Keys

Creating keys such as the RSA pair is an interesting mathematical concept. It falls in the realm of one way functions. You can have the public key and have only remote odds of being able to generate the corresponding private key by a brute force search. But, the other way around is easy. It’s kind of like glass breaking or throwing a cup of water in the ocean, in theory it is all still there, but to put it back together is nearly impossible.

In physics this is what makes “time” on the macro scale. On the quantum level, time really doesn’t matter. You can play particles interactions backwards and forwards and it all works out OK. Feynman diagrams, work both ways. But on the macro level, a lot of things just go one way, just like the hash algorithms that generate the encryption keys. The same thing applies to hashes to generate look up tables, it is easy to go one way, to the lookup from the hash, but harder to go the other way. Ratchets, diodes and worm gears, go one way but not the other.

 Resources

How To Set Up SSH Keys

How do I force SSH to only allow uses with a key to log in?

7 Default OpenSSH Security Options You Should Change in /etc/ssh/sshd_config

Ubuntu Server Guide OpenSSH Server