Tag Archives: server

Automatic Server Status Page Creation Update

Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , , , , , , , , , , , , , ,

In January 2015 I created a post about automatically creating a status page for a Linux server that I have. Typically this is put under a restricted directory and allows you to see a snapshot of what is happening with the server. I run it by putting the scripts in the /etc/cron.hourly directory on a Linux PC and a Raspberry Pi running Linux.

It serves as a simple way to check up on the server without having to use a tool such as Webmin that requires a login. It also keeps a trail of log files that get rotated on a monthly basis, so there is always a few old ones around to track down any problems and patterns in the operation.

I have found this information useful when I have traced down malfunctions that can occur when setting up a server and also when I was trying to get a webcam up and running and had the USB bus hang up a few times when the cam was overloaded with too much light.

In the new script file I fixed a bug by adding parenthesis around a line that I was trying to echo and I added code to run the w command to show a quick picture on who is logged in, how long the server has been up and running and the values for the average load on the server at the 1, 5 and 15 minute marks.

Logcreate Script

#!/bin/dash
# Remove old log
rm /var/www/status/log.txt
# Print logged outputs into new log.txt
date >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
tail /var/log/syslog >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
free >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
df -h >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> /var/www/status/log.txt
ps aux | sort -nrk 4 | head >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Logged in User info using w command
w >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Copy log.txt into the full log that is collected
cat /var/www/status/log.txt >> /var/www/status/fulllog.txt
# Create a free standind copy of the process tree
pstree > /var/www/status/pstree.txt

Alternate Version

I also created a version of the script for a desktop Linux PC that does not have Apache installed.  In it I use a DIR variable to contain the directory that I want the log.txt file stored.

 #!/bin/dash

# User defined variables
# No trailing / on DIR!
DIR=/home/erick/status

# Remove old log
rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt
ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/fulllog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

Rotation of Log

In the /etc/cron.monthly directory I have created a file that is called status-log-rotate and it will save backup copies of 2 months worth of the full concatenated server status logs.

#! /bin/bash
DIR=/home/erick/status
mv $DIR/fulllog.txt.1 $DIR/fulllog.txt.2
mv $DIR/fulllog.txt $DIR/fulllog.txt.1

Tweaks for Raspberry Pi

For the Raspberry Pi which has an SD card that I am trying to be conscious of writing to often. I have recently made some modifications to put the /tmp folder onto RAM using tmpfs. I create the hourly log underneath a folder there. Daily via a script it cron.hourly it gets concatenated into a daily log which is under a status folder that has restricted access. This gets appended once per day to the fulllog which actually lives on the SD card. The end result, no multiple hourly writes to the log file, just one append to the full log per day. The only downside is if the power drops and then some log entries will be lost for the day.

Logcreate runs from /etc/cron.hourly for Raspberry Pi

#!/bin/dash
# Set DIR, on Pi this is a temp location for log
DIR=/tmp/web

# Set fixed DIR FIXDIR for files that have to be stored on SD card
# Nevermind, just make a daily log and then copy that to the full log daily.
#FIXDIR=/var/www/status

# Remove old log

rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt

ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/dailylog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

dailylog-to-fulllog script, runs from /etc/cron.daily

#! /bin/bash

DIR=/tmp/web
FIXDIR=/var/www/status

echo "----------------------------------------------" >> $DIR/dailylog.txt
date >> $DIR/dailylog.txt
echo "----------------------------------------------" >> $DIR/dailylog.txt
cat $DIR/dailylog.txt >> $FIXDIR/fulllog.txt
rm $DIR/dailylog.txt

Logcreate Output from Raspberry Pi

Below is what the logcreate script will output to the log.txt file on a Raspberry Pi that I have running as a web server.

Sun Jul 12 14:17:01 EDT 2015

Jul 12 13:47:51 raspberrypi dhclient: DHCPACK from 192.168.1.1
Jul 12 13:47:52 raspberrypi dhclient: bound to 192.168.1.17 -- renewal in 40673 seconds.
Jul 12 13:59:01 raspberrypi /USR/SBIN/CRON[28010]: (erick) CMD (aplay /opt/sonic-pi/etc/samples/guit_e_fifths.wav)
Jul 12 13:59:07 raspberrypi /USR/SBIN/CRON[28009]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:00:01 raspberrypi /USR/SBIN/CRON[28013]: (erick) CMD (/home/erick/fswebcam/cron-timelapse.sh >> timelapse.log)
Jul 12 14:00:23 raspberrypi /USR/SBIN/CRON[28012]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:01:01 raspberrypi /USR/SBIN/CRON[28022]: (root) CMD (/home/erick/bin/usbreset /dev/bus/usb/001/004)
Jul 12 14:01:02 raspberrypi /USR/SBIN/CRON[28021]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:09:01 raspberrypi /USR/SBIN/CRON[28053]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Jul 12 14:17:01 raspberrypi /USR/SBIN/CRON[28064]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

             total       used       free     shared    buffers     cached
Mem:        445804     424488      21316          0     106768     260516
-/+ buffers/cache:      57204     388600
Swap:       102396          0     102396

Filesystem      Size  Used Avail Use% Mounted on
rootfs          6.3G  3.1G  3.0G  51% /
/dev/root       6.3G  3.1G  3.0G  51% /
devtmpfs        214M     0  214M   0% /dev
tmpfs            44M  240K   44M   1% /run
tmpfs           5.0M  8.0K  5.0M   1% /run/lock
tmpfs            88M     0   88M   0% /run/shm
/dev/mmcblk0p5   60M   19M   41M  32% /boot

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      2071  0.0  3.0  24896 13652 ?        Ss   Jun28   2:24 /usr/sbin/apache2 -k start
www-data 27745  0.0  1.5  25412  7084 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27744  0.0  1.5  24960  6760 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27743  0.0  1.5  25428  7116 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27742  0.0  1.5  25396  7036 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27538  0.0  1.5  25396  7032 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27502  0.0  1.5  25404  7036 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27501  0.0  1.5  25396  7044 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27747  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27746  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start

 14:17:02 up 14 days, 12:56,  1 user,  load average: 0.00, 0.01, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
erick    pts/0    192.168.1.5      14:04   10:39   1.70s  1.70s -bash

Backing up Windows User’s to Folders to a Linux File Server

Linux, Server, , , , , , , , , , ,
Robust File Copy for Windows

robocopy.exe available as part of rktools <-download page, can be used to copy files across the network to a Linux machine into a folder setup using Samba.

The DOS batch file below is called serverbackup.bat on my machine. It can start the Linux server using wolcmd <-download page, from Depicus.com and will copy a users folder to a folder on the server and creates a log file C:\tools\backup.log, then it shuts down the Windows PC with a 120 second delay. This delay is mostly there for testing. If the robocopy command goes belly up, the PC will try to jump right to the shutdown, which makes troubleshooting difficult. So leaving a delay is helpful as one can abort a shutdown by executing…

shutdown /a

…from the command line in Windows to stop the machine from shutting down.

I had to review how robocopy worked before deploying it in a backup script and I found an exact example of what I was looking for on Jacob Surland’s photography blog Caught in Pixels. I reviewed his post How to create a backup script using Robocopy before writing the serverbackup.bat script. I have the script downloadable here named serverbackup.bat, rename and modify as needed.

REM Wake Server, if it is already on, no harm done. It takes about 17 secs for server to start so REM robocopy will get an error and should keep retrying
REM run Depicus Wake on Lan from Command Line. Validated, Works!
 wolcmd yourmacaddr localserveripaddr 0.0.0.0 9
REM Copy User folder to \files\user on server via Samba links
 REM /MIR = purge files from dest. that do not exists in source
 REM /M Copy Archiveable files & reset attribute - not using this yet!!!
 REM /XA:SH exclude system and hidden important for user space
 REM /FFT fixes timing up for LINUX assumes FAT file times ( 2 sec granularity)
robocopy "C:\Documents and Settings\Erick_2" \\UBUNTUSERVER\Erick_Backup /MIR /XA:SH /XJ /FFT /W:1 /R:5 /LOG:C:\tools\backup.log
REM Shutdown PC when backups are done
shutdown -s -f -t 120

So far it works….

Resources

Windows Server 2003 Resource Kit Tools aka rktools

Depicus wolcmd download page

How to create a backup script using Robocopy

 

UNIX and DOS endlines

Linux, Server, , , , ,

I had a moment where I forgot about the entire UNIX and DOS endline incompatibility issue. So when I grabbed the autosuspend script with copy and paste and I brought it into eMacs in Windows, saved it to my /files/public folder on the server and tried to execute it. Lots of “$’\r’: command not found” errors.

The solution is to use dos2unix to convert the endlines, if you don’t have it just…

sudo apt-get install dos2unix

Then do dos2unix filename and it will modify it in place. Which is good but beware of this default behavior. It does have other options, which can be explored using dos2unix –help.

Dos2unix has one and only job, to remove CR-LF (Carriage Return-Line Feed )and just leaving LF ( Line Feed ) as UNIX/Linux wants it to be. If a file acts screwy when brought in from Windows it is most likely this issue. I even had to do it on the autosuspend.conf file!

You can always check a file with the command

cat -e filename

BAD, Windows/DOS example…

#!/bin/bash^M$
^M$
# Source the configuration file^M$
. /etc/autosuspend.conf^M$
^M$

GOOD , UNIX/Linux example….

#!/bin/bash$
# Source the configuration file$
. /etc/autosuspend.conf$
$

The caret M$ is DOS, $ is UNIX.

The Linux File System In General

A website that  has a overview of the Linux file system can be found at…

http://www.tldp.org/LDP/intro-linux/html/chap_03.html

 

Samba on a Linux Server

Server, , , , ,

Samba

Backup /etc/samba/smb.conf  before toying with it! Copy it somethings like /etc/samba/smb.bak or /etc/samba/smb.orig for the original and bak for files that you are modding along the way to getting this working. I admit Samba was a bit of a pain to get working, I fussed around a bit on the server and the Windows machines until success occurred.

One mistake I made was to name the folders by the paths as they appear on the server. Bad idea, Microsoft Windows did not like forward slashes and denied access to the folders. Using slashes and perhaps other non-alphanumeric characters are a no-no in the server folder names.

Make Folders on the Server

I created folders named /files/public and /files/erick on the server. More can be added for additional users. What I am doing with the folders is backing up user profiles from Windows machines in the /files/user folders. The public folder is going to hold things like install files for the Windows machines, anti-malware & etc tools, C compiler and DOS DOS-UNIX equivalent tools and so on.

I executed the following commands on the server…

sudo mkdir /files
cd /files
sudo mkdir public
sudo chmod 777 public
mkdir erick

I believe I did a chmod to 777 on files as well. I made the erick directory with my own credentials, I am owner. Directory is created as a 775 by default…

rwxrwxr-x 2 erick erick 4096 Dec 10 21:12 erick

Later on I created a renee folder. Same drill, I did an su and logged in as the user renee after I created the account and ran a mkdir renee under files.

You need to create a Samba password for yourself and any users. Make it the same as the password that you log into the Win machines, especially important if you want to access home folders.

The command for adding a Samba user and password is…

smbpasswd -a user
Linux Users

While on the users topic adding a Linux user with a home directory is accomplished with the following command…

sudo useradd -d /home/username -m username

Adding the password, don’t skip this, if you forget to do this it will cause problems down the road and it might take a while to figure the problems out.

sudo passwd username

There is a command that can take the contents of the skel directory /etc/skel,  into a users home directory. This sets up the files and folders. Normally this will happen when you use the -d /home/username option on useradd. But if you create a user without a home directory and add one later the following command may be helpful…

mkhomedir_helper username

I followed the method above to add a user renee and then created a /files/renee directory on the server.

Editing the smb.conf file

For the following, I opened my /etc/samba/smb.orig and etc/samba/smb.conf files in the eMacs editor and differenced them. The gray lines and sections show the changes, I have highlighted them with red rounded rectangles for clarity. The biggest change is at the bottom of the file where I added code to allow access to the /files/public, /files/erick and /files/renee directories.

Global Settings Changes in smb.conf
Changes under Global Settings in smb.conf
Changes under Global Settings in smb.conf
Authentication Section changes in smb.conf
Changes under Authentication in smb.conf
Changes under Authentication in smb.conf
Share Definitions sections changes in smb.conf.

This is optional and will allow the home directories of the users to be made accessible with read/write access on the network. In this section the changes are post the most part the uncommenting of the grayed out lines that you see below. I think the only change beyond that was setting read only = no.

Share Definitions sections changes in smb.conf
Share Definitions sections changes in smb.conf
Section added to tail of smb.conf for user defined directories

Follow this example to add your own directories to be accessible from the Windows network.

Don’t use any slashes in the names in the [brackets]. I imagine a lot of non-alphanumeric characters will make this fail. Slashes were my problem. I was trying to be clever and using things like [/files/erick]. Also I went to using an underscore instead of a space in the names. This makes it work better from the Windows CLI and scripts, space does not always translate well. I have had issues with scripts where it takes the first part of the folder name and thinks the 2nd part is a switch to the command or something, resulting in failure. Basically the DOS like Windows CLI (Command Line Interface) environment does not like spaces!

I have not tried setting browsable to no. I imagine it can be only access by knowing the names of the files and probably by navigating using the CLI from Windows. This would be acceptable for the two named directories as they are only backup directories and I don’t imagine I would have to browse to the often.

 

Section added to tail of smb.conf for user defined directories
Section added to tail of smb.conf for user defined directories
Restart

Samba needs to be restarted any time you change the smb.conf file. Use the command….

sudo service smbd restart

…to restart.

Windows Machine

The Windows machine needs to be set to the same workgroup. It is best to have the same user names and passwords to both the Win users and the Samba users, in this manner all will work including home file sharing. When you make changes, sometimes you have to log out and in to the Windows user for them to take effect or else you get errors like the folder is not accessible, and other like it about permissions. Windows will prompt for a username and password to access folders as well, especially if the users and passwords do not match between Windows and the Samba server.

smbclient command

Running smbclient -L servername from the server is a good sanity check that the shares are showing up and that the server actually sees the Windows network. If this looks good generally you are in business with Samba at least from the server side.

erick@ubuntuserver:/etc/samba$ smbclient -L ubuntuserver
Enter erick's password:
Domain=[MSHOME] OS=[Unix] Server=[Samba 3.6.3]

        Sharename       Type      Comment
        ---------       ----      -------
        homes           Disk      Home Directories
        print$          Disk      Printer Drivers
        Erick_Backup    Disk      Erick's Files at /files/erick
        Renee_Backup    Disk      Renee's Files at /files/renee
        Public          Disk      Public Files at /files/public
        IPC$            IPC       IPC Service (ubuntuserver server (Samba, Ubuntu))
        erick           Disk      Home Directories
Domain=[MSHOME] OS=[Unix] Server=[Samba 3.6.3]

        Server               Comment
        ---------            -------
        RENEECOMPUTER        Renee's Computer
        UBUNTUSERVER         ubuntuserver server (Samba, Ubuntu)

        Workgroup            Master
        ---------            -------
        MSHOME               RENEECOMPUTER
smbstatus command

Executing smbstatus from the server command line can tell you what computers are connected and if any files are locked. Try executing it while file operations are in progress to see how it behaves. After seeing it in operation, what is going on becomes obvious for the most part. Without any computers connected to Samba folders, nothing interesting is reported. This means that this tool be helpful troubleshooting Samba if you can’t even connect to the folders. But may be of use to troubleshoot issues when all is working OK and then an issue arises. I also have a script that runs and allows the server to shut down when idle, it executes smbstatus as a test to see if any computers are using Samba so the server won’t shutdown while Samba is in use.

It has command line options which I haven’t explored much myself yet.

For the man page on smbstatus…

https://www.samba.org/samba/docs/man/manpages/smbstatus.1.html

 

The next topic in this series is…
Installing OwnCloud rounds out the server
Additional

 

There is a good YouTube tutorial online that runs through the basics of setting Samba up on Ubuntu Server 12.04. It worked for me.