Category Archives: Server

A series of Posts that involve the entire process of setting up a file server using Ubuntu Server 12.04. Covering installing the OS, LAMP stack, Samba for file sharing with Windows machines, OwnCloud for personal cloud storage.
Also covered is waking the server using wake-on-LAN and using a CRON job script to automatically shut down the server when idle. Thrown in are some educational pieces and tips on utilities. Plus how to backup MS machines to the server by waking it up and performing the backup and then allowing the machine(s) to shutdown.

Simple WebDAV

Linux, Raspberry Pi Server, Server, , , , , ,

WebDAV, the DAV stands for Distributed Authoring and Version. In its simplest form would be a folder that can be accessed from the web that has a username and password to keep the content locked. There are two versions basically, plain and SSL which is secure in that the data that flows in and out of the folder is encrypted as it moves through the web. In this post I am covering the simple non-SSL form for starters.

This post assumes that Apache is installed, if you need to install it do…

sudo apt-get install apache2

Then load the Apache modules for DAV…

sudo a2enmod dav
sudo a2enmod dav_fs

Create a folder for WebDAV

I created a directory at…

/srv/homes/webdav

…the command…

mkdir -p /srv/homes/webdav

…will allow the folders above webdav, such as homes be created if they do not exist.

Edit the Apache default file

The WebDAV folder access is simply controlled by the sites-available/default file. To edit it run…

sudo nano /etc/apache2/sites-available/default

Towards the bottom of the file right above the section that has the ScriptAlias for the /cgi-bin/ directory, I placed the following code…

Alias /webdav  /srv/homes/webdav
<Location /webdav>
 Options Indexes
 DAV On
 AuthType Basic
 AuthName "webdav"
 AuthUserFile /etc/apache2/webdav.password
 Require valid-user
 </Location>

Adding the Password

Use the htpasswd command to add a password to a webdav.password file. it will prompt you for a password. The file will contain hashed passwords which are not readable.

sudo htpasswd -c /etc/apache2/webdav.password username

For an extra level of protection you can change ownership of the file to root with the group of www-data, so no regular users can access the file. Setting the permission to read-write for owner root and read only for the www-data group…

sudo chown root:www-data /etc/apache2/webdav.password
sudo chmod 640 /etc/apache2/webdav.password

Access the Folder

With everything setup the folder will now appear at http://your-url-here.com/webdav, you can browse to it to test it out. You will be prompted for the user-name and password created earlier in the adding the password step.

Further Potential for WebDAV

  • Setup multiple WebDAV folders.
  • Put a web folder on expanded storage on a Raspberry Pi, such as use a bind mount to point to a USB stick plugged into the Pi for extra storage space.
  • It is possible to set up WebDAV with SSL to secure it in a way that the data flowing in and out of the folder will be secured from prying eyes. With my non-SSL WebDAV folder, I don’t put anything up there that is critical or really private data.
  • It is possible to use DAV for support of calendars across devices, something I will explore in the future.
  • There is an app for the iPhone that I have tried that allows easy uploading and downloading to the WebDAV folder. It is easy to drop attachments from email and etc. to the folder for access on a PC.

Resources

https://www.digitalocean.com/community/tutorials/how-to-configure-webdav-access-with-apache-on-ubuntu-12-04 

WebDAV Resources

 

Raspberry Pi

Network File System (NFS)

Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , , , , ,

For a while I have been using Samba to remotely connect Windows computers to my Linux computers and one Linux file server. I can even connect my Linux laptop to my Linux server via Samba. But, recently I bought a Raspberry Pi and I got interested in using NFS for three reasons.

  1. The Raspberry Pi will be running 24/7 and I would like the option to automount the home folder and others from it on my Linux computers when I start them up.
  2. I would like to mount some folders on a server (“big” file storage Linux server) that I can start remotely to the Raspberry Pi so that it will act like expanded storage on the Raspberry Pi. Then I can start the “big” server remotely and mount the folders on the Pi and use the Pi as a proxy. So it is connected to the web and I can navigate to folders that are on the big server via a connection to the Pi with NFS mounted directories.
  3. It would make it easy to backup Linux machines, including the Pi to the big server periodically. Years ago I thought about NFS for mounting folders for backup but I was pretty happy using a scripted FTP system for backups, so I shelved implementing NFS mounts back then.

Implementing NFS was a lot easier than I thought it would be. It was actually much easier than getting Samba to work the way I wanted it to.

The first step ( in my opinion) is to have the machines that you will mount directories from and to on static IP address. On a home network it really does make it easier to have all the machines other than guest machines on static IP’s. This can be done either by setting the machine to have a static IP. Or it may be possible depending on the router, for the router to be configured to hand out the same IP address to a machine with a specific MAC ID. Effectively the results are the same.

Static IP’s are useful as the actual IP addresses will be listed in the export file. It may be possible to use names, however this depends on how DNS is handled on your network. Using the actual IP addresses will make initial setup nearly foolproof. Also an easy way to use names on any machine is to add the static IP’s and names of the machines on the network to the /etc/hosts file.

Install NFS Support

To install support for NFS on the machines run….

sudo apt-get install nfs-kernel-server

Exports File on Server

Modify the servers /etc/exports file to suit your needs. Below is an example from my system on the Raspberry Pi. Remember to restart the NFS server when you have made changes to the file….

sudo service nfs-kernel-server restart
/etc/exports
# /etc/exports: the access control list for filesystems which may be exported
 #        to NFS clients.  See exports(5).
 #
 # Example for NFSv2 and NFSv3:
 # /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
 #
 # Example for NFSv4:
 # /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
 # /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
 #
 /srv/homes       192.168.1.9(rw,sync,no_subtree_check)
 /home/erick      192.168.1.0/24(rw,all_squash,anonuid=1001,anongid=1004,no_subtree_check)
 /         192.168.1.9(rw,no_root_squash,anonuid=1001,anongid=1004,no_subtree_check)

My initial try at this was to semi follow an example and create a /srv/homes directory and export it to one machine at 192.168.1.9. rw = read write access, sync = change both folders on server and client to keep sync’d, no_subtree_check = keeps the machine from having to check consistency of file names, prevents problems if a file is open and the file is renamed.

Then I decided to export my own home directory to all of the machines on the LAN, using 192.168.1.0/24 which allows access from 192.168.1.1-192.168.1.255. This time I am using all_squash which maps all UID’s and GID’s to nobody and nogroup, then setting anonuid=1001, my UID on the Rasp Pi and anongid=1004 my group on the Rasp Pi, they will map over to the correct UID and GID for myself on the other machines. Therefore I have no problem with read write access as the same user on the other machine to the NFS drive.

The next line exports the entire file system to one machine, but has no_root_squash set, which allows the root to access and create files as root on the server. This is one to be careful with, I use it only when I need to mount the entire file system and usually I have to move things around or something as root anyways, but as always be cautious.

Restart Required

After modifying the /etc/exports file the NFS server needs to be restarted using…

service nfs-kernel-server restart

Client Machine

You have to install the common code for NFS on the client machine.

sudo apt-get install nfs-common

Mount Commands

My Home Directory on the Raspberry Pi

In this case I am mounting my home directory from the Pi under /home/erick-pi. I had to use the nolock option because I was getting an error without it, other than that it works fine.

Example of the mount command from the command line…

sudo mount -o nolock 192.168.1.17:/home/erick /home/erick-pi
Entire Root Directory

Occasionally I want to mount the entire file system of the Raspberry Pi at a location on one PC.

sudo mount -o nolock 192.168.1.17:/ /mnt/nfs/srv/
Mount Scripts

For situations that only apply occasionally, such as the above example of mounting the entire directory structure, I have created some scripts and placed them in the bin folder under my home folder and made them executable by using chmod +x filename. Then I can run them as needed by running a script with a filename that makes sense to me. Like the one for the code below is, rasp-pi-mount-root.sh.

#! /bin/bash
sudo mount -o nolock 192.168.1.17:/ /mnt/nfs/srv/
intr option

Note that the intr option for nfs mounts is a good one if the computer loses it’s connection with the server. With the Raspberry Pi this is never an issue for me but it is with other servers. It allows an interupt to stop NFS requests if the server goes down or the connection is lost. If intr is not used NFS will keep trying and the process will hang, requireing a reboot. I have had this occur mostly with servers that are on only part time. Most likely, I started the server and then put my computer on standby. When I start it and the server is off, NFS will hang looking for the mount points that have disappeared. This will hang not only the X window folders, but it will hang any command in a terminal that has to touch NFS, such as df -h which tries to look for something that is not there anymore.

Hard and Soft Options

The hard and soft options are like what they sound like. Soft mounts give up after a timeout and don’t keep trying to write or read from an NFS mount point if it flakes out or goes down. Soft mounts should only be used for read only mounts as data being written can be corrupted if a soft mount gives up where the hard mount will just keep trying until the mount comes back. If there is an issue with a write mount point flaking or dropping it is best to mount hard, the default and use the intr option.

Mounting on Startup

It is possible to set up the /etc/fstab file to mount the NFS drive on start up.  It is as simple as adding the following line to the file…

192.168.1.17:/home/erick    /home/erick-pi    nolock    0    0

On my laptop this did not work, I remembered that Wireless LAN is handled at the user level and not on during bootup when mountpoints are handed out via the /etc/fstab file. So I got an error about the mount point not being found.

On my desktop running Lubuntu 14.04, connected via Ethernet cable, the line above did not work either but I modified it as follows and it worked. It might be that I left out the part with nfs, although I thought that the OS could tell it was a NFS mount from the format, oh well. I also decided to mount it right under my home folder on the desktop…

192.168.1.17:/home/erick    /home/erick/erick-pi nfs   auto,nolock    0    0

On the desktop once the drive is mounted it will stay mounted even if I reboot the Rasp Pi while the desktop PC is running.

Delayed Mount on Startup

To mount an NFS drive on a machine that has wireless, you have to mount it after it connects to the router and by then it is already running at the user level. You have to trick the system into waiting. There are multiple ways of doing this. I chose putting a line into to root crontab and used sleep 60 for the delay. After all most mounting has to be done as root anyways.

So I put a 60 second delay in before the mount command executes in the root crontab using the @reboot directive…

@reboot bash -c "sleep 60; mount -o nolock 192.168.1.17:/home/erick /home/erick-pi"

To edit the root crontab, simple do…

sudo nano crontab -e

To simply list what is in the root crontab, which is how I cut and pasted the code above, simply do…

sudo crontab -l

Using Names

It is entirely possible to use names instead of IP addresses when you mount NFS drives and even in the /etc/exports file. One caution, if DNS is down or flaky on you LAN, it could present a problem with reliably mounting drives.  Therefore I recommend adding the server names to your /etc/hosts file. On my LAN I take it a step further, the servers are all set as static IP and my router has the ability to always hand out the same IP to a machine at a specific MAC address so I use that for laptops & etc that normally connect to the network. So in effect every normally used device has a static IP. Therefore I can put them all in /etc/hosts and I don’t even have to care about DNS on the LAN for the machines on it 99% of the time.

/etc/hosts comes with the top two entries, just add what you want to it. As you can see commenting them out works too. The erick-MS-6183 server is down, probably for good at this point!

 127.0.0.1    localhost
 127.0.1.1    erick-laptop
 #192.168.1.11    erick-MS-6183
 192.168.1.2    renee-pc
 192.168.1.9      erick-laptop
 192.168.1.10     ubuntuserver
 192.168.1.17     raspberrypi

 Gotcha

One time I was backing up my laptop to a laptop-backup directory under my home folder on the big file server. The problem was that I had my home folder on the big file server set as an NFS mount as a folder under my home folder on the laptop. It copied in circles until the harddrive filled up. Oh well, learned the hard way on that one! Be careful of NFS mounts and even symlinks to places when running backups.

NFS and Users

With users there is the notion of the name and then there is the numerical UID. NFS uses the numerical UID to map across machines. If you plan on using NFS on multiple machines, it pays to keep the UID’s lined up between them. For example, if you set up 2 Linux machines from scratch, there will be a user at UID 1000, that would be you, whatever you called it by name. The first user is at 1000. If you use NFS to mount a directory from one machine to another, no problem it all lines up. The user at UID 1000 is the same on both machines, permissions work out, files can be moved back and forth, no problems.

 Resources

Used this one to get started with NFS…

https://help.ubuntu.com/community/SettingUpNFSHowTo

Helped to figure out the whole user and group ID mapping

NFS: Overview and Gotchas

 exports(5) – Linux man page

Easy to follow, I hink I might have started with this one,

Setting up an NFS Server and Client on Debian Wheezy

I need to look at this one for a sanity check on the errors when I launch NFS server on Raspberrry Pi,

Problem with NFS network

 

SSH Keys

Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , ,

On one of my servers once I got it set up right and working smooth there was rarely a need to log into via SSH remotely. So I left the SSH port 22 closed down in the router. When I really needed to log into it, I would log into the router and hit the DMZ button and open up all the ports to the server briefly. Then I would SSH into it using the normal username and password combo do my business and lock down the ports again.

I learned about SSH Keys a few years back while I was doing some volunteer work on a site. The owner of the server had SSH Keys setup on it so that I could use WinSCP to move files up to it. He believed, rightly so in keeping the security beefed up and didn’t bother with FTP at all. Recently (February 2015) I purchased a Raspberry Pi. Eventually it will replace one of the servers I run. For now, it is a test bed and I would like to be able to log right in, no fiddling with the router! Plus why not make it more secure, that is where SSH Keys come in.

I hunted down the method to set up SSH Keys online. Not hard at all. I followed one article that helped set up the keys and it logged in great. But, I still could also still login via username and password, so I had to apply another step beyond what the article explained.

Finally, once you open up port 22, many attempts to login will occur on the port and you can see this in your router log. Mine is setup to email me the router log and I quickly noticed that I was being emailed logs one after the other. I decided to change the default port 22 to map to an obscure number higher than 1024 by adjusting the port forwarding in the router.

Installing SSH Server

In case you haven’t installed the server part of SSH on your machine here is the command line directive…

sudo apt-get install openssh-server

Setting up SSH Keys (Public Key Authentication)

These are examples of the commands that I used to set up the keys while on the client machine. It is best to try this while you are not too far from you machine physically, just in case something goes wrong and you need to get on the machine physically.

Create the RSA Public/Private Key on the client machine

You will be asked where you want the key stored the default is the .ssh directory under your home folder with a filename of id_rsa. Then you will be asked to provide a passphrase, hit enter if you do not want a passphrase. A passphrase provides an extra level of security. If someone gets a hold of your machine or private key, they still need the passphrase to get anything going.

ssh-keygen -t rsa

You will get the following message. Depending on the machine, it may take a few seconds after the first line, while the machine is doing the calculation before you see the second line. The Raspberry Pi took about 3-4 seconds to spit out the second line.

Generating public/private rsa key pair.
Enter file in which to save the key (/home/pi/.ssh/id_rsa):

The file is alright by default, hit enter, unless you have another place that you need it and know what you are doing. I assume some config file in the system expects the key in the .ssh folder.

Next comes the passphrase question…

Enter passphrase (empty for no passphrase):

…and again….

Enter same passphrase again:

Finally the key is generated and a randomart image is generated, interesting looking but nothing we need for this operation…

Your identification has been saved in /home/pi/.ssh/id_rsa.
Your public key has been saved in /home/pi/.ssh/id_rsa.pub.
The key fingerprint is:
d7:33:ed:91:ab:00:a7:bd:15:8d:15:21:fe:ed:6b:df pi@raspberrypi
The key's randomart image is:
+--[ RSA 2048]----+
|            . o. |
|           . . . |
|            . .  |
|           . * o |
|        S o * * .|
|         *   = + |
|        . o . o .|
|           + . .o|
|          . . ..E|
+-----------------+

Copy the Public Key to the Server

Next you will copy the key up to the server using the ssh-copy-id command. It will log you in and you will use your normal password that you have for your login and then it will copy the key to the server. The example shows that the user is pi and the ip=192.168.1.17. Change these to your id and server IP.

ssh-copy-id pi@192.168.1.17

In this example I am installing it on the same machine that I created it. So this is what I see…

pi@raspberrypi ~ $ ssh-copy-id pi@192.168.1.17
The authenticity of host '192.168.1.17 (192.168.1.17)' can't be established.
ECDSA key fingerprint is 7e:f0:94:8a:bd:f2:95:44:f3:a5:36:ff:e3:64:48:a3.
Are you sure you want to continue connecting (yes/no)?
Warning: Permanently added '192.168.1.17' (ECDSA) to the list of known hosts.

…And the key is added.

Test It

Now you can login to your server with the newly created keys. But you still can also login via the username and password combo.

Making it SSH Key Only login

You need to set the sshd_config to explicity allow Public Key Authentication. This step requires editing the sshd_config file. Which I didn’t remember the location of so I used…

sudo find / -name sshd_config

Edit it …

sudo nano /etc/ssh/sshd_config

Find the line that reads PasswordAuthentication which is set to yes by default, as in commented out = yes.

Set it to no and make sure it is uncommented…

PasswordAuthentication no

Check to see that this is set also…

ChallengeResponseAuthentication no

Restart the SSH server…

sudo service ssh restart

Remapping the SSH Port 22 to something less obvious

If you don’t remap to port, lots of hits happen to it. Attempts to login that will fill up your router logs. In theory someone can still find the new port, but they would have to get lucky or scan the ports. So this does cut down on bogus login attempts significantly.

There are two ways of doing this, in the configuration file sshd_config or by setting up the port forwarding in yur router. I left sshd_config set at port 22 and made the change in the router. I care about the port being mapped to something else for the outside world on my LAN it can stay 22. So I can simply use SSH servername and not specify a port.

sshd_config mod method

There is a line near the beginning of the file, change the 22 to something else and restart the sshd server…

# What ports, IPs and protocols we listen for
Port 22

Router Port Forward Mapping Change Method

Or go into your router and find the port forwarding. In my Netgear router it was under Advanced–> Port Forwarding/Port triggering. You will see a list that allows there to be changed…

# Service Name External Start Port External End Port Internal Start Port Internal End Port Internal IP address
External End Port Internal Start Port Internal End Port

Set it up for a port other than 22 for External Start and end Port, 5678 in this example…

 

SSH 5678 5678 22 22 192.168.1.170

More Tightening of SSH Security

I have not done any of this yet on my machine but for FYI. Under the spot in sshd_config where the port is set there is a place where you can place a whitelist of IP addresses that the sshd will listen for. This can restrict the IP space that can connect to the machine..

# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0

It is also possible to further restrict the actual users that are allowed or denied access to the machine via SSH. This is accomplished by using the AllowUsers, AllowGroups,DenyUsers,DenyGroups directives.

Example…

AllowUsers joe bob naomi
AllowGroups workinggroup powerusergroup
DenyUsers tempuser1 tempuser2
DenyGroups gaming

You can also block the ability to login as root, so that users will have to su to root once logged in.

PermitRootLogin on

Encryption and Keys

Creating keys such as the RSA pair is an interesting mathematical concept. It falls in the realm of one way functions. You can have the public key and have only remote odds of being able to generate the corresponding private key by a brute force search. But, the other way around is easy. It’s kind of like glass breaking or throwing a cup of water in the ocean, in theory it is all still there, but to put it back together is nearly impossible.

In physics this is what makes “time” on the macro scale. On the quantum level, time really doesn’t matter. You can play particles interactions backwards and forwards and it all works out OK. Feynman diagrams, work both ways. But on the macro level, a lot of things just go one way, just like the hash algorithms that generate the encryption keys. The same thing applies to hashes to generate look up tables, it is easy to go one way, to the lookup from the hash, but harder to go the other way. Ratchets, diodes and worm gears, go one way but not the other.

 Resources

How To Set Up SSH Keys

How do I force SSH to only allow uses with a key to log in?

7 Default OpenSSH Security Options You Should Change in /etc/ssh/sshd_config

Ubuntu Server Guide OpenSSH Server

Alternatives to FTP

Linux, Server, , , , , ,

One server I have is fairly low on resources, so I opted not to run FTP. It would just mean yet another service that would have to run on a low RAM unit. So to move files to and from this server I use scp or sftp from Linux and WinSCP from Windows.

SCP Example

These examples assumes you can SSH into your server!

Using a FQDN

The following example shows downloading a directories content from a remote server using a fully qualified domain name.

 scp -r username@serverlocation.com:/home/username/dir /home/username/dir

Using a IP address

On the local network in this example using an ip address, copying remote to local.

 scp -r user@192.168.1.101:/home/user/fswebcam /home/user/fswebcam

Example of uploading a single file to a remote server from the home directory of the user to a specific location under the users home directory tree on a remote computer, note the tilde (~) means home directory of user.

scp ~/fswebcam/timelapse/dusk.avi user@12.34.56.78:/home/user/files/public/timelapse-video/dusk.avi

SFTP

To connect using sftp, a ftp tunnel using SSH, typically you can use the “Connect to Server” found for instance in Ubuntu under Places.

  • Set connection type to SSH
  • Set the server
  • IP address or FQDN
  • Port is set to 22, the standard SSH port
  • Folder is set to any folder that the user has permission to get into, /home/user is a safe bet.
  • Username is set
Connect to Server in Ubuntu, Place Menu
Connect to Server in Ubuntu, Place Menu

 

  • You can add a bookmark to keep getting in to this connection
  • It will ask for your login password upon connecting

SFTP via Browser

Also from a Firefox browser, Haven’t tried this on others! you can simply put sftp://user@serveraddress in the address bar. This will connect you to your home folder after you give the password at the prompt. I noticed that in Ubuntu, it will do the same thing that the “Connect to Server” option will do. It will show a folder on the desktop
after connecting with the browser that it the sftp connection

WinSCP

From Windows I have used the tool WinSCP for years as it supports FTP, SFTP and SCP. http://winscp.net/eng/index.php
It also loads support, by editing the registry perhaps for using the sftp:// type of connection via Windows Explorer.

rsync

For Linux there is also the command rsync, remotely synchronize directories. I haven’t used this but once or twice so I don’t have much to say about it yet.

One more comment on SSH. Typically I leave SSH (Port 22) closed and open it up only when needed on this server. I do this by remotely logging into a my router and opening it and closing it. Alternatively you could configure a firewall to only allow certain IP numbers a connection to SSH and denying all others. This can be done using the direct method of editing the iptables ( I will write more on this, TBD) or using a tool such as UFW or the graphical version of it called GUFW to  handle this.

Automatic Server Status Page Creation

Linux, Server, , , , , , , , , , ,

On one of the servers I ran in 2013-14, I used Webmin to keep track of what was going on with the server, memory usage, drive space and so on. It was a bit overkill, I thought I would need it more than I really did.

The server I am trying this out on is resource limited, low RAM mostly, only 512MB. So I was concerned about too many processes weighing it down and was trimming RAM use for Apache, mySql and PHP. I wanted an easy way to look at what is going on with the server, web based, so putting the info on a dynamically created page seemed like the way to go.

Restricting Directory Access with Apache

I don’t want just anyone to have access to the status directory. Clever folks might gain too much insight from what is shown there, a potential hack risk. On the server the location /var/www/status is restricted. What I mean by this is that I have edited the Apache default file to restrict access by IP, as I am only accessing this from a few IP’s. Below is an example of the mod to the Apache default file. Obviously I want to allow from my local net, so that is 192.168.1 ranging from 0-255. In the default file you don’t have to list the entire IP if you want to cover a range. Additionally at the time, there were a few IP’s in the 74.67.XX.XX range so I opened that range up for testing access to. Basically you can add as many as you want. Another option would be to password protect the directory, but for now this is all I need.

To edit the Apache default file, make a backup copy first, then on Ubuntu at least…

sudo nano /etc/apache2/sites-available/default
Example code from Apache default file to allow certain IP’s access to a directory and deny all others
<Directory /var/www/status/>
        Order deny,allow
        Allow from 192.168.1
                Allow from 74.67
    
        Deny from all
</Directory>

Logcreate

With this server instead of using Webmin to look at the status of the server,  I made a simple file called logcreate, ran by putting it in the cron.hourly folder and chmodding it +x! It makes a status page at /var/www/status/log.txt. Also generated is /var/www/status/fulllog.txt a concatenated version of the log.txt added to on an hourly basis. I used dash instead of bash, it’s a slight improvement in memory use when called. Don’t use an extension, cron won’t run files such as logcreate.sh.

Logcreate basically it gives you a synopsis of the servers state in text form…

  • Date and time stamp on top ( date )
  • Tail of the syslog ( tail /var/log/syslog )
  • Memory usage ( free )
  • Drive space usage ( df -h )
  • Processes sorted by RAM usage ( ps aux | sort -nrk 4 | head )
  • Free standing copy of the process tree ( pstree )

 

The code for logcreate, the file to be placed in /etc/cron.hourly
#!/bin/dash
# Remove old log
rm /var/www/status/log.txt
# Print logged outputs into new log.txt 
# Starting with date stamp
date >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Grab the tail of the syslog file
tail /var/log/syslog >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Log RAM usage
free >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Disk Usage
df -h >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
#echo 'USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND ' >> log.txt
ps aux | sort -nrk 4 | head >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Copy log.txt into the full log that is collected from the hourly updates.
cat /var/www/status/log.txt >> /var/www/status/fulllog.txt
# Create a free standing copy of the process tree
pstree > /var/www/status/pstree.txt

 

Resources

Figuring out a good command to list the running processes sorted by RAM use was something I needed some help figuring out as far as the best way to do it. The link below was where I got my info from.

Top memory using processes:  http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage

Backing up Windows User’s to Folders to a Linux File Server

Linux, Server, , , , , , , , , , ,
Robust File Copy for Windows

robocopy.exe available as part of rktools <-download page, can be used to copy files across the network to a Linux machine into a folder setup using Samba.

The DOS batch file below is called serverbackup.bat on my machine. It can start the Linux server using wolcmd <-download page, from Depicus.com and will copy a users folder to a folder on the server and creates a log file C:\tools\backup.log, then it shuts down the Windows PC with a 120 second delay. This delay is mostly there for testing. If the robocopy command goes belly up, the PC will try to jump right to the shutdown, which makes troubleshooting difficult. So leaving a delay is helpful as one can abort a shutdown by executing…

shutdown /a

…from the command line in Windows to stop the machine from shutting down.

I had to review how robocopy worked before deploying it in a backup script and I found an exact example of what I was looking for on Jacob Surland’s photography blog Caught in Pixels. I reviewed his post How to create a backup script using Robocopy before writing the serverbackup.bat script. I have the script downloadable here named serverbackup.bat, rename and modify as needed.

REM Wake Server, if it is already on, no harm done. It takes about 17 secs for server to start so REM robocopy will get an error and should keep retrying
REM run Depicus Wake on Lan from Command Line. Validated, Works!
 wolcmd yourmacaddr localserveripaddr 0.0.0.0 9
REM Copy User folder to \files\user on server via Samba links
 REM /MIR = purge files from dest. that do not exists in source
 REM /M Copy Archiveable files & reset attribute - not using this yet!!!
 REM /XA:SH exclude system and hidden important for user space
 REM /FFT fixes timing up for LINUX assumes FAT file times ( 2 sec granularity)
robocopy "C:\Documents and Settings\Erick_2" \\UBUNTUSERVER\Erick_Backup /MIR /XA:SH /XJ /FFT /W:1 /R:5 /LOG:C:\tools\backup.log
REM Shutdown PC when backups are done
shutdown -s -f -t 120

So far it works….

Resources

Windows Server 2003 Resource Kit Tools aka rktools

Depicus wolcmd download page

How to create a backup script using Robocopy

 

Server Hardware Swaps

RAM Upgrade

When I initially built the server using a Dell Dimension 4200, I added 1GB of RAM on top of the 512MB that was factory installed. The board can support up to 2GB, but 1.5GB seems sufficient for what I am doing. One of the first steps is to run MEMTEST by booting off of a Linux CD that I had laying around. This test ran overnight (15+ hours) with no problems, it’s always a good idea to run MEMTEST with any memory changes.

Memory upgrade, added 1GB stick to the exiting 512MB
Memory upgrade, added 1GB stick to the exiting 512MB
Running MEMTEST to check for flaws in the RAM, before loading Ubuntu Server
Running MEMTEST to check for flaws in the RAM, before loading Ubuntu Server

 

 

 

 

 

 

 

 

 

 

 

Second Hard Drive
HD in floppy bay. Tight a bit tough getting the screws in.
HD in floppy bay. Tight, a bit tough getting the screws into the holder.
Pulled lower CD burner, replaced with DVD drive
Pulled lower CD burner, replaced with DVD drive
DVD drive goes in bottom slot, ready to load Ubuntu 12.04
DVD drive goes in bottom slot, ready to load Ubuntu 12.04

I removed the floppy drive and added a second 120GB hard drive. I also replaced one of the CD drives with a DVD reader. Ubuntu Server 12.04 gets burned onto a DVD so I needed to boot off of the DVD. The other option would be to boot from a USB drive. I swapped the IDE connector off of one of the CD drives and used it for the secondary hard drive mounted in the floppy drive bay.

Disconnected CD drive, hooked IDE HD in same bus as DVD drive.
Disconnected CD drive, hooked IDE HD in same bus as DVD drive.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Swap 120GB hard drive for 500GB

I soon was well on my way to filling up the primary 200GB drive so it was time to consider putting in a bigger secondary hard drive in preparation for the future.

I have installed the 500gb drive in the server and formatted for use with Linux. Linux can use a drive formatted as NTFS but I formatted it as EXT4 for Linux so the disk checks and fixes can be more precise. EXT4 can handle extremely large drives 1EB partition size, an amount of data I cannot even imagine! EXT3 is good up to 32TB partitions, which is still very big! The new extra drive will give me much more space as the main 200gb drive is almost full. I will move some files onto it, mostly the backups from the other computers at home. This is primarily the goal with second drive. There is a software manger in Linux that can manage the drive, so called “Logical Volume Management” (LVM), the primary drive is managed using this feature. In theory I can create a “snapshot” of the drive and copy the image onto a backup external or the second drive, but I have not considered doing this yet. The primary drive contains the OS, whatever software I have loaded, which doesn’t take up much space. The files that I have loaded onto OwnCloud take up a good deal of space, but 200GB will be plenty of space for the primary drive for a while.

To LVM or not?

At first I was going to connect the first and second drive into one large “logical” drive using LVM. But, there is a risk if the system treats the 500gb+200gb = 700gb logical drive. If one drive fails it can ruin the entire “logical” drive composed of both drives. One disk failing out of two might be a bad risk, so I might leave the drives connected normally, mounted separately and not as a big logical drive.

500GB drive mounted in the floppy drive holder
500GB drive mounted in the floppy drive holder
120GB drive out and 500GB drive in
120GB drive out and 500GB drive in
Resources

For more information on Linux file systems…

https://help.ubuntu.com/community/LinuxFilesystemsExplained

About files and the file system…

http://www.tldp.org/LDP/intro-linux/html/chap_03.html

 

 

 

 

UNIX and DOS endlines

Linux, Server, , , , ,

I had a moment where I forgot about the entire UNIX and DOS endline incompatibility issue. So when I grabbed the autosuspend script with copy and paste and I brought it into eMacs in Windows, saved it to my /files/public folder on the server and tried to execute it. Lots of “$’\r’: command not found” errors.

The solution is to use dos2unix to convert the endlines, if you don’t have it just…

sudo apt-get install dos2unix

Then do dos2unix filename and it will modify it in place. Which is good but beware of this default behavior. It does have other options, which can be explored using dos2unix –help.

Dos2unix has one and only job, to remove CR-LF (Carriage Return-Line Feed )and just leaving LF ( Line Feed ) as UNIX/Linux wants it to be. If a file acts screwy when brought in from Windows it is most likely this issue. I even had to do it on the autosuspend.conf file!

You can always check a file with the command

cat -e filename

BAD, Windows/DOS example…

#!/bin/bash^M$
^M$
# Source the configuration file^M$
. /etc/autosuspend.conf^M$
^M$

GOOD , UNIX/Linux example….

#!/bin/bash$
# Source the configuration file$
. /etc/autosuspend.conf$
$

The caret M$ is DOS, $ is UNIX.

The Linux File System In General

A website that  has a overview of the Linux file system can be found at…

http://www.tldp.org/LDP/intro-linux/html/chap_03.html

 

Auto-shutdown and Auto-suspending a Linux Server

Server, , , , , , , , , ,

The article below talks about auto-suspending…

http://rolandtapken.de/blog/2013-07/suspend-nas-when-idle

A Simpler Idea

I found another article on auto-suspending that requires only a simple bash script that I have placed in /etc/cron.hourly.

WordPress did not like me uploading autosuspend.sh, for security reasons, it will give an error, so I have the script autosuspend.sh , named autosuspend.sh.txt  here->  autosuspend.sh . The file goes in /etc/cron.hourly naming it just autosuspend, cron won’t run if the filename has an extension.

The file must be owned by root and executable. So you have to use the following commands before running it.

sudo chown root:root autosuspend.sh
sudo chmod u+x autosuspend.sh

I used it as autosuspend.sh and ran it a few times manually running sudo autosuspend.sh, just to see it run properly before sticking the file renamed as autosuspend and placed it into /etc/cron.hourly.

And the autosuspend.conf  named as autosuspend.conf.txt here-> autosuspend.conf  goes in the /etc/ directory.

Both are UNIX formatted files, modify them accordingly for your use.

syslog

CRON logs things when it runs autosuspend into /var/log/syslog, so you can execute…

tail /var/log/syslog

…to see if everything is OK by seeing the traces, the autosuspend script  gives good useful error messages. It also will send an email on the server to root@yourservername, every time it runs. You can use mailx from the CLI ( or some other program ) to read the local email. Mailx is very simple and good enough to quickly page through CRON emails, using return to move down through the unread ones.

Resources

The article I got the script from is…

https://bbs.archlinux.org/viewtopic.php?id=157268

…it does not give you the autosuspend.conf file, but it references another article in German…

http://wiki.ubuntuusers.de/Skripte/AutoSuspend

…that does have the autosuspend.conf file and it seems to work, at least it runs fine so far with some mods.

Files

Once again below are the script and conf file from those sites, labeled with a txt extension. I put them here in case those sites disappear for some reason. This is good knowledge and it works so well, I’d hate to see it get lost.

The script taken from the Archlinux page, requires systemd and uses systemctl suspend to suspend the machine, named autosuspend.sh.txt. Formatted for UNIX/Linux.

autosuspend.sh

Original autosuspend.sh that uses pm-utils from the German ubuntuusers.de site, named as pm-utils_autosuspend.sh.txt and the autosuspend.conf named as autosuspend.conf.txt. Formatted for UNIX/Linux

pm-utils_autosuspend.sh
autosuspend.conf

Auto Shutdown – Mods

I decided to modify the autosuspend.sh file rather than loading the package that it needed (systemd) to execute systemctl suspend, which is what the script file from the first article uses. The other option would be to use pm-utils as the second German article has the original autosuspend.sh formatted to use. For more info on pm-utils see https://wiki.archlinux.org/index.php/pm-utils

Instead of auto suspending, I decided that since the server starts fast enough from a cold boot (17 secs. to usable), why not just replace the…

systemctl suspend

…line with…

shutdown -P +5

This will shut the server down, with a 5 minute warning and guard band. I say guard band, because it can guard against a potential loop. If I play with the script more and make a mistake, I do not want to wind up with a server that starts, jumps to the script and starts shutting down immediately. I know I put the file in /etc/cron.hourly, so it will kick off every hour, but I am just guarding against unforeseen things to be safe and it’s only 5 minutes of delay. If it goes to shut down while testing at some point, I have 5 minutes to execute a shutdown -c to cancel.

I also put the line…

ethtool eth0 -s wol g

…before the shut down line, because that same piece of code, which I tried put into rules.d. But it was not setting the wake on to g, When I ran ethtool, it was staying at d. Not sure why, but since I will be allowing this server to shutdown by itself 90%+ of the time, I opted to put it right in the shutdown script. After a second thought, I also put that line into the /etc/rc.local ( which runs at start up ) as well so it is armed even if I shutdown manually! See the post of Wake On Lan via Ubuntu Linux for more info on Wake on LAN.

Here is the modded autosuspend called autoshutdown.txt. Remove the txt extension and place into the /etc/cron.hourly folder, it is formatted for UNIX.

I forgot about the UNIX and DOS endlines being different while I was working on this. See my post on UNIX vs DOS file endlines, as I had a bit of brain fog and struggled a bit with this while working on the autoshutdown script.

Winbind

Once I got the autoshutdown running. I realized that the Linux machine was not able to resolve the names the Windows machines on the network. The server could only ping the Windows machines by IP address and not their names! I saw this when I was logged out of the server and logged in a while later and the shutdown script had recorded failed pings into syslog, when checking to see if the server was idle. The script correctly saw that no one is logged it by executing, who | wc -l yielding a zero and next it was testing for attached clients ( the Windows machines named in the autosuspend.conf file) using ping $i -c1. And ping was failing as the names were unreachable.

  • arp -a could see all the machines by IP address from both Linux and Windows.
  • net view on the Windows machine could see all the machines by name.
  • smbstatus can see every computer by name fine from my Linux server machine. Particularly since I had installed Samba, the servers name is visible from Windows PC’s due to Samba.
    Samba must send out net-bios information about itself, I see in the config file for Samba where it can act as a wins server as well.

In order for the autosuspend/shutdown script to work pinging by name is a must. To fix, install winbind and configure /etc/nsswitch.conf.

sudo apt-get install winbind

In /etc/nsswitch.conf add wins to the end of the line that starts with
hosts: Mine now reads…

hosts: files dns wins

I got the info from…
http://www.serenux.com/2009/09/howto-configure-ubuntu-to-be-able-to-use-and-respond-to-netbios-hostname-queries-like-windows-does/

Samba Connected test in shutdown script

The autosuspend script does a test to see if anyone is accessing files using Samba via smbstatus. Smbstatus is great to see what is going on, it is good to troubleshoot Samba when you can make connections. It is interesting once you play with it when various computers are accessing the server to understand what it is telling you.
But the script is just looking to see if computers are accessing Samba
shares. The autosuspend.conf shows an IP address for the test using
$SAMBANETWORK as that value and grepping on it. I am not sure how this works as I don’t see any IP numbers when I run smbstatus. So for now I decided that I will use the word Public in the autosuspend.conf instead of 192.168.1. Most likely if a computer is accessing Samba shares on my network and the computers name is not one of the “clients” ( my own machines at home, that have listed names) it is going to be only accessing the Public Samba share. For now this seems to work!

Test used in autosuspend script to look for machines accessing Samba…

/usr/bin/smbstatus | grep $SAMBANETWORK | wc -l
Other conditions for shutdown

The other two tests that autosuspend does (IsRunning() and
IsDaemonActive() ), I have not validated them.

That is a TBD. So far, so good, the server has not shutdown unexpectedly and I have not seen it held up by IsRunning() yet, based on it’s tests. If something is running and a shutdown occurs, a sigterm is generated as the system is going down, so anything in process should terminate cleanly in theory. I’d like to test for OwnCloud activity at some point, I have shut the machine down and restarted a few minutes later on purpose with an OwnCloud file transfer in progress and it picks back up. I have to figure out a test for this, TBD.

IsRunning() tests for the following applications…

 APPLICATIONS='"^nxagent$" "^rsnapshot$" "^wsus$" "^wget$" "^screen$" "^mlnetp$" "^apt-get$" "^aptitude$" "^dpkg$" "^cp$"'

The following features were not copied from the autosuspend.sh on the German site to the one on https://bbs.archlinux.org/viewtopic.php?id=157268 that I used…

# Turning suspend by day (8 a.m. to 3 a.m.) off
 DONT_SUSPEND_BY_DAY='no'
# Automatically reboot once a week when the system isn't in use
 REBOOT_ONCE_PER_WEEK='yes'
  • DONT_SUSPEND_BY_DAY seems to control suspending by blocking it out during the day between 8AM and 3PM, it uses /sys/class/rtc/rtc0/wakealarm. I wasn’t interested in this so I was fine with it being carved out.
  • REBOOT_ONCE_PER_WEEK uses cat /proc/uptime | cut -d’ ‘ -f1-1\` / 3600 / 24 )>= 7\ as a test to see if the machine has been running for more that one week and then it reboots the next time it is idle. This is not of interest to me as my machine will shutdown rather than suspend, so this is not needed either.

Interestingly, I do see a test to see if power management is supported in the original autosuspend.sh that relies on pm-utils. This does not exist in the modified script that uses systemctl, perhaps it is not neccessary as calling systemctl is fine without or it was omitted, because such a test does not exist when using systemctl.

  /usr/bin/pm-is-supported

Basically I am fine with the simpler script, if I need to add features back in, so be it!

I have been using the shutdown script for over a month with no issues so far.

Follow Up

I have been using this code on two servers, one for almost three years and one for a year. The older one does not suspend and it requires a shutdown and the newer one suspends nicely via systemctl suspend.

I decided to modify the code a bit to allow a hybrid-sleep and also allow for restarts when the system requires them. Read more about this here….

Autoshutdown Code Modded to hybrid-sleep and allow required restarts

 

Additional utilities for a Linux Server

Server, , , , , ,

vsftp

Sometimes it is nice to have an ftp server, you might have Samba and ownCloud, but sometimes you really need ftp to do something. It is the right tool at the right time and I can’t imagine running a server without FTP installed.

sudo apt-get install vsftpd
Edit the configuration file

Back it up first then do an edit

sudo cp /etc/vsftpd.conf /etc/vsftpd.orig
sudo nano /etc/vsftpd.conf

uncomment local_enable = YES

uncomment write_enable = YES

In this manner you will be able to read and write to your home directory. With SSH and FTP you can do just about anything remotely to your server. You can ( put ) FTP a file up to your home and move it anywhere and in the opposite direction also ( get ).

For example I downloaded the zip file for the OwnCloud Music App on a Windows computer, then FTP’d it the Linux server into my home directory and moved and unzipped it in the proper directory using SSH. Zip/unzip is not loaded by default with the Ubuntu Server disc, to get it see below.

This is powerful and with that power comes danger. You don’t want anyone to be able to SSH and FTP in, so be careful when opening these ports. I get “hits” on port 22 for SSH a lot, I don’t even open port 21 for FTP outside of my LAN. When I mean hits, I mean I can see IP addresses come in on my routers log that are from outside the US, by looking them up, or browsing to them. Sometimes using a ping command to the IP a return will come from another IP. These cyber-criminals try to get in on open ports.

vsftp website…

https://security.appspot.com/vsftpd.html

Zip/unzip is not loaded by default with the Ubuntu Server disc

Zipping and unzipping files from the CLI is an important thing to be able to do.To get it…

sudo apt-get install zip unzip

More info on how to use it….

http://askubuntu.com/questions/86849/how-to-unzip-a-zip-file-from-the-terminal

 

dos2unix

Editing shell or config files on a Windows machine, presents you with the CR-LF and LF issue, for Win and UNIX respectively. Scripts won’t run, problems happen with config files when they are not in the right format. Frequently I encounter this when I coy and paste some code from the Web into eMacs or Notepad, then save it on the Linux server. Then I need to execute dos2unix on it to make it run right.

UNIX and DOS endlines

I had a brain dead moment where I forgot about the entire UNIX and DOS endline thing when I was working on getting the server to auto shutdown.
So when I grabbed the autosuspend script with copy and paste and I brought it into eMacs in Windows, saved it to my /files/public folder on the server and tried to execute it. Lots of $’\r’: command not found.

The solution is to use dos2unix to convert the endlines, if you don’t have it, just do…

sudo apt-get install dos2unix

Then do dos2unix filename and it will modify it in place. Which is good but beware of this default behavior. It does have other options, which can be explored using dos2unix –help.

It’s one and only job is to remove CR-LF (Carriage Return-Line Feed )and just leaving LF ( Line Feed ) as UNIX/Linux wants it to be. If a file acts screwy when brought in from Windows it is most likely this issue. I even had to do it on the autosuspend.conf file!

You can always check a file with the command

cat -e filename

BAD example…

#!/bin/bash^M$
 ^M$
 # Source the configuration file^M$
 . /etc/autosuspend.conf^M$
 ^M$

GOOD example….

#!/bin/bash$
 # Source the configuration file$
 . /etc/autosuspend.conf$
 $

The caret M$ is DOS, $ is UNIX.

Emails using ssmtp

It is great that CRON and other applications send an email to the root on a Linux server, which can be read simply by using mailx from the CLI. But, what if you are not logging into the machine very often at all. Using ssmtp might work well for those situations. Even my Netgear N150 router has something similar as far as sending email. On the router, you input email account settings on it and will email you the log file and other information you would like at regular intervals. Ssmtp may be of interest to me with regards to the server at some point and I have noted it for reference.

It would be interesting and a great idea to have the server be able to send emails of certain things, issues it may be encountering.

This looks interesting, I might do this at some point….

How to send email alerts from Ubuntu Server using ssmtp