Daily Archives: October 31, 2014

UNIX and DOS endlines

I had a moment where I forgot about the entire UNIX and DOS endline incompatibility issue. So when I grabbed the autosuspend script with copy and paste and I brought it into eMacs in Windows, saved it to my /files/public folder on the server and tried to execute it. Lots of “$’\r’: command not found” errors.

The solution is to use dos2unix to convert the endlines, if you don’t have it just…

sudo apt-get install dos2unix

Then do dos2unix filename and it will modify it in place. Which is good but beware of this default behavior. It does have other options, which can be explored using dos2unix –help.

Dos2unix has one and only job, to remove CR-LF (Carriage Return-Line Feed )and just leaving LF ( Line Feed ) as UNIX/Linux wants it to be. If a file acts screwy when brought in from Windows it is most likely this issue. I even had to do it on the autosuspend.conf file!

You can always check a file with the command

cat -e filename

BAD, Windows/DOS example…

#!/bin/bash^M$
^M$
# Source the configuration file^M$
. /etc/autosuspend.conf^M$
^M$

GOOD , UNIX/Linux example….

#!/bin/bash$
# Source the configuration file$
. /etc/autosuspend.conf$
$

The caret M$ is DOS, $ is UNIX.

The Linux File System In General

A website that  has a overview of the Linux file system can be found at…

http://www.tldp.org/LDP/intro-linux/html/chap_03.html

 

Auto-shutdown and Auto-suspending a Linux Server

The article below talks about auto-suspending…

http://rolandtapken.de/blog/2013-07/suspend-nas-when-idle

A Simpler Idea

I found another article on auto-suspending that requires only a simple bash script that I have placed in /etc/cron.hourly.

WordPress did not like me uploading autosuspend.sh, for security reasons, it will give an error, so I have the script autosuspend.sh , named autosuspend.sh.txt  here->  autosuspend.sh . The file goes in /etc/cron.hourly naming it just autosuspend, cron won’t run if the filename has an extension.

The file must be owned by root and executable. So you have to use the following commands before running it.

sudo chown root:root autosuspend.sh
sudo chmod u+x autosuspend.sh

I used it as autosuspend.sh and ran it a few times manually running sudo autosuspend.sh, just to see it run properly before sticking the file renamed as autosuspend and placed it into /etc/cron.hourly.

And the autosuspend.conf  named as autosuspend.conf.txt here-> autosuspend.conf  goes in the /etc/ directory.

Both are UNIX formatted files, modify them accordingly for your use.

syslog

CRON logs things when it runs autosuspend into /var/log/syslog, so you can execute…

tail /var/log/syslog

…to see if everything is OK by seeing the traces, the autosuspend script  gives good useful error messages. It also will send an email on the server to root@yourservername, every time it runs. You can use mailx from the CLI ( or some other program ) to read the local email. Mailx is very simple and good enough to quickly page through CRON emails, using return to move down through the unread ones.

Resources

The article I got the script from is…

https://bbs.archlinux.org/viewtopic.php?id=157268

…it does not give you the autosuspend.conf file, but it references another article in German…

http://wiki.ubuntuusers.de/Skripte/AutoSuspend

…that does have the autosuspend.conf file and it seems to work, at least it runs fine so far with some mods.

Files

Once again below are the script and conf file from those sites, labeled with a txt extension. I put them here in case those sites disappear for some reason. This is good knowledge and it works so well, I’d hate to see it get lost.

The script taken from the Archlinux page, requires systemd and uses systemctl suspend to suspend the machine, named autosuspend.sh.txt. Formatted for UNIX/Linux.

autosuspend.sh

Original autosuspend.sh that uses pm-utils from the German ubuntuusers.de site, named as pm-utils_autosuspend.sh.txt and the autosuspend.conf named as autosuspend.conf.txt. Formatted for UNIX/Linux

pm-utils_autosuspend.sh
autosuspend.conf

Auto Shutdown – Mods

I decided to modify the autosuspend.sh file rather than loading the package that it needed (systemd) to execute systemctl suspend, which is what the script file from the first article uses. The other option would be to use pm-utils as the second German article has the original autosuspend.sh formatted to use. For more info on pm-utils see https://wiki.archlinux.org/index.php/pm-utils

Instead of auto suspending, I decided that since the server starts fast enough from a cold boot (17 secs. to usable), why not just replace the…

systemctl suspend

…line with…

shutdown -P +5

This will shut the server down, with a 5 minute warning and guard band. I say guard band, because it can guard against a potential loop. If I play with the script more and make a mistake, I do not want to wind up with a server that starts, jumps to the script and starts shutting down immediately. I know I put the file in /etc/cron.hourly, so it will kick off every hour, but I am just guarding against unforeseen things to be safe and it’s only 5 minutes of delay. If it goes to shut down while testing at some point, I have 5 minutes to execute a shutdown -c to cancel.

I also put the line…

ethtool eth0 -s wol g

…before the shut down line, because that same piece of code, which I tried put into rules.d. But it was not setting the wake on to g, When I ran ethtool, it was staying at d. Not sure why, but since I will be allowing this server to shutdown by itself 90%+ of the time, I opted to put it right in the shutdown script. After a second thought, I also put that line into the /etc/rc.local ( which runs at start up ) as well so it is armed even if I shutdown manually! See the post of Wake On Lan via Ubuntu Linux for more info on Wake on LAN.

Here is the modded autosuspend called autoshutdown.txt. Remove the txt extension and place into the /etc/cron.hourly folder, it is formatted for UNIX.

I forgot about the UNIX and DOS endlines being different while I was working on this. See my post on UNIX vs DOS file endlines, as I had a bit of brain fog and struggled a bit with this while working on the autoshutdown script.

Winbind

Once I got the autoshutdown running. I realized that the Linux machine was not able to resolve the names the Windows machines on the network. The server could only ping the Windows machines by IP address and not their names! I saw this when I was logged out of the server and logged in a while later and the shutdown script had recorded failed pings into syslog, when checking to see if the server was idle. The script correctly saw that no one is logged it by executing, who | wc -l yielding a zero and next it was testing for attached clients ( the Windows machines named in the autosuspend.conf file) using ping $i -c1. And ping was failing as the names were unreachable.

  • arp -a could see all the machines by IP address from both Linux and Windows.
  • net view on the Windows machine could see all the machines by name.
  • smbstatus can see every computer by name fine from my Linux server machine. Particularly since I had installed Samba, the servers name is visible from Windows PC’s due to Samba.
    Samba must send out net-bios information about itself, I see in the config file for Samba where it can act as a wins server as well.

In order for the autosuspend/shutdown script to work pinging by name is a must. To fix, install winbind and configure /etc/nsswitch.conf.

sudo apt-get install winbind

In /etc/nsswitch.conf add wins to the end of the line that starts with
hosts: Mine now reads…

hosts: files dns wins

I got the info from…
http://www.serenux.com/2009/09/howto-configure-ubuntu-to-be-able-to-use-and-respond-to-netbios-hostname-queries-like-windows-does/

Samba Connected test in shutdown script

The autosuspend script does a test to see if anyone is accessing files using Samba via smbstatus. Smbstatus is great to see what is going on, it is good to troubleshoot Samba when you can make connections. It is interesting once you play with it when various computers are accessing the server to understand what it is telling you.
But the script is just looking to see if computers are accessing Samba
shares. The autosuspend.conf shows an IP address for the test using
$SAMBANETWORK as that value and grepping on it. I am not sure how this works as I don’t see any IP numbers when I run smbstatus. So for now I decided that I will use the word Public in the autosuspend.conf instead of 192.168.1. Most likely if a computer is accessing Samba shares on my network and the computers name is not one of the “clients” ( my own machines at home, that have listed names) it is going to be only accessing the Public Samba share. For now this seems to work!

Test used in autosuspend script to look for machines accessing Samba…

/usr/bin/smbstatus | grep $SAMBANETWORK | wc -l
Other conditions for shutdown

The other two tests that autosuspend does (IsRunning() and
IsDaemonActive() ), I have not validated them.

That is a TBD. So far, so good, the server has not shutdown unexpectedly and I have not seen it held up by IsRunning() yet, based on it’s tests. If something is running and a shutdown occurs, a sigterm is generated as the system is going down, so anything in process should terminate cleanly in theory. I’d like to test for OwnCloud activity at some point, I have shut the machine down and restarted a few minutes later on purpose with an OwnCloud file transfer in progress and it picks back up. I have to figure out a test for this, TBD.

IsRunning() tests for the following applications…

 APPLICATIONS='"^nxagent$" "^rsnapshot$" "^wsus$" "^wget$" "^screen$" "^mlnetp$" "^apt-get$" "^aptitude$" "^dpkg$" "^cp$"'

The following features were not copied from the autosuspend.sh on the German site to the one on https://bbs.archlinux.org/viewtopic.php?id=157268 that I used…

# Turning suspend by day (8 a.m. to 3 a.m.) off
 DONT_SUSPEND_BY_DAY='no'
# Automatically reboot once a week when the system isn't in use
 REBOOT_ONCE_PER_WEEK='yes'
  • DONT_SUSPEND_BY_DAY seems to control suspending by blocking it out during the day between 8AM and 3PM, it uses /sys/class/rtc/rtc0/wakealarm. I wasn’t interested in this so I was fine with it being carved out.
  • REBOOT_ONCE_PER_WEEK uses cat /proc/uptime | cut -d’ ‘ -f1-1\` / 3600 / 24 )>= 7\ as a test to see if the machine has been running for more that one week and then it reboots the next time it is idle. This is not of interest to me as my machine will shutdown rather than suspend, so this is not needed either.

Interestingly, I do see a test to see if power management is supported in the original autosuspend.sh that relies on pm-utils. This does not exist in the modified script that uses systemctl, perhaps it is not neccessary as calling systemctl is fine without or it was omitted, because such a test does not exist when using systemctl.

  /usr/bin/pm-is-supported

Basically I am fine with the simpler script, if I need to add features back in, so be it!

I have been using the shutdown script for over a month with no issues so far.

Follow Up

I have been using this code on two servers, one for almost three years and one for a year. The older one does not suspend and it requires a shutdown and the newer one suspends nicely via systemctl suspend.

I decided to modify the code a bit to allow a hybrid-sleep and also allow for restarts when the system requires them. Read more about this here….

Autoshutdown Code Modded to hybrid-sleep and allow required restarts