Category Archives: Server

A series of Posts that involve the entire process of setting up a file server using Ubuntu Server 12.04. Covering installing the OS, LAMP stack, Samba for file sharing with Windows machines, OwnCloud for personal cloud storage.
Also covered is waking the server using wake-on-LAN and using a CRON job script to automatically shut down the server when idle. Thrown in are some educational pieces and tips on utilities. Plus how to backup MS machines to the server by waking it up and performing the backup and then allowing the machine(s) to shutdown.

Penguin

guestbook.cgi

Linux, Server, Unpolished Gems, , ,

The best way to understand a thing is to try to comprehend it.

I ran John Callendar’s guestbook.cgi on one of my servers. It wasn’t a very popular location on the web and the guestbook did not get abused fortunately! It would be very easy for comment spammers, trollers and the like to have a field day with it.
One thing I did to keep an eye on it and see if it got posted to was a script in /etc/cron.weekly to check for updates of the guestbook. It was located on a Raspberry Pi running Raspberian.
The script requires that ssmtp or some mail program is installed.

guestbook-check.sh

#!/bin/bash

# Checks to see if the Guestbook has been written to in the past week.
 # Sends out notification if it has been written to.
 find /usr/lib/cgi-bin/guestbookrev.txt -mtime -7 -exec mail -s "Guestbook Updated" myemail@myisp.net \;

The trick is in the find command where the option -mtime -7 means check to see if the file have been has a modification time of less than 7 days, if so, then execute via the -exec option, whatever comes next on the command line until the \;

Subject line

The mail command mails only a subject line of “Guestbook Updated”. It would be possible to have something in the body and even “cat” the guestbook in to send it in the email.

Auto de Spam

If the guestbook got abused by a spammer or something nasty it might be possible to run a script that would periodically do a cleanup on the file via a search and replace. Using a list of blacklisted words to search on and then replace them with a null character or space.

Reverse Order in Guestbook

It is possible to flip the posts on the guestbook as well if you want them ordered in the opposite order, Last In On Top versus the default of First In On Top. This is done by using…

 pop @all_entries instead of shift @all_entries

…in the code.

More on CGI and Perl

If you are new to CGI and/or Perl scripts be sure to check out John Callender’s tutorial that covers the workings of guestbook.cgi.
guestbook.cgi

Linux Mail From the Command Line

Local Mail Using Postfix and Mail

Info Links, Linux, My Thoughts, Server, , , , , , ,

Mail on the command line. This was once a thing that was used much more often. When I was in college in the 1990’s it was one of the easiest ways to get mail when on campus and off. It was taught to technical and non-technical people alike as part of orientation and you were given written instructions right at registration time. Instructions came along with the email account that they made up for you using a student id number followed by @binghamton.edu. In fact, finally in the last year I was there they got rid of this cumbersome email and started allowing users to have email with their real names.

Getting to mail back then from the command line involved logging into your Unix account from one of the many terminals spread throughout the campus. Or using the  insecure but, OK at the time rlogin, remotely via dialup. Then you could either use mail or pine which was a bit more sophisticated as it was based off the pico editor, of which the popular nano editor is a derivative. It at least has somewhat of an interface that accepts up and down arrow movement and displays the shortcuts on the bottom. The standard mail program is a bit like vi, spartan but still useful.

Having access to mail locally on the command line is useful when you might be running CRON tasks or any other automated scripts that call other code as it allows you to get notified of when they have run and most importantly if they have had errors.

The other alternative is to set up ssmtp and have mail sent out of the local machine using SMTP from another established account. Of course you can also set up a full blown mail server but, that can be overkill if you are just monitoring what is happening on a few machines that you regularly log in to.

Setting up Local Mail

Below is a great Github post on how to set up local mail on a Linux Machine. I followed it and added a variation to get local mail running using the command line mail program.

It works great.  I really liked the instructions, very easy to set up. I was glad I found this as, I thought it might be tricky and with these instructions it was a few minutes on each machine.

Setup a Local Only SMTP Email Server (Linux, Unix, Mac)

 

My Additions

Server

For use with mail the program, it might not be necessary to have a local host.com in the host file. I have not tested this.

I followed the tutorial up to and including the step of restarting postfix.

Then I installed mail instead of Thunderbird.

On the server, which only has a CLI,  I wound up using mail instead of T-bird, installed via sudo apt-get install mailutils
It can be tested by sending a message to yourself by using…
mail -s “test” (your user name)@localhost

Hit enter when in the screen to bypass CC , type something and end by using Ctrl D.

Then enter the command

mail

…and you should see the email. Hit enter at the? prompt and the message is presented.

Enter q to quit.

Another test message can be sent to test if a message to any address gets sent to you. As long as the domain is local host it will work and catch it. Other domains will fall and send you an email from the system reporting the email as undeliverable.
It works great to get the CRON messages on the machine.
Just type mail and you get a CLI email list. mail basics are, q to quit, m to compose, enter and spacebar to move through messages. Entering question mark ?, brings up command help.

Desktop Install

One gotcha that caught me is that I already had T-bird installed and therefore it had a default SMTP server already. For me this required what I would call step 6A to add a local STMP server.
6A. In the pane above “Account Actions” scroll, using the bar to the bottom “Outgoing Server (SMTP)”.
Click Add
For description I wrote Local SMTP
Server Name: localhost
Port: 25
Username : (the user name)@localhost
Authentication Method: Password, transmitted insecurely
Connection Security: None

Then I went back into the account settings for the mail set up in step 6 and set the Outgoing Server (SMTP) to the Local SMTP

Host File Aliases

Also in /etc/hosts you can put in localhost.com as a alias and it works fine, like this…
127.0.0.1       localhost localhost.com

This is the way to put in aliases in a host file, for example you can have the machine name and then a shortcut to it if you have it set to a static IP. This way you can just type server to SSH to it and use that as a short name wherever you want to in scripts and etc.

192.168.1.10  Dell-Optiplex-620     server

127.0.0.1 localhost localhost.com

 

 

Create a hidden WordPress page using bash on the command line

Linux, Server, WordPress, , , , , ,

Recently I was searching around looking for a way to create a hidden page on a WordPress site. It is a hosted site, not on wordpress.com. It is on a Linux server to which I have shell access.

Initially I tried using a plugin that I found that hides pages and posts. Plugins, you got to love or hate them. Love then when they work great right out of the box, hate them when they take a long time to troubleshoot.

Rather than waste too much time with the plugin, I went straight to the command line.

Screenshot_2018-04-03_18-28-35-shows-making-hidden-page

It turns out that if you publish a page and then log into the hosting server, make a directory somewhere under your public_html, change directory into it and execute…

 wget -x -nH your-page-url-to-hide-here

 

Screenshot_2018-04-03_18-38-33-draft-or-private-wp-setting
Set to Draft or Private

…then go back it and make the page a draft or under review, so it “disappears” from the menu structure. It will still work as a “cached” HTML page that has been downloaded to the folder that you have created. It will work, pictures and what not that you have loaded in it will be fully functional.

Example of a hidden page

http://erick.heart-centered-living.org/hidden/i-am-a-hidden-page/

Once the original page is put into draft/under review or private mode, it is gone…

http://erick.heart-centered-living.org/i-am-a-hidden-page/

Caveat

I have noticed that caching can get in the way. If your server caches pages, wget may not see the page updated when you make changes. A quick remedy is to set the page to draft/pending review or private, delete the hidden page. I usually use rm -rf from the directory above it and then force it to download the “404” page. Then  you can publish the page re-run wget and it will force it to get the fresh version. Keep note of the size of the file as a hint that it is getting the right one.

Upcoming: Do this with a CGI Script

In an upcoming post, I will cover how to make a CGI script that will allow you to create a hidden page easily without having to use SSH to login to the server.

 

wget options used in this example, from the man page

-x
–force-directories
The opposite of -nd—create a hierarchy of directories, even if
one would not have been created otherwise.  E.g. wget -x
http://fly.srk.fer.hr/robots.txt will save the downloaded file to
fly.srk.fer.hr/robots.txt.

-nH
–no-host-directories
Disable generation of host-prefixed directories.  By default,
invoking Wget with -r http://fly.srk.fer.hr/ will create a
structure of directories beginning with fly.srk.fer.hr/.  This
option disables such behavior.

Wget Resources

https://www.lifewire.com/uses-of-command-wget-2201085

https://www.labnol.org/software/wget-command-examples/28750/

The Ultimate Wget Download Guide With 15 Awesome Examples

http://www.linuxjournal.com/content/downloading-entire-web-site-wget

Autoshutdown Code Modded to hybrid-sleep and allow required restarts

Linux, Server, , , , ,

Hybrid Sleep Code

I decided to use hybrid sleep instead of a suspend. I have been using the code for autoshutdown as both autoshutdown, using the shutdown command and suspending. A server that I have been using for a year now supports suspend and I have used systemctl suspend successfully with it. But, if the power goes out, the next time it is as if it was shut down and gets a fresh boot. The way around that is to use systemctl hybrid sleep which puts the RAM content into swap and then suspends. This way if the power goes out it will just resume from hibernate.

Reboot code

After setting up the machine with hybrid sleep. I realized that the mechine needs a reboot once and a while after unattended updates and thought that it would be nice to automate that process. I looked on line and found a piece of code that will reboot the machine if a reboot is required. This is done via detecting the presence of the reboot-required file. So far testing once today 8/12/2017, OK so far!

https://muffinresearch.co.uk/how-do-i-know-if-my-ubuntu-server-needs-a-restart/

Snippit of code added to autosuspend a.k.a autoshutdown code that was covered in the original post on this topic …

# If the reboot-required file is present, restart and l$
 if [ -f /var/run/reboot-required ]
 then
 logit "RESTART REQUIRED"
 echo 'Restart required' >> /var/www/html/shutdown.txt
 date >> /var/www/html/shutdown.txt
 echo "------------------------------------" >> /var/www$
 systemctl reboot
 fi

# Fall through and hybrid-sleep it!

systemctl hybrid-sleep
 # Switched to hybrid-sleep 08122017 systemctl suspend

Samba and Linux Mint

Linux, Server, ,

In my original post on Samba I covered a lot of basics. Recently I learned a bit more using Linux Mint. In Mint Samba is already loaded in the process of installing the OS. I remembered that I had to add all the computers to the same workgroup to get Linux and Windows to play together. But I couldn’t get two Mint machines to work via Samba. Name resolution was the issue.

I kept getting a “Failed to Retrieve Share List from Server” error. I was able to move files from a Mint machine to the Windows machine and then to the other Mint machine. The only thing I had to do is open up permissions on the folder to let Samba write to it. I used the Public folder under the /home/user directory. I would think that adding Samba to my user group would also work, I have checked into this and this is the answer I find…

http://unix.stackexchange.com/questions/206309/how-to-create-a-samba-share-that-is-writable-from-windows-without-777-permission

 

I did install Winbind thinking that it might help out. Windows machine can see Public folder, go into it and read and write with 777 permissions on it.

winbind is a component of the Samba suite of programs that solves the unified logon problem. Winbind uses a UNIX implementation of Microsoft RPC calls, Pluggable Authentication Modules (PAMs), and the name service switch (NSS) to allow Windows NT domain users to appear and operate as UNIX users on a UNIX machine.

https://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/winbind.html

 

I started with this post which got me sort of there…

Samba Basic – Lesson 1: Samba Simply

https://community.linuxmint.com/tutorial/view/672

 

Name Resolution with Samba

http://www.oreilly.com/openbook/samba/book/ch07_03.html

How to Fix ‘Failed to Retrieve Share List from Server’ in Ubuntu 12.04 / 11.10 when File Sharing with Windows

How to Fix ‘Failed to Retrieve Share List from Server’ in Ubuntu 12.04 / 11.10 when File Sharing with Windows

 

Using mount with bind to access usb drive via vsFTP

Info Links, Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , , ,

I have a USB stick plugged into my Raspberry Pi for external storage, mostly to put music on for the Sockso Music Server to get at. But I wanted to use it a bit more for generic storage. FTP is great, you can get to it from any machine and the command line for it is the same on Win or Linux. So I can walk up to any machine, not have to install a thing and reach into a folder with FTP.

For instance, I have an infected Windows Machine, I don’t dare stick a USB stick in it. Instead I go to the command line, ftp to the Raspberry Pi and grab the tools I need from there.

The Issue

The issue was that I tried to symlink from the ftp directory to the USB drive. vsFTP will not follow symlinks for security reasons.

The Solution

Mount the directory you want under the FTP directory using bind. /media/sda is the USB stick mount point and the whole thing gets mounted under the FTP dir using…

sudo mount --bind /media/sda/ /home/ftpuser/usb-drive/

Resources

FTP on Raspberry Pi. An easy way to make shared folders

Alternatives to FTP

https://radu.cotescu.com/vsftpd-and-symbolic-links/

Sockso Music Server on Linux

Linux, Server, , , , , ,

The Sockso Music Server is very functional and quite easy to set up in standalone or daemon mode. It is cross platform as it only depends on a Java runtime environment being installed on the target computer.

Recently I loaded it on my desktop which runs Lubuntu 14.04. I tested it out on the desktop before loading it onto my Ubuntu  server PC, which holds my music repository.

  • I will outline installing the Java run time environment needed to run Sockso on an Ubuntu machine
  • The Sockso install procedure
  • Getting it to run as a daemon
  • Getting it to find your music
  • At the bottom of the page I will have some links to resources that I followed and will provide information for running Sockso on other platforms.

The Sockso install procedure

It is not so much an install like compiling/installing, apt-get or adding a package. It is a simple old school download and drop files in a directory install.

  1. Download the Sockso zip file.  You can do steps 2 and 3 while waiting for  the download!
  2. Create /usr/share/sockso directory as root or via sudo so all files are set to root:root. ( sudo mkdir /usr/share/sockso )
  3. Create Sockso data directory /var/sockso as root or via sudo. ( sudo mkdir /var/sockso )  If sockso is terminated uncleanly, the files in this directory can get corrupted and it will need to be rebuilt
  4. Extract the files to /usr/share/sockso/ ( sudo unzip sockso-1.5.3.zip -d /usr/share/sockso/ ) I am not 100% on my unzip usage, so this command actually made a sockso-1.5.3 folder under /usr/share/sockso. Then I needed to use sudo mv .. to move all the files and dirs up one level.

 

Install Java

On my server that runs headless I performed the following after I typed in java on the command line and it told me that it was missing. It usually resides at /usr/bin/java in a Debian/Ubuntu type of file system. If it is installed it will dump out a help file. Using the command which java will also tell you if it is installed…

The program 'java' can be found in the following packages:
 * default-jre
 * gcj-4.6-jre-headless
 * openjdk-6-jre-headless
 * gcj-4.5-jre-headless
 * openjdk-7-jre-headless
Try: sudo apt-get install <selected package>

I went for version 6 headless for starters. I am not sure what the difference between all the versions are, but version 6 worked for me.

erick@ubuntuserver:/tmp$ sudo apt-get install openjdk-6-jre-headless
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  ca-certificates-java icedtea-6-jre-cacao icedtea-6-jre-jamvm java-common
  libnspr4 libnss3 libnss3-1d openjdk-6-jre-lib tzdata-java
Suggested packages:
  default-jre equivs libnss-mdns sun-java6-fonts ttf-dejavu-extra
  fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei ttf-wqy-zenhei
  ttf-indic-fonts-core ttf-telugu-fonts ttf-oriya-fonts ttf-kannada-fonts
  ttf-bengali-fonts
The following NEW packages will be installed:
  ca-certificates-java icedtea-6-jre-cacao icedtea-6-jre-jamvm java-common
  libnspr4 libnss3 libnss3-1d openjdk-6-jre-headless openjdk-6-jre-lib
  tzdata-java
0 upgraded, 10 newly installed, 0 to remove and 5 not upgraded.
Need to get 44.2 MB of archives.

 

…and so on as it installed.

Reading Java Version

If you already have java and want to view the version…

java -version

…will get you the version, such as listed on my desktop PC…

erick@Precision-WorkStation-530-MT:/var/sockso$ java -version
java version "1.7.0_91"
OpenJDK Runtime Environment (IcedTea 2.6.3) (7u91-2.6.3-0ubuntu0.14.04.1)
OpenJDK Client VM (build 24.91-b01, mixed mode, sharing)

Test run

Before making it run as a daemon I wanted to test drive it. So the following command will start it up…

sudo sh /usr/share/sockso/linux.sh --nogui --datadir /var/sockso

When you terminate it, try to shut it down clean via a sigterm when you kill the process. I have read that killing it uncleanly can screw up the data directory ( /var/sockso ). Then you have to empty the directory and rebuild it’s contents. I haven’t had it screw up the directory yet.

Running Sockso as a daemon

Running Sockso as a daemon is an advantage when you are running on a server. It will startup when the machine starts and the machine will take care of closing it down cleanly upon shutdown.

Perl script for running sockso as a daemon

After moving the Sockso files to the proper location there will be a Perl file at /usr/share/sockso/scripts/init.d/sockso

Copy the sockso run file written in perl from…

 /usr/share/sockso/scripts/init.d/sockso

…to…

/etc/init.d/sockso

…using…

sudo cp /usr/share/sockso/scripts/init.d/sockso /etc/init.d/sockso

 

Edit the file and change the directory at the top of the file to point to where sockso is installed ( /usr/share/sockso ).

Also make it executable.

sudo nano /etc/init.d/sockso


sudo chmod +x /etc/init.d/sockso

Now that it is in the init.d directory, the following should work…

sockso (start|stop|restart)

Remember to change the directory at the top of the sockso to point to the /usr/share/sockso dir.

 

Starting Sockso on boot

Follow the Steps 4,5,6 on this blog post…

https://samiux.wordpress.com/2009/07/17/howto-sockso-1-1-8-music-server-on-ubuntu-9-04-server/

I have a copy here as a PDF –> sockso-start-on-boot , just in case the link above disappears.

 

Sockso Command Prompt

Sockso comes with it’s own command prompt to administer it. help will list the commands. You can use the Sockso command line to add music to Sockso’s collections, add and delete users and perform other maintenance to it.

There is also a management webpage where you can perform the same functions as via the command line.

Finding Music

There is a command line mode for sockso where you can point it to certain directories to index music from.

Run sockso to bring up it’s command line. At it;s command line use coladd and then the path to the folder that your music is in to add it. It takes a while to do this, it is indexing it into a database so be patient. You can add multiple directories into it’s collections. If you add music to a directory in the collection, sockso will find it and add it. By default it scans directories in it’s collections every 30 minutes. I’ve tested it and it is pretty cool, dump in some music and a little while later, it’s there like magic.

coladd /home/username/Music

collist will list all the collections. coldel deletes collections.

colscan will force a scan for new collections that have been added.

Symbolic Links to Music Folder

The sockso coladd command has issues with spaces in directory names. What I have done is made a bunch of symbolic links using ln -s directory of music directory-of-music. This makes it easy to see where all the music is and sockso just has to deal with my Music directory and if I add or remove music it will figure it out on it’s own. I show an example below in the Raspberry Pi section.

Sockso on Raspberry Pi

I just ( April 2016 ) installed Sockso on my Raspberry Pi. I got the idea of sticking a USB stick into one of it’s open ports and dump my music repository on it. Them with sockso I can get to it whenever I want. Previously I had it set up on my main server that I have to use Wake on LAN to start up when I am not at home. Having Sockso on the Rasp Pi allows me to get at it instantly and saves energy by not having to run a full fledged server just to play music remotely.

Below is a tree of the Music directory that I created under my home directory. As can be seen there are symlinks without spaces that point to locations on the usb stick, mounted at /media/sda.

erick@raspberrypi ~/Music $ tree -L 1
.
├── main-collection -> /media/sda/music
└── renee-ipod-music -> /media/sda/Renee's iPod/iTunes_Control/Music/

The USB stick is formatted it’s default way that it came, FAT32. I use pmount /dev/sda1 /media/sda to mount it. In this was it is mounted not as root, it is mounted by my user, so all files are easily accessed by my own user, locally and remotely using NFS or SSHFS. In this way I can add and remove files easily.

 

 Users

In Sockso there is a concept of users. You can have multiple people logged in and have personalized settings. You can even authorize uploads by setting that option.

Adding users at the Sockso command line works similar to adding users in Linux.

useradd NAME PASS EMAIL ISADMIN 1/0     Adds a new user

Commands:
userlist                                Lists the users
useradd NAME PASS EMAIL ISADMIN 1/0     Adds a new user
userdel ID                              Deletes a user
useradmin ID ISADMIN 1/0                Sets a user to be admin/non-admin
useractive ID ISACTIVE (1/0)            Toggles users between being active or not
coladd PATH                             Adds a folder to the collection
coldel PATH                             Removes a folder from the collection
collist                                 Lists the folders in the collection
colscan DIR (optional)                  Start a collection scan
propset NAME VALUE                      Sets a property
propdel NAME                            Deletes a property
proplist FILTER                         Lists properties
version                                 Show version information
exit                                    Exit Sockso

 

Resources

Where to get Sockso, it’s official site

http://sockso.pu-gh.com/

This site is a bit dated but still helpful.

https://samiux.wordpress.com/2009/07/17/howto-sockso-1-1-8-music-server-on-ubuntu-9-04-server/

Sockso Read Me

Requirements
————

Sockso should come packaged with everything it needs to run,
all you have to do is have Java installed on your computer.
You can download the latest Java version for free from
the Sun website at: http://www.java.com

To run Sockso under Windows just double click “Run Sockso”.
Easy!

“Linux”
——-

If you’re running Linux or something similiar then you may
just be able to double click the “linux.sh” shell script.
If this doesn’t work for you then you can run this script
from a terminal with:

$> sh linux.sh

Feedback
——–

If you’ve used Sockso then I’d love to hear what you think, so
please send me some email at: rod(at symbol)pu-gh(dot)com

Running as daemon
—————–
Usage: sockso (start|stop|restart)

 

 

 

 

 

sSMTP Installing and Configuration and Use Tips

Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , , , , , , ,

Recently I was looking at creating a method of sending a warning email when ever my house temperature went below a threshold. I remembered that sSMTP was a simple way to send automated emails and CRON emails. I have some simple notes on what I did.

Installation

Very easy, just use apt-get from the command line…

sudo apt-get install ssmtp

Configuring

The configuration file (/etc/ssmtp/ssmtp.conf) can be edited using any test editor you typically use.

 

Config at /etc/ssmtp/ssmtp.conf

Below is my config file with the critical info blocked out. Lines in Red are what I modded to get ssmtp working for me.

The key pieces to get it working for me at least were…

hostname = My ISP’s domain

root = my complete email that I use at the ISP

mailhub = I looked it up in Thunderbird, it is the smtp.myispsdomain.net part.

AuthUser=my complete email that I use at the ISP. It might be different for you. Years ago it used to be just the user name part of email without the domain.

AuthPass = The password that goes along with my email.

I commented out the defaults for the ones that existed in the code.

The config file is a bit ugly after I touched it but I was trying to get this up and running quick and didn’t clean it up. But, hey it works!

 

#
 # Config file for sSMTP sendmail
 #
 # The person who gets all mail for userids < 1000
 # Make this empty to disable rewriting.
 #root=postmaster  <--- comment out
 
# The place where the mail goes. The actual machine name is required no
 # MX records are consulted. Commonly mailhosts are named mail.domain.com
  #mailhub=mail <-- comment out
 
# Where will the mail seem to come from?
 #rewriteDomain=
# The full hostname
 #hostname=raspberrypi <--- I was testing and kill this, failed to work
 # hostname has to be the mail domain! Or else it complains about
  # the raspberrypi part! The STMP server at frontier does that is.
  hostname=myispdomain.net
# Are users allowed to set their own From: address?
 # YES - Allow the user to specify their own From: address
 # NO - Use the system generated From: address
 #FromLineOverride=YES <-- Commented out and set below, I was testing!
# New Code put here 11302015
  root=me@myispdomain.net
  mailhub=smtp.myispdomain.net
AuthUser=me@myispdomain.net
AuthPass=myemailpassword
FromLineOverride=YES
#UseSTARTTLS=YES <-- Tried this, I didn't need it for my ISP.

CRON Email

Once installed if you or root on the machine have any CRON jobs, you will start to get email from them. You can stop this by appending …

> /dev/null 2>&1

to the end of the commands that are being run by CRON. Which will cut back on the emails that you will receive.

 Testing

I installed mail utils to allow sending simple messages…

sudo apt-get install mailutils

Then I sent a message via the command line…

echo "Test" | mail -s "Test Subject" me@myispsdomain.net

…and I was able to see it work OK.

Send files via email

If you want to send files you have to install mpack.

sudo apt-get install mpack

 

Then you can send files to your email like this…

mpack -s "Test" /tmp/web/log.txt me@myispsdomain.net

 Command Line Usage

If you execute ssmtp with an email address it will let you create an email from the command line. Which is good for quick emails to for example remind yourself of something, or send a snippet of code to yourself. You edit the email in the form of the example below and hit Ctrl-D when done and then it will send out.
ssmtp recipient_email@example.com
The following is an example right off the command line. Note the one line of space after the Subject, this is a must have…
erick@raspberrypi ~ $ ssmtp me@myispdomain.net
To:me@myispdomain.net
From:me@myispdomain.net
Subject:This is a test of ssmtp from the command line!

Hello there this is a test of the ssmtp from the command line tool. It could be used to send a reminder or small snips of code. Use Ctrl-D when you are done.

It is called up by using ssmtp emailtosendto@domain.com

Bye,
Me

Example of Sending CPU Temp Warning Emails

When I am away from home I can infer if my house is running to cold, which may indicate a problem with the furnace. The Raspberry Pi is light loaded, usually just idling, so the CPU temperature tracks the room temperature, with an offset. When I am away, I set the house thermostat at 47 degrees F. If it drops below this value the CPU temperature of the Raspberry Pi will drop below 34 degrees Celsius. So I can just have it send me an email if this happens. Then I can double check a log that is created of the temperature reading to see what is going on. Also I run a webcam pointed at an actual thermometer for a sanity check, this is logged by using fswebcam to take an hourly snapshot. So I have my bases covered for the most part. Obviously if the power is out, I am in the dark about the temperature, because the whole thing is down! Solving that is a future project.

Below is the snippet of code from a shell script that sits in /etc/cron.hourly that handles the warning emails that are sent to 2 addresses. variables mailaddr and mailaddr2.

temp is the CPU temperature in Celsius as an integer stripped using cut from the thermal_zone0 reading.

minimum and maximum are my temperature thresholds. I don’t care much about maximum but I have it set at 65 Deg. C. just in case.

# Read the temp and cut it to grab leftmost 2 characters, integer Temp
temp="`cat /sys/class/thermal/thermal_zone0/temp | cut -c1-2`"
#echo $temp

# Mail if about or below the limits
if (( $temp > $maximum )); then
   #echo "above"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr2

elif (( $temp < $minimum )); then
   #echo "below"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr2

fi

Boot Email

I want to know if an when the Raspberry Pi I run 24/7 ever reboots due to a power outage, so I have it send me an email. The line of code below handles it and is in the root crontab. I have it sleep for 180 seconds first, then send the email. This allows the cascaded routers which I have the Pi connected to and the cable modem, time to come on line.

@reboot sleep 180 && echo "Rasp Pi Rebooted" | mail -s "Rasp Pi Reboot!" me@myispsdomain.net

I also log boots in a file that I can view online, just to keep track in one record.

@reboot date >> /var/www/bootlog.txt

Keeping track of boots helps for instance if I am away from home and the power goes out. If I get the email that the Pi rebooted, I can check to see how long the power was down and what the temperature of the house is to see if all is well.

Every hour I take a time/date stamped webcam snapshot of a thermometer so I can just look to see how many are missing and have a rough estimate of how long the power was out and how cold the house got and verify that it is getting warmer because the furnace is on!

In the future I will connect a BME280 sensor to the Raspberry Pi that will be able to read ambient room temperature directly, along with humidity and barometric pressure. So I won’t have to infer the house temperature via the CPU temperature.

Resources

This is the page I used to configure ssmtp on the Rasp Pi.

http://www.raspberry-projects.com/pi/software_utilities/email/ssmtp-to-send-emails

FTP on Raspberry Pi. An easy way to make shared folders

Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , , , , , ,

The idea with FTP is to have folders that can be reachable between Linux and Windows, locally and remotely and easily. FTP is not secure, but it can be made secure, that info can be found on the web. For now I am covering the basics of FTP here.

For most things that I need to do, I don’t need the files to be secure anyways, 90% of the time nothing critical is going back and forth across remotely. If it is I would use a secure method of sending files via SSH via SFTP or an SSHFS.

FTP is an old protocol but it just plain works and is compatible with Windows, Linux and Mac. I have tried WebDAV in the past but it is compatible to only a degree with various Windows operating systems. I have had a hard time getting it working correctly on versions of Windows beyond XP, resorting in installing patches to Windows and etc. Generally not easy to implement.

I was also looking at FTP as a native tool typical of server installs. I have experimented with cloud setups such as OwnCloud and Sparkleshare, but with FTP I was looking for something simple and quick to setup, no special software, no mySQL databases running on the Raspberry Pi, no special software on client PCs, that sort of thing.

vsFTP

sudo apt-get install vsftpd

Edit the configuration file

Back it up first then do an edit.

sudo cp /etc/vsftpd.conf /etc/vsftpd.orig
sudo nano /etc/vsftpd.conf

uncomment local_enable = YES

uncomment write_enable = YES

Find this and check that it is set this way…

local_umask=022

Enabling PASV

I have read online that enabling the PASV capability for FTP is a good idea. Frequently when I have FTP’d to various ISP’s sites I have seen them operate in PASV mode. So it stands to reason that if the pro’s are have it set up that way it may have it’s advantages.

Add the following lines to the /etc/vsftp.conf file.

pasv_enable= Yes
pasv_min_port=40000
pasv_max_port=40100

There is nothing magic about the numbers of the port range other than they should be unused by anything else that your setup might require and generally I have seen high numbers used commonly. To work out side of your local network you must enable port forwarding of the range of port numbers through your router configuration.

Changes to vsFTP

With the newer versions of vsFTP there is a change that has occurred since I wrote my previous post about vsFTP (  http://oils-of-life.com/blog/linux/server/additional-utilities-for-a-linux-server/ )

The change has to do with the fact that the root directory of the user has to be non-writable and I have read online that it is best to make it owned by root as well. This is covered below, after the section on adding a user. You need to have a user first before modifying their permissions!

FTP User

To create an FTP user, create it in a way that it does not have a login shell. So that someone who can log in to the FTP account can’t execute shell commands. The line /sbin/nologin may not be in the /etc/shell file and in that case it needs to be added in there. The user basically has to be jailed in their directory and has to have no login shell.

sudo useradd -m -s /sbin/nologin -d /home/user user

I added Documents, public_html directories to the /home/user as well. Then made the users root folder /home/user, owned by root and nonwritable.

cd /home/user
chown user:user Documents
chown user:user public_html

chown root:root /home/user
Make Root of user non writable
sudo chmod a-w /home/user

FTPing on the PC

Now that ftp is set up on the server you will want to be able to connect to it!

Options for connecting…

Command Line, WIndows and Linux

ftp yoursite.com

That gets you into FTP via the command line. The command prompt will now start with ftp> ,that is how you know that you are within the ftp command shell.

It is archaic, but worth knowing when you have to stick a file up or pull it down right at the command line. The commands the ftp prompt accepts are basic, but good enough to get most work done. Type help at the prompt to get a list of commands.

Via Folders

Linux

Just enter the location of the ftp server right into the top of the directory folder and you will be prompted for a password and taken there.

Windows
Windows7/Vista:
  1. Open Computer by clicking the “Start” button, and then clicking Computer.
  2. Right-click anywhere in the folder, and then click Add a Network Location.
  3. In the wizard, select Choose a custom network location, and then click Next.
  4. To use a name and password, clear the Log on anonymously check box.

From: https://www.google.com/search?q=connect+to+ftp+windows+7&ie=utf-8&oe=utf-8

 

 

Automatic Server Status Page Creation Update

Linux, Raspberry Pi, Raspberry Pi Server, Server, , , , , , , , , , , , , , , , ,

In January 2015 I created a post about automatically creating a status page for a Linux server that I have. Typically this is put under a restricted directory and allows you to see a snapshot of what is happening with the server. I run it by putting the scripts in the /etc/cron.hourly directory on a Linux PC and a Raspberry Pi running Linux.

It serves as a simple way to check up on the server without having to use a tool such as Webmin that requires a login. It also keeps a trail of log files that get rotated on a monthly basis, so there is always a few old ones around to track down any problems and patterns in the operation.

I have found this information useful when I have traced down malfunctions that can occur when setting up a server and also when I was trying to get a webcam up and running and had the USB bus hang up a few times when the cam was overloaded with too much light.

In the new script file I fixed a bug by adding parenthesis around a line that I was trying to echo and I added code to run the w command to show a quick picture on who is logged in, how long the server has been up and running and the values for the average load on the server at the 1, 5 and 15 minute marks.

Logcreate Script

#!/bin/dash
# Remove old log
rm /var/www/status/log.txt
# Print logged outputs into new log.txt
date >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
tail /var/log/syslog >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
free >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
df -h >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> /var/www/status/log.txt
ps aux | sort -nrk 4 | head >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Logged in User info using w command
w >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Copy log.txt into the full log that is collected
cat /var/www/status/log.txt >> /var/www/status/fulllog.txt
# Create a free standind copy of the process tree
pstree > /var/www/status/pstree.txt

Alternate Version

I also created a version of the script for a desktop Linux PC that does not have Apache installed.  In it I use a DIR variable to contain the directory that I want the log.txt file stored.

 #!/bin/dash

# User defined variables
# No trailing / on DIR!
DIR=/home/erick/status

# Remove old log
rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt
ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/fulllog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

Rotation of Log

In the /etc/cron.monthly directory I have created a file that is called status-log-rotate and it will save backup copies of 2 months worth of the full concatenated server status logs.

#! /bin/bash
DIR=/home/erick/status
mv $DIR/fulllog.txt.1 $DIR/fulllog.txt.2
mv $DIR/fulllog.txt $DIR/fulllog.txt.1

Tweaks for Raspberry Pi

For the Raspberry Pi which has an SD card that I am trying to be conscious of writing to often. I have recently made some modifications to put the /tmp folder onto RAM using tmpfs. I create the hourly log underneath a folder there. Daily via a script it cron.hourly it gets concatenated into a daily log which is under a status folder that has restricted access. This gets appended once per day to the fulllog which actually lives on the SD card. The end result, no multiple hourly writes to the log file, just one append to the full log per day. The only downside is if the power drops and then some log entries will be lost for the day.

Logcreate runs from /etc/cron.hourly for Raspberry Pi

#!/bin/dash
# Set DIR, on Pi this is a temp location for log
DIR=/tmp/web

# Set fixed DIR FIXDIR for files that have to be stored on SD card
# Nevermind, just make a daily log and then copy that to the full log daily.
#FIXDIR=/var/www/status

# Remove old log

rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt

ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/dailylog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

dailylog-to-fulllog script, runs from /etc/cron.daily

#! /bin/bash

DIR=/tmp/web
FIXDIR=/var/www/status

echo "----------------------------------------------" >> $DIR/dailylog.txt
date >> $DIR/dailylog.txt
echo "----------------------------------------------" >> $DIR/dailylog.txt
cat $DIR/dailylog.txt >> $FIXDIR/fulllog.txt
rm $DIR/dailylog.txt

Logcreate Output from Raspberry Pi

Below is what the logcreate script will output to the log.txt file on a Raspberry Pi that I have running as a web server.

Sun Jul 12 14:17:01 EDT 2015

Jul 12 13:47:51 raspberrypi dhclient: DHCPACK from 192.168.1.1
Jul 12 13:47:52 raspberrypi dhclient: bound to 192.168.1.17 -- renewal in 40673 seconds.
Jul 12 13:59:01 raspberrypi /USR/SBIN/CRON[28010]: (erick) CMD (aplay /opt/sonic-pi/etc/samples/guit_e_fifths.wav)
Jul 12 13:59:07 raspberrypi /USR/SBIN/CRON[28009]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:00:01 raspberrypi /USR/SBIN/CRON[28013]: (erick) CMD (/home/erick/fswebcam/cron-timelapse.sh >> timelapse.log)
Jul 12 14:00:23 raspberrypi /USR/SBIN/CRON[28012]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:01:01 raspberrypi /USR/SBIN/CRON[28022]: (root) CMD (/home/erick/bin/usbreset /dev/bus/usb/001/004)
Jul 12 14:01:02 raspberrypi /USR/SBIN/CRON[28021]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:09:01 raspberrypi /USR/SBIN/CRON[28053]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Jul 12 14:17:01 raspberrypi /USR/SBIN/CRON[28064]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

             total       used       free     shared    buffers     cached
Mem:        445804     424488      21316          0     106768     260516
-/+ buffers/cache:      57204     388600
Swap:       102396          0     102396

Filesystem      Size  Used Avail Use% Mounted on
rootfs          6.3G  3.1G  3.0G  51% /
/dev/root       6.3G  3.1G  3.0G  51% /
devtmpfs        214M     0  214M   0% /dev
tmpfs            44M  240K   44M   1% /run
tmpfs           5.0M  8.0K  5.0M   1% /run/lock
tmpfs            88M     0   88M   0% /run/shm
/dev/mmcblk0p5   60M   19M   41M  32% /boot

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      2071  0.0  3.0  24896 13652 ?        Ss   Jun28   2:24 /usr/sbin/apache2 -k start
www-data 27745  0.0  1.5  25412  7084 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27744  0.0  1.5  24960  6760 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27743  0.0  1.5  25428  7116 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27742  0.0  1.5  25396  7036 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27538  0.0  1.5  25396  7032 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27502  0.0  1.5  25404  7036 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27501  0.0  1.5  25396  7044 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27747  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27746  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start

 14:17:02 up 14 days, 12:56,  1 user,  load average: 0.00, 0.01, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
erick    pts/0    192.168.1.5      14:04   10:39   1.70s  1.70s -bash