Tag Archives: Perl

Penguin

guestbook.cgi

Linux, Server, Unpolished Gems, , ,

The best way to understand a thing is to try to comprehend it.

I ran John Callendar’s guestbook.cgi on one of my servers. It wasn’t a very popular location on the web and the guestbook did not get abused fortunately! It would be very easy for comment spammers, trollers and the like to have a field day with it.
One thing I did to keep an eye on it and see if it got posted to was a script in /etc/cron.weekly to check for updates of the guestbook. It was located on a Raspberry Pi running Raspberian.
The script requires that ssmtp or some mail program is installed.

guestbook-check.sh

#!/bin/bash

# Checks to see if the Guestbook has been written to in the past week.
 # Sends out notification if it has been written to.
 find /usr/lib/cgi-bin/guestbookrev.txt -mtime -7 -exec mail -s "Guestbook Updated" myemail@myisp.net \;

The trick is in the find command where the option -mtime -7 means check to see if the file have been has a modification time of less than 7 days, if so, then execute via the -exec option, whatever comes next on the command line until the \;

Subject line

The mail command mails only a subject line of “Guestbook Updated”. It would be possible to have something in the body and even “cat” the guestbook in to send it in the email.

Auto de Spam

If the guestbook got abused by a spammer or something nasty it might be possible to run a script that would periodically do a cleanup on the file via a search and replace. Using a list of blacklisted words to search on and then replace them with a null character or space.

Reverse Order in Guestbook

It is possible to flip the posts on the guestbook as well if you want them ordered in the opposite order, Last In On Top versus the default of First In On Top. This is done by using…

 pop @all_entries instead of shift @all_entries

…in the code.

More on CGI and Perl

If you are new to CGI and/or Perl scripts be sure to check out John Callender’s tutorial that covers the workings of guestbook.cgi.
guestbook.cgi

Penguin

Pastepile

Linux, Unpolished Gems, Web Hosting, , , , , , , , ,

Future development depends on the determination of future generations to contribute towards building their own society.

Pastepile is a fork of guestbook.cgi – a simple guestbook script, written by John Callender in 1999. It is the last part of his Beginner’s Guide to CGI Scripting with Perl, Running a Guestbook.
This new version, a.k.a Pastepile, is a dirt simple Perl CGI program that creates a blog like pasting tool to be able to paste notes online or on a local server. Allows grabbing snippets of text or code for later use on whether on the local machine or another on the network. I made it sometime after studying John Callender’s tutorial when I was digging into learning about CGI and Perl several years ago.

History

Pastepile originated as a way to paste snippets of code and write short notes on configuration changes that I make on servers, including a Raspberry Pi that I run. The only requirement is the installation of a web server such as Apache. This little Pastepile tool made it easy to keep track of information related to the servers in one place  and search-able via a browser. After using it in the is mode for a while, I cloned it and started to use it as a general place to paste info and write short notes. This makes it easy to save something while on one machine on the network and be able to retrieve it later on that machine or another.

Example of What it Looks Like

Subject: Running a form-to-email gateway at 19:04:44, Sat Feb 3, 2018 from 192.168.1.7

http://www.resoo.org/docs/perl/begperl/form_to_email.html

Subject: Slackware Linux Essentials at 12:10:03, Thu Feb 1, 2018 from 192.168.1.179

www.slackbook.org/html/index.html

Additions:Remote Address,Time,Reversed Order

Besides stripping down guestbook.cgi to remove code specifc to it’s guestbook function and cleaning up the code to reflect it’s new use, three functional code changes were made. One is to add a timestamp to keep track of when the entry was created. Another change made was showing the IP of the machine the post originates from by reading the environment variable $REMOTE_ADDR to keep track of where the post originates from. Many machines on my network have static IP’s so this is handy for me at least. The final change was to reverse the order of the listing. In guestbook.cgi a First In On Top ordering is the layout of the guestbook posts. For Pastepile I reversed it so the most recent and not the oldest post is on top of the list, Last In On Top ordering. This made sense as the most recently written notes are more likely to be important and I want those to be close to the top of the pile and let the file get long with the aging information at the bottom.

/p/

For ease of access via a shorter link a simple index.html redirector page was place in /var/www/p/, following a 4chan-esqe style of folder naming and a nice short link to get to the pastepile.cgi script, which lives in the /usr/lib/cgi-bin/ directory. The /p/ directory also holds the pastepile.html file that is created by pastepile.cgi as it’s data page.

 Redirector Example for /var/www/p/index.html

<HTML>
<HEAD>
<meta http-equiv="refresh" content="0; url='http://192.168.1.17/cgi-bin/pastepile.cgi'"/>
</HEAD>
<BODY>
<p><a href="http://192.168.1.17/cgi-bin/pastepile.cgi">Redirect</a></p>

</BODY>
</HTML>

paste-pile-mover.sh

A helper script called pastepile-file-mover.sh, moves the pastepile.html, the data file created by pastepile.cgi from it’s default location to a date stamped file in a location set in the script BASEDIR/dir/filename. Where BASEDIR is your choice ( I use /var/www/p/), dir is the current year and filename is YYYYMMDD.html, so that that there is a  year and date hierarchy. I allow this script to run at the start of the month via root CRON to “clean” out the Pastepile and archive the old one, much the same way that log are rotated.

0 0 1 * * /home/erick/bin/pastepile-file-mover.sh

pastepile-file-mover.sh

#!/bin/bash

# Move the pastepile.html file from it's default location to a date stamped
# file in a location BASEDIR/dir/filename
# So that it has year and date heirarchy

# For now, Monthly, Move pastepile over to year dir and datestamped HTML file
#0 0 1 * * /home/erick/bin/pastepile-file-mover.sh



BASEDIR=/var/www/p
# Testing
#BASEDIR=/tmp

dir=$(date +"%Y")
#echo $dir

#File name timestamped
filename=$(date +"%Y%m%d").html
#echo $filename

# Make the YEAR dir if it dow not exist.
if [ ! -d "$BASEDIR/$dir" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
  mkdir $BASEDIR/$dir
fi

# Do the move to the YEAR directory with the YYYYMMDD.html filename.
mv $BASEDIR/pastepile.html $BASEDIR/$dir/$filename

# This is needed make a new pastepile.html and chmod 666
# else pastepile.cgi does not work, it can't make it's own output file.
touch $BASEDIR/pastepile.html
chmod 666 $BASEDIR/pastepile.html

Last but not least pastepile.cgi

#!/usr/bin/perl -Tw

# pastepile.cgi a fork of...
# guestbook.cgi - a simple guestbook script

# This program is copyright 1999 by John Callender.

# This program is free and open software. You may use, copy, modify,
# distribute and sell this program (and any modified variants) in any way
# you wish, provided you do not restrict others from doing the same.

# pastepile.cgi - guestbook.cgi, modded to become # a pastepile program. Erick Clasen Jan 25,2018
# This new version is a dirt simple CGI program that creates a blog like
# pasting tool to be able to paste notes online or on a local server.
# allows grabbing snippets of text or code for later use on whether
# on the local machine or another on the network.

$data_file = '/var/www/p/pastepile.html';

$max_entries = 0; # how many guestbook entries to save?
                   # set to '0' (zero) for infinite entries...

use CGI;
use Fcntl;
$query = new CGI;

unless ($action = $query->param('action')) {
    $action = 'none';
}

print <<"EndOfText";
Content-type: text/html

<HTML>
<HEAD>
<TITLE>Raspberry Pi Server Paste Pile</TITLE>
</HEAD>
<BODY>
<H1>Raspberry Pi Server Paste Pile</H1> 
<P><EM>$ENV{DOCUMENT_ROOT}/p/</EM></P>



<A HREF="../status/index.html">Back to Status Index</A>&nbsp;
<A HREF="../p/2018">2018 Paste Archive</A>

<P>You can <A HREF="#form">add your own subject and entry</A> using the form at the bottom of the page. Here we is has your pastes...</P>

<HR>
EndOfText

# Input action to add a new entry. ----------------------
if ($action eq 'Add entry') {

    # process the form submission
    # and assemble the guestbook entry


    # Input the subect and the paste which is called a comment here in this code.
    $subject = $query->param('subject');

    $comment = $query->param('comment');

    # clean up and fiddle with $subject
 unless ($subject) {
        $subject = 'No Subject';
   if (length($subject) > 50) {
        $subject = 'Subject line too long >50 chars';
    }
# End Input action to add a new entry. ----------------------



    }

    # Add a time stamp, put in variable theTime. This allows the paste to be timestamped.
    
    @months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
    @weekDays = qw(Sun Mon Tue Wed Thu Fri Sat Sun);
    @digits = qw(00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 59 59);
    ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek) = localtime();
    $year = 1900 + $yearOffset;
    $theTime = "$digits[$hour]:$digits[$minute]:$digits[$second], $weekDays[$dayOfWeek] $months[$month] $dayOfMonth, $year";

    # untaint variable
    unless ($theTime =~ /^([^<]*)$/) {
        die "couldn't untaint name: $theTime\n";
    }
    $theTime = $1;

    

    # clean up and fiddle with $subject--------------------------------

    $subject_clean = "$subject";
    $subject_clean =~ s/, , /, /;        # remove duplicate ', '
    $subject_clean =~ s/^, //;           # remove initial ', '
    $subject_clean =~ s/, $//;           # remove final ', '
    if ($subject_clean =~ /^[,\s]+$/) {
        # nothing but commas and whitespace
        $subject_clean = 'Subject format wrong!!';
    }
    
    if (length($subject_clean) > 75) {
        $subject_clean = 'Subject too long.';
    }

    # disable HTML tags
    $subject_clean =~ s/</&lt;/g;

    # untaint variable
    unless ($subject_clean =~ /^([^<]*)$/) {
        die "couldn't untaint subject_clean: $subject_clean\n";
    }
    $subject_clean = $1;

    

    # clean up and fiddle with $comment----------------------------

    if (length($comment) > 32768) {
        $comment = '...overloaded blog buffer chars > 32768.';
    }
    unless ($comment) {
        $comment = '...nothing to speak of.';
    }

    # fix line-endings
    $comment =~ s/\r\n?/\n/g;

    # lose HTML tags
    $comment =~ s/</&lt;/g;

    # untaint variable
    unless ($comment =~ /^([^<]*)$/) {
        die "couldn't untaint comment: $comment\n";
    }
    $comment = $1;
    # end cleanup #comment-----------------------------------------

    

    # assemble finished guestbook entry -- anything after this line until EndofText will show in the post!!!!!

    # Enviroment variable for REMOTE_ADDR is grabbed and printed directly. No untainting, but probably safe to do.

    $entry = <<"EndOfText";

<P><STRONG> Subject: $subject_clean </STRONG> at <EM>$theTime </EM> from <EM>$ENV{REMOTE_ADDR} </EM> <BR>
<BLOCKQUOTE>$comment</BLOCKQUOTE></P>
<HR>
EndOfText

    # open non-destructively, read old entries, write out new
   $all_entries .= $entry;
    sysopen(ENTRIES, "$data_file", O_RDWR)
                             or die "can't open $data_file: $!";
    flock(ENTRIES, 2)        or die "can't LOCK_EX $data_file: $!";
    while(<ENTRIES>) {
        $all_entries .= $_;
    }

 if ($max_entries) {

          # lop the tail off the guestbook, if necessary

          @all_entries = split(/<HR>/i, $all_entries);
          $entry_count = @all_entries - 1;

          while ($entry_count > $max_entries) {
              pop @all_entries;
              $entry_count = @all_entries - 1;
          }

          $all_entries = join('<HR>', @all_entries);

      }


   

    # now write out to $data_file

    seek(ENTRIES, 0, 0)        or die "can't rewind $data_file: $!";
    truncate(ENTRIES, 0)       or die "can't truncate $data_file: $!";
    print ENTRIES $all_entries or die "can't print to $data_file: $!";
    close(ENTRIES)             or die "can't close $data_file: $!";

}

# display the guestbook a.k.a pastepile.html

open (IN, "$data_file") or die "Can't open $data_file for reading: $!";
flock(IN, 1)            or die "Can't get LOCK_SH on $data_file: $!";
while (<IN>) {
    print;
}
close IN                or die "Can't close $data_file: $!";

# display the form    

print <<"EndOfText";
<A NAME="form"><H2>Add a comment/entry to the paste pile (no HTML):</H2></A>

<FORM METHOD="POST" ACTION="pastepile.cgi">
<TABLE>



<TR>
<TD ALIGN="right"><STRONG>Subject: </STRONG></TD>
<TD><INPUT NAME="subject" SIZE=30></TD>
</TR>

<TR>
<TD ALIGN="right"><STRONG>Entry :</STRONG></TD>
<TD>
<TEXTAREA NAME="comment" ROWS=5 COLS=30 WRAP="virtual"></TEXTAREA>
</TD>
</TR>

<TR><TD COLSPAN=2> </TD></TR>
<TR>
<TD> </TD>
<TD><INPUT TYPE="submit" NAME="action" VALUE="Add entry"></TD>
</TR>
</TABLE>

</FORM>
</BODY>
</HTML>
EndOfText

 

Getting CGI and Perl scripts up and running on Minimal Ubuntu

Linux, Raspberry Pi, Web Hosting, , , , ,

I was trying, again to get this up and running. I have a piece of code notestack-example.cgi that uses Perl and the Perl Module for CGI. I had this working after fiddling with it the first time I flashed the SD card that I set up for a Pine 64 that was set up with minimal Ubuntu Xenial.

The problem is I wrote down only sketchy instructions and had to re-figure it out. After flashing another card, the first one had a slow moving corruption that would cause the machine to halt after a while, I got more clear with the process.

I have posted it here for myself, if I get stuck again and for anyone else who might need to know the process, if they get stuck. It is a rough outline. I copied what commands I issued from the history and added comments and some test code that I had on a RaspberryPi which has been running the notestack-example.cgi among other items for years so that was my baseline.

Outline getting CGI and Perl running for Apache

In this outline it is assumed that Apache2 is installed and configured.

Optional: To make it easier to get to the cgi directory from your home, create a symlink in the user home directory to be able to move into the cgi folder easier.

ln -s /usr/lib/cgi-bin cgi

Enable CGI support for Apache…
sudo a2enmod cgi

modify /etc/apache2/sites-enabled/000-default.conf

Putting in the code that enables cgi in the Apache config file for sites, this was take from the Raspberrypi and added into /etc/apache2/sites-enabled/000-default.conf
it was put right above the </VirtualHost> line.

——————————————————
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory “/usr/lib/cgi-bin”>
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>

——————————————————

Test with a simple BASH CGI script

Use the following code as printenv.cgi to test basic CGI with a bash script.
in the /usr/lib/cgi-bin directory.
——————————————————
#!/bin/bash -f
echo “Content-type: text/html”
echo “”
echo “<PRE>”
env || printenv
echo “</PRE>”
exit 0
——————————————————

test with…
which needs to be 755 (+x) permissions. This file is from the RPI, was used years ago to test it out.

sudo nano printenv.cgi
sudo chmod +x printenv.cgi
  /printenv.cgi
If all is well this will also be accessible from the web…
curl http://localhost/cgi-bin/printenv.cgi

curl is installed in the minimal build.

If you are using this on your own intranet OK, if this machine is accessible from the web get rid of printenv.cgi so the whole world can’t see the output if it gets found.

 

Test Perl

Test Perl scripts, normally Perl is installed even in the minimal Ubuntu build. It’s
presence can be tested for by using ‘which perl‘.
Use the following code as perl-test.cgi
in the /usr/lib/cgi-bin directory.
——————————————————
#!/usr/bin/perl
print “Content-type: text/html\n\n”;
print “Hello CGI\n”;
EOF
——————————————————

sudo nano perl-test.cgi
 sudo chmod 755 perl-test.cgi
 ./perl-test.cgi

Does it work via Apache?
  curl http://localhost/cgi-bin/perl-test.cgi

Perl CGI Module

Get Perl scripts (Any Perl code that uses CGI.pm in it) running and web accessible, ones that require the CGI Perl Module…

Got ERROR #1,missing CGI.pm, followed by #2 complained about a missing Util.pm, see below.

Got this error when the Perl Module was missing got a similar error after installing PM, complaint about missing CGI/Util.pm
ERROR 1 and 2
Can’t locate CGI.pm in @INC (you may need to install the CGI module) (@INC contains: /etc/perl /usr/local/lib/aarch64-linux-gnu/perl/5.22.1 /usr/local/share/$
BEGIN failed–compilation aborted at ./notestack-example.cgi line 36.
Grabbed the CGI.pm and Util.pm from my Raspberry Pi by searching for them….

sudo find / -name CGI.pm

sudo find / -name Util.pm

and copying using rcp to/tmp on the board I was setting up, a Pine 64 in this case.

rcp /usr/share/perl/5.14.2/CGI.pm ubuntu@192.168.1.31:/tmp
rcp /usr/share/perl/5.14.2/CGI/Util.pm ubuntu@192.168.1.31:/tmp

The code I was trying to run, the goal of all this work was a script named notestack-example.cgi.
sudo cp /tmp/CGI.pm /etc/perl
./notestack-example.cgi      <— Got ERROR #1
sudo cp /tmp/Util.pm /etc/perl/
./notestack-example.cgi      <— Got ERROR #2
sudo mkdir /etc/perl/CGI
sudo mv /etc/perl/Util.pm /etc/perl/CGI

Final error ERROR 3

Can’t use ‘defined(@array)’ (Maybe you should just omit the defined()?) at /etc/perl/CGI.pm line 528.
Compilation failed in require at ./notestack-example.cgi line 36.
BEGIN failed–compilation aborted at ./notestack-example.cgi line 36.

This one requires removing defined as this is old and not compatible with the current version of Perl.
Just removed the defined on line 528…

ubuntu@pine64:~$ diff /etc/perl/CGI.pm /tmp/CGI.pm
528c528
< if (@QUERY_PARAM && !$initializer) {
—
> if (defined(@QUERY_PARAM) && !defined($initializer)) {

Learned about this trick about removing the ‘defined‘ from…
https://github.com/shenlab-sinai/diffreps/issues/6
https://rt.cpan.org/Public/Bug/Display.html?id=79917

CPU Temperature Monitoring on the Raspberry Pi

Linux, Raspberry Pi, , , , , , , ,

One of my thoughts for using the Raspberry Pi is to monitor ambient temperature in my house and send me warnings if it is too low. This would be helpful when away from home in the winter and there is an issue with the heat that I might need to know about fairly quickly.

Eventually I might consider building in a weather station capability into the Raspberry Pi by using both an indoor temperature, humidity and barometric sensor and an outdoor temperature and humidity sensor. So with that in mind I also want to make some simple graphs of data every hour or so. Something that can be just saved into a text file and requires no extra graphing code on the client or the server.

CPU Temperature Experiment

I started out by experimenting with taking reading of the CPU temperature of the Raspberry Pi in order to get a little practice with some live data while I was awaiting the arrival of a Bosch BME280 ambient temp., humidity and barometric pressure sensor.

sSMTP

I plan on using sSMTP to send me warnings, such as when my ambient temperature falls below a critical value. I have already worked up some code to monitor the CPU temperature and email me if it goes out of a specific range, to get a bit of practice with coding this into a script. Plus it was a good way to try out sSMTP. sSMTP will be covered in more detail in a separate post.

Snippet of script that sends email if the upper and lower CPU temp limits are exceeded

The temperature read by the cat /sys/class/thermal/thermal_zone0/temp is in milliCelsius, so using cut on it will grab whole degree values. The rest of the code is pretty straightforward. The variables minimum and maximum are setup to be the appropriate values in whole degrees Celsius. mailaddr is exactly what it sounds like. What is nice is that it just gives you a very simple method of sending a basic email warning when the temperatures get out of bounds.

# Read the temp and cut it to grab leftmost 2 characters, integer Temp
temp="`cat /sys/class/thermal/thermal_zone0/temp | cut -c1-2`"

#echo $temp

# Mail if about or below the limits
if (( $temp > $maximum )); then
   #echo "above"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr
elif (( $temp < $minimum )); then
   #echo "below"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr

fi

 

Simple Graphical Temperature Logging

To just monitor the CPU temperature on a hourly basis and have a simple graphical representation of it I had to do some digging online.

I dug around the web and found a bit of Perl code to make a simple ASCII graph of the temperatures that I was measuring for the CPU. I wanted to be able to simply post to the web a running capture of temperatures, grabbed hourly by using CRON. I did not want to have to rely on graphing tools either on the Raspberry Pi or the client computer to interpret data. I spent a while searching for something simple and this was about as easy as it can get. I found a one line, long line! Method for creating a simple graph using Perl. I looked around a lot on line and wanted to avoid anything that would have to be installed on either the Raspberry Pi or the client machine to be able to plot data. So after a long search this piece of Perl code is about as simple as it gets and will plot and ASCII based graph across one line each time it executes.

The key variables in the code are min, max and w. The variable w controls the width of the plot in characters that it will be allowed to reach when the input value is at max. The variables min and max control the minimum and maximum input values expected and scale the graph accordingly. PLOTTABLE_FILE is the path/filename of the file that the output will be sent to.

 PLOTTABLE_FILE=/var/www/tmp/cputemp-milli-celsius.txt

Perl Code Snippet

The code resides in a script in the /etc/cron.hourly directory and produces a file that is readable via the web, with a new entry every hour. The filename is cputemp, not the lack of a .sh extension. This is important when you create scripts that will reside in the “CRON” folders. It also has to be made executable by using sudo chmod +x filename.

(cat /sys/class/thermal/thermal_zone0/temp ) | perl -ne '$min=20000; $max=65000; $w=79; use POSIX; $d=ceil(log($max)/log(10)); $w-=$d; $v=$_<$min?0:$_>$max?$max:$_; $s=$w*$v/($max-$min); $bar=join("", ("*")x$s); $bar=~s/.$/|/ if $v==$max; print sprintf("%${d}d ",$_)."$bar\n";' >> $PLOTTABLE_FILE

Sample Output

Below of a capture of a few lines of output from the text file that is generated by the Perl code. It does not look as clean as it would by looking at the text file. So I have a capture of that here…. cputemp-milli-celsius

42236 *********************************************************************
41160 *******************************************************************
41160 *******************************************************************
41160 *******************************************************************
40084 *****************************************************************
41160 *******************************************************************
42774 **********************************************************************
44388 ************************************************************************
45464 **************************************************************************
45464 **************************************************************************
46002 ***************************************************************************

Alternate Method to Read CPU Temperture for the Raspberry Pi

There is another way to read the CPU in a more human readable form. Executing the line below…

/opt/vc/bin/vcgencmd measure_temp

…will show a result like this…

temp=45.5'C

Resources

Source of Perl Script