Category Archives: Linux

Posts that cover information that I have learned and have posted, all related to Linux.

Hacker Cat

Using AWK to pull data from a field within a text file

Linux,

Intelligence is the power which gives us the ability to distinguish when we are not conscious.

# awk -F” ” ‘{print $11}’ pressure-delta.txt
tail -1 $PLOTTABLE_DIR/$PRESSURE_DELTA_TAIL_FILE > $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt

# Use awk with -F” “, which uses space as field seperator.
# pressure delta in inHg is in the 11th field.
# Multiply by 1000 because bash does not do floating point math natively and we want to compare so use milli-inHg to do the bash math.
# Operate on the file $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt

pressure_delta=`awk -F” ” ‘{print $11*1000}’ $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt`

Penguin

guestbook.cgi

Linux, Server, Unpolished Gems, , ,

The best way to understand a thing is to try to comprehend it.

I ran John Callendar’s guestbook.cgi on one of my servers. It wasn’t a very popular location on the web and the guestbook did not get abused fortunately! It would be very easy for comment spammers, trollers and the like to have a field day with it.
One thing I did to keep an eye on it and see if it got posted to was a script in /etc/cron.weekly to check for updates of the guestbook. It was located on a Raspberry Pi running Raspberian.
The script requires that ssmtp or some mail program is installed.

guestbook-check.sh

#!/bin/bash

# Checks to see if the Guestbook has been written to in the past week.
 # Sends out notification if it has been written to.
 find /usr/lib/cgi-bin/guestbookrev.txt -mtime -7 -exec mail -s "Guestbook Updated" myemail@myisp.net \;

The trick is in the find command where the option -mtime -7 means check to see if the file have been has a modification time of less than 7 days, if so, then execute via the -exec option, whatever comes next on the command line until the \;

Subject line

The mail command mails only a subject line of “Guestbook Updated”. It would be possible to have something in the body and even “cat” the guestbook in to send it in the email.

Auto de Spam

If the guestbook got abused by a spammer or something nasty it might be possible to run a script that would periodically do a cleanup on the file via a search and replace. Using a list of blacklisted words to search on and then replace them with a null character or space.

Reverse Order in Guestbook

It is possible to flip the posts on the guestbook as well if you want them ordered in the opposite order, Last In On Top versus the default of First In On Top. This is done by using…

 pop @all_entries instead of shift @all_entries

…in the code.

More on CGI and Perl

If you are new to CGI and/or Perl scripts be sure to check out John Callender’s tutorial that covers the workings of guestbook.cgi.
guestbook.cgi

Apple Notepad

Termbin

Info Links, Linux, Unpolished Gems, , , ,

I’d like to make sure everybody is at home when they are on vacation.

http://termbin.com/
An easy to use Pastebin like tool that allows “pasting” from the command line.

Requires that netcat is installed on the PC, which is by default on Linux and can be installed on a Windows computer as well.

.bashrc

I recommended adding an alias to your .bashrc in Linux to make a shorthand to post to termbin…

alias tb='nc termbin.com 9999'

 

Posts stay active for a month as an example I posted the link to this post online via termbin.

Privatebin

Other Bins that are install-able
https://privatebin.net/
https://github.com/PrivateBin/PrivateBin/wiki/Installation

How to Setup A Hastebin Server

http://sergiogervacio.com/host-hastebin-server/

 

termbin.com is powered by Fiche – open source command line pastebin server. There is a link to github repository: https://github.com/solusipse/fiche.

Penguin

Pastepile

Linux, Unpolished Gems, Web Hosting, , , , , , , , ,

Future development depends on the determination of future generations to contribute towards building their own society.

Pastepile is a fork of guestbook.cgi – a simple guestbook script, written by John Callender in 1999. It is the last part of his Beginner’s Guide to CGI Scripting with Perl, Running a Guestbook.
This new version, a.k.a Pastepile, is a dirt simple Perl CGI program that creates a blog like pasting tool to be able to paste notes online or on a local server. Allows grabbing snippets of text or code for later use on whether on the local machine or another on the network. I made it sometime after studying John Callender’s tutorial when I was digging into learning about CGI and Perl several years ago.

History

Pastepile originated as a way to paste snippets of code and write short notes on configuration changes that I make on servers, including a Raspberry Pi that I run. The only requirement is the installation of a web server such as Apache. This little Pastepile tool made it easy to keep track of information related to the servers in one place  and search-able via a browser. After using it in the is mode for a while, I cloned it and started to use it as a general place to paste info and write short notes. This makes it easy to save something while on one machine on the network and be able to retrieve it later on that machine or another.

Example of What it Looks Like

Subject: Running a form-to-email gateway at 19:04:44, Sat Feb 3, 2018 from 192.168.1.7

http://www.resoo.org/docs/perl/begperl/form_to_email.html

Subject: Slackware Linux Essentials at 12:10:03, Thu Feb 1, 2018 from 192.168.1.179

www.slackbook.org/html/index.html

Additions:Remote Address,Time,Reversed Order

Besides stripping down guestbook.cgi to remove code specifc to it’s guestbook function and cleaning up the code to reflect it’s new use, three functional code changes were made. One is to add a timestamp to keep track of when the entry was created. Another change made was showing the IP of the machine the post originates from by reading the environment variable $REMOTE_ADDR to keep track of where the post originates from. Many machines on my network have static IP’s so this is handy for me at least. The final change was to reverse the order of the listing. In guestbook.cgi a First In On Top ordering is the layout of the guestbook posts. For Pastepile I reversed it so the most recent and not the oldest post is on top of the list, Last In On Top ordering. This made sense as the most recently written notes are more likely to be important and I want those to be close to the top of the pile and let the file get long with the aging information at the bottom.

/p/

For ease of access via a shorter link a simple index.html redirector page was place in /var/www/p/, following a 4chan-esqe style of folder naming and a nice short link to get to the pastepile.cgi script, which lives in the /usr/lib/cgi-bin/ directory. The /p/ directory also holds the pastepile.html file that is created by pastepile.cgi as it’s data page.

 Redirector Example for /var/www/p/index.html

<HTML>
<HEAD>
<meta http-equiv="refresh" content="0; url='http://192.168.1.17/cgi-bin/pastepile.cgi'"/>
</HEAD>
<BODY>
<p><a href="http://192.168.1.17/cgi-bin/pastepile.cgi">Redirect</a></p>

</BODY>
</HTML>

paste-pile-mover.sh

A helper script called pastepile-file-mover.sh, moves the pastepile.html, the data file created by pastepile.cgi from it’s default location to a date stamped file in a location set in the script BASEDIR/dir/filename. Where BASEDIR is your choice ( I use /var/www/p/), dir is the current year and filename is YYYYMMDD.html, so that that there is a  year and date hierarchy. I allow this script to run at the start of the month via root CRON to “clean” out the Pastepile and archive the old one, much the same way that log are rotated.

0 0 1 * * /home/erick/bin/pastepile-file-mover.sh

pastepile-file-mover.sh

#!/bin/bash

# Move the pastepile.html file from it's default location to a date stamped
# file in a location BASEDIR/dir/filename
# So that it has year and date heirarchy

# For now, Monthly, Move pastepile over to year dir and datestamped HTML file
#0 0 1 * * /home/erick/bin/pastepile-file-mover.sh



BASEDIR=/var/www/p
# Testing
#BASEDIR=/tmp

dir=$(date +"%Y")
#echo $dir

#File name timestamped
filename=$(date +"%Y%m%d").html
#echo $filename

# Make the YEAR dir if it dow not exist.
if [ ! -d "$BASEDIR/$dir" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
  mkdir $BASEDIR/$dir
fi

# Do the move to the YEAR directory with the YYYYMMDD.html filename.
mv $BASEDIR/pastepile.html $BASEDIR/$dir/$filename

# This is needed make a new pastepile.html and chmod 666
# else pastepile.cgi does not work, it can't make it's own output file.
touch $BASEDIR/pastepile.html
chmod 666 $BASEDIR/pastepile.html

Last but not least pastepile.cgi

#!/usr/bin/perl -Tw

# pastepile.cgi a fork of...
# guestbook.cgi - a simple guestbook script

# This program is copyright 1999 by John Callender.

# This program is free and open software. You may use, copy, modify,
# distribute and sell this program (and any modified variants) in any way
# you wish, provided you do not restrict others from doing the same.

# pastepile.cgi - guestbook.cgi, modded to become # a pastepile program. Erick Clasen Jan 25,2018
# This new version is a dirt simple CGI program that creates a blog like
# pasting tool to be able to paste notes online or on a local server.
# allows grabbing snippets of text or code for later use on whether
# on the local machine or another on the network.

$data_file = '/var/www/p/pastepile.html';

$max_entries = 0; # how many guestbook entries to save?
                   # set to '0' (zero) for infinite entries...

use CGI;
use Fcntl;
$query = new CGI;

unless ($action = $query->param('action')) {
    $action = 'none';
}

print <<"EndOfText";
Content-type: text/html

<HTML>
<HEAD>
<TITLE>Raspberry Pi Server Paste Pile</TITLE>
</HEAD>
<BODY>
<H1>Raspberry Pi Server Paste Pile</H1> 
<P><EM>$ENV{DOCUMENT_ROOT}/p/</EM></P>



<A HREF="../status/index.html">Back to Status Index</A>&nbsp;
<A HREF="../p/2018">2018 Paste Archive</A>

<P>You can <A HREF="#form">add your own subject and entry</A> using the form at the bottom of the page. Here we is has your pastes...</P>

<HR>
EndOfText

# Input action to add a new entry. ----------------------
if ($action eq 'Add entry') {

    # process the form submission
    # and assemble the guestbook entry


    # Input the subect and the paste which is called a comment here in this code.
    $subject = $query->param('subject');

    $comment = $query->param('comment');

    # clean up and fiddle with $subject
 unless ($subject) {
        $subject = 'No Subject';
   if (length($subject) > 50) {
        $subject = 'Subject line too long >50 chars';
    }
# End Input action to add a new entry. ----------------------



    }

    # Add a time stamp, put in variable theTime. This allows the paste to be timestamped.
    
    @months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
    @weekDays = qw(Sun Mon Tue Wed Thu Fri Sat Sun);
    @digits = qw(00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 59 59);
    ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek) = localtime();
    $year = 1900 + $yearOffset;
    $theTime = "$digits[$hour]:$digits[$minute]:$digits[$second], $weekDays[$dayOfWeek] $months[$month] $dayOfMonth, $year";

    # untaint variable
    unless ($theTime =~ /^([^<]*)$/) {
        die "couldn't untaint name: $theTime\n";
    }
    $theTime = $1;

    

    # clean up and fiddle with $subject--------------------------------

    $subject_clean = "$subject";
    $subject_clean =~ s/, , /, /;        # remove duplicate ', '
    $subject_clean =~ s/^, //;           # remove initial ', '
    $subject_clean =~ s/, $//;           # remove final ', '
    if ($subject_clean =~ /^[,\s]+$/) {
        # nothing but commas and whitespace
        $subject_clean = 'Subject format wrong!!';
    }
    
    if (length($subject_clean) > 75) {
        $subject_clean = 'Subject too long.';
    }

    # disable HTML tags
    $subject_clean =~ s/</&lt;/g;

    # untaint variable
    unless ($subject_clean =~ /^([^<]*)$/) {
        die "couldn't untaint subject_clean: $subject_clean\n";
    }
    $subject_clean = $1;

    

    # clean up and fiddle with $comment----------------------------

    if (length($comment) > 32768) {
        $comment = '...overloaded blog buffer chars > 32768.';
    }
    unless ($comment) {
        $comment = '...nothing to speak of.';
    }

    # fix line-endings
    $comment =~ s/\r\n?/\n/g;

    # lose HTML tags
    $comment =~ s/</&lt;/g;

    # untaint variable
    unless ($comment =~ /^([^<]*)$/) {
        die "couldn't untaint comment: $comment\n";
    }
    $comment = $1;
    # end cleanup #comment-----------------------------------------

    

    # assemble finished guestbook entry -- anything after this line until EndofText will show in the post!!!!!

    # Enviroment variable for REMOTE_ADDR is grabbed and printed directly. No untainting, but probably safe to do.

    $entry = <<"EndOfText";

<P><STRONG> Subject: $subject_clean </STRONG> at <EM>$theTime </EM> from <EM>$ENV{REMOTE_ADDR} </EM> <BR>
<BLOCKQUOTE>$comment</BLOCKQUOTE></P>
<HR>
EndOfText

    # open non-destructively, read old entries, write out new
   $all_entries .= $entry;
    sysopen(ENTRIES, "$data_file", O_RDWR)
                             or die "can't open $data_file: $!";
    flock(ENTRIES, 2)        or die "can't LOCK_EX $data_file: $!";
    while(<ENTRIES>) {
        $all_entries .= $_;
    }

 if ($max_entries) {

          # lop the tail off the guestbook, if necessary

          @all_entries = split(/<HR>/i, $all_entries);
          $entry_count = @all_entries - 1;

          while ($entry_count > $max_entries) {
              pop @all_entries;
              $entry_count = @all_entries - 1;
          }

          $all_entries = join('<HR>', @all_entries);

      }


   

    # now write out to $data_file

    seek(ENTRIES, 0, 0)        or die "can't rewind $data_file: $!";
    truncate(ENTRIES, 0)       or die "can't truncate $data_file: $!";
    print ENTRIES $all_entries or die "can't print to $data_file: $!";
    close(ENTRIES)             or die "can't close $data_file: $!";

}

# display the guestbook a.k.a pastepile.html

open (IN, "$data_file") or die "Can't open $data_file for reading: $!";
flock(IN, 1)            or die "Can't get LOCK_SH on $data_file: $!";
while (<IN>) {
    print;
}
close IN                or die "Can't close $data_file: $!";

# display the form    

print <<"EndOfText";
<A NAME="form"><H2>Add a comment/entry to the paste pile (no HTML):</H2></A>

<FORM METHOD="POST" ACTION="pastepile.cgi">
<TABLE>



<TR>
<TD ALIGN="right"><STRONG>Subject: </STRONG></TD>
<TD><INPUT NAME="subject" SIZE=30></TD>
</TR>

<TR>
<TD ALIGN="right"><STRONG>Entry :</STRONG></TD>
<TD>
<TEXTAREA NAME="comment" ROWS=5 COLS=30 WRAP="virtual"></TEXTAREA>
</TD>
</TR>

<TR><TD COLSPAN=2> </TD></TR>
<TR>
<TD> </TD>
<TD><INPUT TYPE="submit" NAME="action" VALUE="Add entry"></TD>
</TR>
</TABLE>

</FORM>
</BODY>
</HTML>
EndOfText

 

laptop

Scraping a WordPress site for text to use with char-rnn

Linux, Scripts, WordPress, , ,

As with anything else in life, it is possible to change nothing but yourself.
The first step toward making change is simply to change yourself.

In this quick example the gist of scraping a WordPress website using Linux and Lynx will be shown. Wget is great at scraping from the web but, I have found out that it does not always work well with WordPress sites.

Grab some text

Step one of the process is to grab some text to work with. In the example I tried I grabbed all of the text of the posts of this blog by scraping it with wget. I also used the text of the US Constitution to see what the tools would do with it as well. Generally the more text, the better the machine learning code will be at generating something interesting.

Scraping the posts

Using the command line web browser lynx in a script I was able to download the text of the posts on this site. Initially I thought to use wget. But, I remembered that wget will do a good job downloading static sites and sometimes will not do so well with ones like this one that is created in WordPress.

There is probably a way to loop this code in bash, and increment a counter for the pages. But, being that this is a one time thing, I opted for a quick approach instead of thinking too hard on making a loop.

#!/bin/bash
lynx -dump -nolist http://erick.heart-centered-living.org/ > my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/2/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/3/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/4/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/5/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/6/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/7/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/8/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/9/ >> my-posts.txt

This code will output a file that contains all of the text from the posts on this site. Up to Fall of 2018 when I ran it.

iPhone Mounting

Linux, Unpolished Gems,

The beauty of love is not in your beauty but in your ability to make it manifest.

Mostly because I have had to look up how to mount an iPhone on Linux Mint 18.0 a few times and had to look it up,I made a helper script. – iPhone-mount.sh

#!/bin/bash
 # Allow non-root users to mount with ifuse!
 #sudo nano /etc/fuse.conf
 # Allow non-root users to specify the allow_other or allow_root mount op$
 # UNCOMMENT user_allow_other

# Pair the iPhone
 idevicepair pair
 sleep 5
 # Mount it
 ifuse /home/erick/iPhone

Allow users to mount via ifuse

In /etc/fuse.conf uncomment user_allow_other to let users mount devices. This makes it easy to mount the device in your home directory for instance with full access to it.

Mint and Mounting

The iPhone mounts fine on Mint 18.0. My desktop has a newer version and I had to do work to it before mounting the iPhone worked. Apparently something got lost between versions.

https://www.dedoimedo.com/computers/linux-iphone-6.html

Penguin

Raspberry Pi Bridge from Wireless to Ethernet

Linux, Raspberry Pi, Raspberry Pi Server, Unpolished Gems, , , , , , ,

The sky, when the stars rise, the stars fly away, the stars weep.

Setting up a bridge wlan0 to eth0 on a Raspberrypi

This is for an ipv4 Bridge connection between WAN and LAN using a Raspberrypi. Another Linux computer would also work instead of a Raspberrypi. Some of this is basically a dump of note that I took while trying to get this up and running in 2017. This post may be of use to someone who is struggling to get this sort of thing going as some of the sources on the net were not always clear on how to do it and validate it’s operation.

The idea with this plan is to have the R-Pi act as a router and hand out addresses to devices on the network and do a DNS masquerade on the eth0 connection. It will also forward ipV4 packets in BOTH directions, thereby bridging a wired and wireless networks. Note: Devices running on the LAN will require setting up a default route to the gateway on the WLAN to see devices on the WLAN.

Why Setup a wlan0 to eth0 Bridge

I had to do this to allow a connection between a ZTE WiFi hotspot that did not have any Ethernet connection port and a need to connect a set of desktop computers that only have Ethernet ports to the Internet. Essentially I have two networks, one is WiFi, one hardwired and they machines have to be able to reach the Internet and each other from both sides. The Ethernet machines are connected to a switch with a router connected to it acting as an access point, DHCP set to off. This network is on an “island” that needed to be bridge via WLAN to get out to the Internet via the ZTE hotspot.

Install dnsmasq

dnsmasq is a lightweight program that will run as a service that will take care of the DNS and the DHCP functions that are required to make the R-Pi act as a router and bridge.

sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get install rpi-update dnsmasq -y
 sudo rpi-update

Static IP on Ethernet Connection

Next, setup a static ip for the dhcp server. Edit /etc/network/interfaces to make the R-Pi reside at a static address on the wired network. In this example it is setup for 192.168.1.17, the typical gateway address for a router on a 192.168.1.0 network, would be 192.168.1.1 which was the gateway at one time on this network. There is nothing magical about the 192.168.1.1 address, a default gateway can exist on any valid address, excluding 192.168.1.0 and 255.

NOTICE THAT THE DEFAULT GATEWAY FOR THE eth0 IS NOT SET! This is important because the default gateway should be grabbed from the WiFi network and not the wired, which in my case is not connected to the Internet. It will go to the Ethernet first by default if there are two default gateways, WLAN and LAN.

Router Settings

If there is a router on the network, it is important to turn off DHCP on it as it does not have to hand out addresses anymore. It should just behave as a switch instead, just forwarding packets in/out of all ports including WiFi if it has it and this option is wanted, then it will function as an AP ( Access Point) as well on it’s own network (192.168.1.0/24 in my case).

 

erick@raspberrypi ~ $ cat /etc/network/interfaces
 auto lo

iface lo inet loopback
 #iface eth0 inet dhcp

iface eth0 inet static
 address 192.168.1.17
 netmask 255.255.255.0
 network 192.168.1.0
 broadcast 192.168.1.255
 #gateway 192.168.1.1
 # nameservers
 dns-nameservers 8.8.8.8 8.8.4.4

allow-hotplug wlan0
 iface wlan0 inet manual
 wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
 iface default inet dhcp

Sanity Check using mtr, a.k.a. My Trace Route

Running mtr shows a direct route to Optiplex-790, running mtr from Optiplex-790 to 192.168.128.1 shows a bounce via the R-pi.

See Appendix 1 for more info.

Configure dnsmasq via /etc/dnsmasq.conf

Copy the rather wordy original to a backup copy and use sudo nano to edit in new details. The listen address will be the same as entered into the /etc/network/interfaces file for the R-Pi now that it is at a static address, mine is at 192.168.1.17 for example. Server is the dns server that dnsmasq will be using to do it’s masquerade magic. This can be a comma separated list. I have Google in there for DNS at 8.8.8.8, but an ISP would do as well. Sometimes the router upstream includes a caching DNS and it can be included as well. If the upstream router does have a caching DNS this helps a bit with lookups as the lookup table will be maintained locally as a cache of frequently visited web addresses. Having a local lookup for DNS has less delays than reaching out on the web for every word address to IP numeric address translation.

Address Reservation

For dncp-range I am choosing from 192.168.1.20-192.168.1.255 as the ones below 20 on my network are kept in reserve for static addresses.

Backup and then edit…

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig
 sudo nano /etc/dnsmasq.conf

/etc/dnsmasq.conf example

This is the one that I am using for the R-pi bridge. R-Pi is static on 192.168.1.17. DNS server is Googles 8.8.8.8 and I am reserving addresses from 192.168.1.0-20.

interface=eth0      # Use interface eth0
 listen-address=192.168.1.17 # Explicitly specify the address to listen on
 bind-interfaces      # Bind to the interface to make sure we aren't sending things elsewhere
 server=8.8.8.8       # Forward DNS requests to Google DNS
 domain-needed        # Don't forward short names
 bogus-priv           # Never forward addresses in the non-routed address spaces.
 dhcp-range=192.168.1.20,192.168.1.255,12h # Assign IP addresses between 192.168.1.20,192.168.1.255  with a 12 hour lease time

 

Enable IPv4 forwarding

The R-pi kernel has to be told explicitly to forward IPv4 packets between wlan0 and eth0.

sudo nano /etc/sysctl.conf

Fin and UNCOMMENT the following line

 net.ipv4.ip_forward=1

TO APPLY CHANGE WITHOUT A REBOOT

sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

IPTABLES rules update

iptables needs a few rules added to it to cover the DNS masquerading and accepting packets forwarded from wlan0 to eth0 and in the other direction. Execute the following commands to add the rules to iptables.

sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
 sudo iptables -A FORWARD -i wlan0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
 sudo iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPT

iptables will keep it’s info as long as the machine is booted up. Needs a reload when rebooted.

THE STUFF BELOW IS NOT NEEDED AS THE PI HAS A METHOD TO RESTORE THE IPTABLES VIA iptable-save.sh
HAVE NOT DONE THIS YET as of 12/12/2017

sudo sh -c “iptables-save > /etc/iptables.ipv4.nat”

HOOK FILE

sudo nano /lib/dhcpcd/dhcpcd-hooks/70-ipv4-nat

ADD…

iptables-restore < /etc/iptables.ipv4.nat

—————————————————-

 

—————————————————–

On the PC – Optiplex-790

I  set up the main desktop PC as static IP via Edit Connections on the GUI.

Configure Wired Connection 1 as IPv4 Settings, Manual with address as 192.168.1.18 netmask 255.255.255.0 and gateway as the R-pi at 192.168.1.17

DNS servers, the R-pi itself 192.168.1.17 and Google at 8.8.8.8. It might be possible to have 192.168.128.1 as well as that might be a caching DNS on the upstream ZTE WiFi hotspot. It might have a caching DNS server inside of the Sprint Box itself, not sure and requires looking at the specs for it.

Feasibility Test Using a PC as a Bridge

Before I went through the trouble of setting up the R-Pi as a bridge I did a sanity check/prototype by using a PC as a bridge. By taking the WiFi USB dongle and plugging it into the PC, Optiplex-790 running Linux Mint. I was able to test the feasibility out beforehand.

Create shared connection

On an initial test of setting up the network, before digging into the R-pi to do this, I shared the wifi to the eth via editing the Wired Connection 1 and sharing under IPv4 Settings. But it puts it on a 10.42.0.x network. Use the following to change this…

In versions before 1.4.2, 10.42.0.x is hard-coded into NetworkManager. The choice is either upgrade to Ubuntu 17.04, with version 1.4.4, or go the easy way and use the following command from Thomas Haller to set the host IP and class. For my setup it was 192.168.1.18…

nmcli connection modify $CONNECTION_ID +ipv4.addresses 192.168.1.18/24

where $CONNECTION_ID if found via…

 nmcli connection show

… Afterwards, verify with…

nmcli connection show $CONNECTION_ID.

from …
https://askubuntu.com/questions/609645/configure-connection-sharing-with-specific-ip-address

ANY CHANGES MADE TO THE CONNECTION EDITOR REQUIRES A DISCONNECT AND RECONNECT TO APPLY THE CHANGES!!

Uncheck and Recheck Enable Networking

erick@OptiPlex-790 ~ $ nmcli connection show
 NAME                   UUID                                  TYPE             DEVICE
 Wired connection 1     1a2d9768-104d-3714-814c-57ea2faff63b  802-3-ethernet   eno1
 NETGEAR63              13f543ac-ea4b-455e-9ff7-9e0ecaddb139  802-11-wireless  --
 SprintHotspot2.4-BA3A  b85d60f9-5875-4eaa-a0c0-df43d174c869  802-11-wireless  --
...Verification that the change took hold after setting via nmcli connection modify command...
 erick@OptiPlex-790 ~ $ nmcli connection show 1a2d9768-104d-3714-814c-57ea2faff63b
 connection.id:                          Wired connection 1
 connection.uuid:                        1a2d9768-104d-3714-814c-57ea2faff63b
 connection.interface-name:              --
 connection.type:                        802-3-ethernet
 connection.autoconnect:                 yes
 connection.autoconnect-priority:        -999
 connection.timestamp:                   1513298233
 connection.read-only:                   no
 connection.permissions:
 connection.zone:                        --
 connection.master:                      --
 connection.slave-type:                  --
 connection.autoconnect-slaves:          -1 (default)
 connection.secondaries:
 connection.gateway-ping-timeout:        0
 connection.metered:                     unknown
 connection.lldp:                        -1 (default)
 802-3-ethernet.port:                    --
 802-3-ethernet.speed:                   0
 802-3-ethernet.duplex:                  full
 802-3-ethernet.auto-negotiate:          yes
 802-3-ethernet.mac-address:             18:03:73:D1:52:FC
 802-3-ethernet.cloned-mac-address:      --
 802-3-ethernet.mac-address-blacklist:
 802-3-ethernet.mtu:                     auto
 802-3-ethernet.s390-subchannels:
 802-3-ethernet.s390-nettype:            --
 802-3-ethernet.s390-options:
 802-3-ethernet.wake-on-lan:             1 (default)
 802-3-ethernet.wake-on-lan-password:    --
 ipv4.method:                            manual
 ipv4.dns:                               192.168.1.17,8.8.8.8
 ipv4.dns-search:
 ipv4.dns-options:                       (default)
 ipv4.dns-priority:                      0
  ipv4.addresses:                         192.168.1.18/24
 ipv4.gateway:                           192.168.1.17
 ipv4.routes:
 ipv4.route-metric:                      -1
 ipv4.ignore-auto-routes:                no
 ipv4.ignore-auto-dns:                   no
 ipv4.dhcp-client-id:                    --
 ipv4.dhcp-timeout:                      0
 ipv4.dhcp-send-hostname:                yes
 ipv4.dhcp-hostname:                     --
 ipv4.dhcp-fqdn:                         --
 ipv4.never-default:                     no
 ipv4.may-fail:                          yes
 ipv4.dad-timeout:                       -1 (default)
 ipv6.method:                            auto
 ipv6.dns:
 ipv6.dns-search:
 ipv6.dns-options:                       (default)
 ipv6.dns-priority:                      0
 ipv6.addresses:
 ipv6.gateway:                           --
 ipv6.routes:
 ipv6.route-metric:                      -1
 ipv6.ignore-auto-routes:                no
 ipv6.ignore-auto-dns:                   no
 ipv6.never-default:                     no
 ipv6.may-fail:                          yes
 ipv6.ip6-privacy:                       0 (disabled)
 ipv6.addr-gen-mode:                     stable-privacy
 ipv6.dhcp-send-hostname:                yes
 ipv6.dhcp-hostname:                     --
 GENERAL.NAME:                           Wired connection 1
 GENERAL.UUID:                           1a2d9768-104d-3714-814c-57ea2faff63b
 GENERAL.DEVICES:                        eno1
 GENERAL.STATE:                          activated
 GENERAL.DEFAULT:                        yes
 GENERAL.DEFAULT6:                       no
 GENERAL.VPN:                            no
 GENERAL.ZONE:                           --
 GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/14
 GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/1
 GENERAL.SPEC-OBJECT:                    /
 GENERAL.MASTER-PATH:                    --
  IP4.ADDRESS[1]:                         192.168.1.18/24
 IP4.GATEWAY:                            192.168.1.17
 IP4.ROUTE[1]:                           dst = 169.254.0.0/16, nh = 0.0.0.0, mt = 1000
 IP4.DNS[1]:                             192.168.1.17
 IP4.DNS[2]:                             8.8.8.8
 IP6.ADDRESS[1]:                         fe80::7abb:ec07:22dc:c7bd/64
 IP6.GATEWAY:

Looking at the ZTE WiFi Hotspot as seen from the Optiplex-790

erick@OptiPlex-790 ~ $ nmcli connection show b85d60f9-5875-4eaa-a0c0-df43d174c869
 connection.id:                          SprintHotspot2.4-BA3A
 connection.uuid:                        b85d60f9-5875-4eaa-a0c0-df43d174c869
 connection.interface-name:              --
 connection.type:                        802-11-wireless
 connection.autoconnect:                 yes
 connection.autoconnect-priority:        0
 connection.timestamp:                   1513128371
 connection.read-only:                   no
 connection.permissions:
 connection.zone:                        --
 connection.master:                      --
 connection.slave-type:                  --
 connection.autoconnect-slaves:          -1 (default)
 connection.secondaries:
 connection.gateway-ping-timeout:        0
 connection.metered:                     unknown
 connection.lldp:                        -1 (default)
 802-11-wireless.ssid:                   SprintHotspot2.4-B838

802-11-wireless.mode:                   infrastructure
 802-11-wireless.band:                   --
 802-11-wireless.channel:                0
 802-11-wireless.bssid:                  --
 802-11-wireless.rate:                   0
 802-11-wireless.tx-power:               0
 802-11-wireless.mac-address:            08:86:3B:04:85:88
 802-11-wireless.cloned-mac-address:     --
 802-11-wireless.mac-address-blacklist:
 802-11-wireless.mac-address-randomization:default
 802-11-wireless.mtu:                    auto
 802-11-wireless.seen-bssids:            34:69:87:BB:B8:38
 802-11-wireless.hidden:                 no
 802-11-wireless.powersave:              default (0)
 802-11-wireless-security.key-mgmt:      wpa-psk
 802-11-wireless-security.wep-tx-keyidx: 0
 802-11-wireless-security.auth-alg:      --
 802-11-wireless-security.proto:
 802-11-wireless-security.pairwise:
 802-11-wireless-security.group:
 802-11-wireless-security.leap-username: --
 802-11-wireless-security.wep-key0:      <hidden>
 802-11-wireless-security.wep-key1:      <hidden>
 802-11-wireless-security.wep-key2:      <hidden>
 802-11-wireless-security.wep-key3:      <hidden>
 802-11-wireless-security.wep-key-flags: 0 (none)
 802-11-wireless-security.wep-key-type:  0 (unknown)
 802-11-wireless-security.psk:           <hidden>
 802-11-wireless-security.psk-flags:     0 (none)
 802-11-wireless-security.leap-password: <hidden>
 802-11-wireless-security.leap-password-flags:0 (none)
 ipv4.method:                            auto
 ipv4.dns:                               8.8.8.8,8.8.4.4
 ipv4.dns-search:
 ipv4.dns-options:                       (default)
 ipv4.dns-priority:                      0
 ipv4.addresses:
 ipv4.gateway:                           --
 ipv4.routes:
 ipv4.route-metric:                      -1
 ipv4.ignore-auto-routes:                no
 ipv4.ignore-auto-dns:                   no
 ipv4.dhcp-client-id:                    --
 ipv4.dhcp-timeout:                      0
 ipv4.dhcp-send-hostname:                yes
 ipv4.dhcp-hostname:                     --
 ipv4.dhcp-fqdn:                         --
 ipv4.never-default:                     no
 ipv4.may-fail:                          yes
 ipv4.dad-timeout:                       -1 (default)
 ipv6.method:                            auto
 ipv6.dns:
 ipv6.dns-search:
 ipv6.dns-options:                       (default)
 ipv6.dns-priority:                      0
 ipv6.addresses:
 ipv6.gateway:                           --
 ipv6.routes:
 ipv6.route-metric:                      -1
 ipv6.ignore-auto-routes:                no
 ipv6.ignore-auto-dns:                   no
 ipv6.never-default:                     no
 ipv6.may-fail:                          yes
 ipv6.ip6-privacy:                       0 (disabled)
 ipv6.addr-gen-mode:                     stable-privacy
 ipv6.dhcp-send-hostname:                yes
 ipv6.dhcp-hostname:                     --

Note: Needed to Add a Route on a Machine Connected to Raspberry Pi via Ethernet

I needed to add a route to the rpi to get to the 192.168.1.0/24 network.

I THOUGHT that this had worked automatically initially. It seemed that I could at least get to the pi at http://raspberrypi and 192.168.1.17.

But really a route is needed to the 192.168.1.0/24 network via the raspberrypi.rputer on 192.168.128.X

sudo route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.128.46

there is a helper file for this in ~/bin add-route-to-192.168.1.0.sh

erick@media-pc ~/Music $ route
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 default         192.168.128.1   0.0.0.0         UG    600    0        0 wlx08863b02838f
 link-local      *               255.255.0.0     U     1000   0        0 wlx08863b02838f
 192.168.1.0     raspberrypi.rou 255.255.255.0   UG    0      0        0 wlx08863b02838f
 192.168.128.0   *               255.255.255.0   U     600    0        0 wlx08863b02838f

using mtr to a device on the 192.168.1.0 network confirms hw packets are routed through!

————————————————————————————–

NETSTAT VERY HELPFUL

netstat –help
usage: netstat [-vWeenNcCF] [<Af>] -r         netstat {-V|–version|-h|–help}
netstat [-vWnNcaeol] [<Socket> …]
netstat { [-vWeenNac] -i | [-cWnNe] -M | -s }

-r, –route              display routing table
-i, –interfaces         display interface table
-g, –groups             display multicast group memberships
-s, –statistics         display networking statistics (like SNMP)
-M, –masquerade         display masqueraded connections

-v, –verbose            be verbose
-W, –wide               don’t truncate IP addresses
-n, –numeric            don’t resolve names
–numeric-hosts          don’t resolve host names
–numeric-ports          don’t resolve port names
–numeric-users          don’t resolve user names
-N, –symbolic           resolve hardware names
-e, –extend             display other/more information
-p, –programs           display PID/Program name for sockets
-c, –continuous         continuous listing

-l, –listening          display listening server sockets
-a, –all, –listening   display all sockets (default: connected)
-o, –timers             display timers
-F, –fib                display Forwarding Information Base (default)
-C, –cache              display routing cache instead of FIB

<Socket>={-t|–tcp} {-u|–udp} {-w|–raw} {-x|–unix} –ax25 –ipx –netrom
<AF>=Use ‘-6|-4’ or ‘-A <af>’ or ‘–<af>’; default: inet
List of possible address families (which support routing):
inet (DARPA Internet) inet6 (IPv6) ax25 (AMPR AX.25)
netrom (AMPR NET/ROM) ipx (Novell IPX) ddp (Appletalk DDP)
x25 (CCITT X.25)

—————————————————————————————

APPENDIX 1:

https://raspberrypi.stackexchange.com/questions/8010/internet-access-via-1-of-2-network-interfaces

You need to verify that you DO NOT have a default gateway set on your ETH0 interface. It has probably been assigned by DHCP, so you may have to address it statically, edit your router configuration. You will also need to verify that you have a default gateway on your WLAN interface.

Post the output of ip route show when both interfaces are connected for more detailed response.

You also need to ensure that your two routers are on different subnets. For example, the network connected to ETH0 could be 192.168.1.0 255.255.255.0, and WLAN0 could be 192.168.0.0 255.255.255.0, but they have to be on different networks. More on subnet mask

Finally you may want to read up on local routing for Debian systems.

Sorry I can’t be more specific, but there could be a book written to explain this topic. If you get stuck, or have a more specific question after doing a little reading, please let me know. I would be happy to help.

EDIT: Based on the added ip route show, you need to re-address one of your networks so the pi knows they are not connected. You may still have gateway issues, but that is where you need to start.
shareimprove this answer

edited Jun 18 ’13 at 16:13

answered Jun 17 ’13 at 22:38
Butters
1,339522

add a comment
up vote
3
down vote

eth0 is always preferred interface over wireless, you will need to issue command route -n to see your routes and then probably change default routing using:

$ sudo route add default gw 192.168.1.1 wlan0

just use correct address for your wireless router.

Blue Screen of Death Again!

Wget an ISO or other large file in the background

Linux, Unpolished Gems,

Let us forget the past. And remember that the past is a gift of the present, not a substitute for the future.

I was trying to download the Debian testing DVD ISO and it looked like it would take a while, many hours and I wanted to power off the machine.  This was back a while ago with slower internet but, this topic is still relevant. Normally I use the torrent for the distro file, but on the testing branch of Debian, none were available.

The solution

I have a Raspberry Pi, it’s running 24/7, let it do the work overnight and I can just power down my machine and not worry about the download.
Instead of downloading the file itself, I grabbed the link to the download location.
Then executed

wget -c https://gensho.ftp.acc.umu.se/cdimage/buster_di_alpha2/amd64/iso-dvd/debian-buster-DI-alpha2-amd64-DVD-1.iso
Output...
 --2018-02-07 18:15:27-- https://gensho.ftp.acc.umu.se/cdimage/buster_di_alpha2/amd64/iso-dvd/debian-buster-DI-alpha2-amd64-DVD-1.iso
 Resolving gensho.ftp.acc.umu.se (gensho.ftp.acc.umu.se)... 194.71.11.176, 2001:6b0:19::176
 Connecting to gensho.ftp.acc.umu.se (gensho.ftp.acc.umu.se)|194.71.11.176|:443... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 3864182784 (3.6G) [application/x-iso9660-image]
 Saving to: `debian-buster-DI-alpha2-amd64-DVD-1.iso'

Suceess!

Now all I have to do is put the task in the background via Ctrl-Z and then bg and I detach from SSH’ing into the R-Pi and it will just download in the background to the harddrive tethered to it’s USB port. When you enter bg it will still print it’s progress to the screen, but the terminal can be closed out fine.
There is a -b option for wget that will launch it, into the background from the start as well.

ps aux|grep wget

…will confirm that it is running still…

 erick 12438 7.0 2.2 13120 10996 ? S 18:15 2:46 wget -c https://gensho.ftp.acc.umu.se/cdimage/buster_di_alpha2/amd64/iso-dvd/debian-buster-DI-alpha2-amd64-DVD-1.iso

Watch

While in the directory that it is downloading a watch command can be used to see the progress of the download…

watch ls -l debian-buster-DI-alpha2-amd64-DVD-1.iso

 

Output…

Every 2.0s: ls -l debian-buster-DI-alpha2-amd64-DVD... Wed Feb 7 18:56:25 2018

-rw-r--r-- 1 erick erick 280608768 Feb 7 18:56 debian-buster-DI-alpha2-amd64-DV
 D-1.iso

This will show a progressive increase in file size, in case you want to monitor it.