Category Archives: Raspberry Pi Server

Penguin

Raspberry Pi Bridge from Wireless to Ethernet

The sky, when the stars rise, the stars fly away, the stars weep.

Setting up a bridge wlan0 to eth0 on a Raspberrypi

This is for an ipv4 Bridge connection between WAN and LAN using a Raspberrypi. Another Linux computer would also work instead of a Raspberrypi. Some of this is basically a dump of note that I took while trying to get this up and running in 2017. This post may be of use to someone who is struggling to get this sort of thing going as some of the sources on the net were not always clear on how to do it and validate it’s operation.


The idea with this plan is to have the R-Pi act as a router and hand out addresses to devices on the network and do a DNS masquerade on the eth0 connection. It will also forward ipV4 packets in BOTH directions, thereby bridging a wired and wireless networks. Note: Devices running on the LAN will require setting up a default route to the gateway on the WLAN to see devices on the WLAN.

Why Setup a wlan0 to eth0 Bridge

I had to do this to allow a connection between a ZTE WiFi hotspot that did not have any Ethernet connection port and a need to connect a set of desktop computers that only have Ethernet ports to the Internet. Essentially I have two networks, one is WiFi, one hardwired and they machines have to be able to reach the Internet and each other from both sides. The Ethernet machines are connected to a switch with a router connected to it acting as an access point, DHCP set to off. This network is on an “island” that needed to be bridge via WLAN to get out to the Internet via the ZTE hotspot.

Install dnsmasq

dnsmasq is a lightweight program that will run as a service that will take care of the DNS and the DHCP functions that are required to make the R-Pi act as a router and bridge.

sudo apt-get update && sudo apt-get upgrade -y && sudo apt-get install rpi-update dnsmasq -y
 sudo rpi-update

Static IP on Ethernet Connection

Next, setup a static ip for the dhcp server. Edit /etc/network/interfaces to make the R-Pi reside at a static address on the wired network. In this example it is setup for 192.168.1.17, the typical gateway address for a router on a 192.168.1.0 network, would be 192.168.1.1 which was the gateway at one time on this network. There is nothing magical about the 192.168.1.1 address, a default gateway can exist on any valid address, excluding 192.168.1.0 and 255.

NOTICE THAT THE DEFAULT GATEWAY FOR THE eth0 IS NOT SET! This is important because the default gateway should be grabbed from the WiFi network and not the wired, which in my case is not connected to the Internet. It will go to the Ethernet first by default if there are two default gateways, WLAN and LAN.

Router Settings

If there is a router on the network, it is important to turn off DHCP on it as it does not have to hand out addresses anymore. It should just behave as a switch instead, just forwarding packets in/out of all ports including WiFi if it has it and this option is wanted, then it will function as an AP ( Access Point) as well on it’s own network (192.168.1.0/24 in my case).

 

erick@raspberrypi ~ $ cat /etc/network/interfaces
 auto lo

iface lo inet loopback
 #iface eth0 inet dhcp

iface eth0 inet static
 address 192.168.1.17
 netmask 255.255.255.0
 network 192.168.1.0
 broadcast 192.168.1.255
 #gateway 192.168.1.1
 # nameservers
 dns-nameservers 8.8.8.8 8.8.4.4

allow-hotplug wlan0
 iface wlan0 inet manual
 wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
 iface default inet dhcp

Sanity Check using mtr, a.k.a. My Trace Route

Running mtr shows a direct route to Optiplex-790, running mtr from Optiplex-790 to 192.168.128.1 shows a bounce via the R-pi.

See Appendix 1 for more info.

Configure dnsmasq via /etc/dnsmasq.conf

Copy the rather wordy original to a backup copy and use sudo nano to edit in new details. The listen address will be the same as entered into the /etc/network/interfaces file for the R-Pi now that it is at a static address, mine is at 192.168.1.17 for example. Server is the dns server that dnsmasq will be using to do it’s masquerade magic. This can be a comma separated list. I have Google in there for DNS at 8.8.8.8, but an ISP would do as well. Sometimes the router upstream includes a caching DNS and it can be included as well. If the upstream router does have a caching DNS this helps a bit with lookups as the lookup table will be maintained locally as a cache of frequently visited web addresses. Having a local lookup for DNS has less delays than reaching out on the web for every word address to IP numeric address translation.

Address Reservation

For dncp-range I am choosing from 192.168.1.20-192.168.1.255 as the ones below 20 on my network are kept in reserve for static addresses.

Backup and then edit…

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig
 sudo nano /etc/dnsmasq.conf

/etc/dnsmasq.conf example

This is the one that I am using for the R-pi bridge. R-Pi is static on 192.168.1.17. DNS server is Googles 8.8.8.8 and I am reserving addresses from 192.168.1.0-20.

interface=eth0      # Use interface eth0
 listen-address=192.168.1.17 # Explicitly specify the address to listen on
 bind-interfaces      # Bind to the interface to make sure we aren't sending things elsewhere
 server=8.8.8.8       # Forward DNS requests to Google DNS
 domain-needed        # Don't forward short names
 bogus-priv           # Never forward addresses in the non-routed address spaces.
 dhcp-range=192.168.1.20,192.168.1.255,12h # Assign IP addresses between 192.168.1.20,192.168.1.255  with a 12 hour lease time

 

Enable IPv4 forwarding

The R-pi kernel has to be told explicitly to forward IPv4 packets between wlan0 and eth0.

sudo nano /etc/sysctl.conf

Fin and UNCOMMENT the following line

 net.ipv4.ip_forward=1

TO APPLY CHANGE WITHOUT A REBOOT

sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"

IPTABLES rules update

iptables needs a few rules added to it to cover the DNS masquerading and accepting packets forwarded from wlan0 to eth0 and in the other direction. Execute the following commands to add the rules to iptables.

sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
 sudo iptables -A FORWARD -i wlan0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
 sudo iptables -A FORWARD -i eth0 -o wlan0 -j ACCEPT

iptables will keep it’s info as long as the machine is booted up. Needs a reload when rebooted.

THE STUFF BELOW IS NOT NEEDED AS THE PI HAS A METHOD TO RESTORE THE IPTABLES VIA iptable-save.sh
HAVE NOT DONE THIS YET as of 12/12/2017

sudo sh -c “iptables-save > /etc/iptables.ipv4.nat”

HOOK FILE

sudo nano /lib/dhcpcd/dhcpcd-hooks/70-ipv4-nat

ADD…

iptables-restore < /etc/iptables.ipv4.nat

—————————————————-

 

—————————————————–

On the PC – Optiplex-790

I  set up the main desktop PC as static IP via Edit Connections on the GUI.

Configure Wired Connection 1 as IPv4 Settings, Manual with address as 192.168.1.18 netmask 255.255.255.0 and gateway as the R-pi at 192.168.1.17

DNS servers, the R-pi itself 192.168.1.17 and Google at 8.8.8.8. It might be possible to have 192.168.128.1 as well as that might be a caching DNS on the upstream ZTE WiFi hotspot. It might have a caching DNS server inside of the Sprint Box itself, not sure and requires looking at the specs for it.

Feasibility Test Using a PC as a Bridge

Before I went through the trouble of setting up the R-Pi as a bridge I did a sanity check/prototype by using a PC as a bridge. By taking the WiFi USB dongle and plugging it into the PC, Optiplex-790 running Linux Mint. I was able to test the feasibility out beforehand.

Create shared connection

On an initial test of setting up the network, before digging into the R-pi to do this, I shared the wifi to the eth via editing the Wired Connection 1 and sharing under IPv4 Settings. But it puts it on a 10.42.0.x network. Use the following to change this…

In versions before 1.4.2, 10.42.0.x is hard-coded into NetworkManager. The choice is either upgrade to Ubuntu 17.04, with version 1.4.4, or go the easy way and use the following command from Thomas Haller to set the host IP and class. For my setup it was 192.168.1.18…

nmcli connection modify $CONNECTION_ID +ipv4.addresses 192.168.1.18/24

where $CONNECTION_ID if found via…

 nmcli connection show

… Afterwards, verify with…

nmcli connection show $CONNECTION_ID.

from …
https://askubuntu.com/questions/609645/configure-connection-sharing-with-specific-ip-address

ANY CHANGES MADE TO THE CONNECTION EDITOR REQUIRES A DISCONNECT AND RECONNECT TO APPLY THE CHANGES!!

Uncheck and Recheck Enable Networking

erick@OptiPlex-790 ~ $ nmcli connection show
 NAME                   UUID                                  TYPE             DEVICE
 Wired connection 1     1a2d9768-104d-3714-814c-57ea2faff63b  802-3-ethernet   eno1
 NETGEAR63              13f543ac-ea4b-455e-9ff7-9e0ecaddb139  802-11-wireless  --
 SprintHotspot2.4-BA3A  b85d60f9-5875-4eaa-a0c0-df43d174c869  802-11-wireless  --
...Verification that the change took hold after setting via nmcli connection modify command...
 erick@OptiPlex-790 ~ $ nmcli connection show 1a2d9768-104d-3714-814c-57ea2faff63b
 connection.id:                          Wired connection 1
 connection.uuid:                        1a2d9768-104d-3714-814c-57ea2faff63b
 connection.interface-name:              --
 connection.type:                        802-3-ethernet
 connection.autoconnect:                 yes
 connection.autoconnect-priority:        -999
 connection.timestamp:                   1513298233
 connection.read-only:                   no
 connection.permissions:
 connection.zone:                        --
 connection.master:                      --
 connection.slave-type:                  --
 connection.autoconnect-slaves:          -1 (default)
 connection.secondaries:
 connection.gateway-ping-timeout:        0
 connection.metered:                     unknown
 connection.lldp:                        -1 (default)
 802-3-ethernet.port:                    --
 802-3-ethernet.speed:                   0
 802-3-ethernet.duplex:                  full
 802-3-ethernet.auto-negotiate:          yes
 802-3-ethernet.mac-address:             18:03:73:D1:52:FC
 802-3-ethernet.cloned-mac-address:      --
 802-3-ethernet.mac-address-blacklist:
 802-3-ethernet.mtu:                     auto
 802-3-ethernet.s390-subchannels:
 802-3-ethernet.s390-nettype:            --
 802-3-ethernet.s390-options:
 802-3-ethernet.wake-on-lan:             1 (default)
 802-3-ethernet.wake-on-lan-password:    --
 ipv4.method:                            manual
 ipv4.dns:                               192.168.1.17,8.8.8.8
 ipv4.dns-search:
 ipv4.dns-options:                       (default)
 ipv4.dns-priority:                      0
  ipv4.addresses:                         192.168.1.18/24
 ipv4.gateway:                           192.168.1.17
 ipv4.routes:
 ipv4.route-metric:                      -1
 ipv4.ignore-auto-routes:                no
 ipv4.ignore-auto-dns:                   no
 ipv4.dhcp-client-id:                    --
 ipv4.dhcp-timeout:                      0
 ipv4.dhcp-send-hostname:                yes
 ipv4.dhcp-hostname:                     --
 ipv4.dhcp-fqdn:                         --
 ipv4.never-default:                     no
 ipv4.may-fail:                          yes
 ipv4.dad-timeout:                       -1 (default)
 ipv6.method:                            auto
 ipv6.dns:
 ipv6.dns-search:
 ipv6.dns-options:                       (default)
 ipv6.dns-priority:                      0
 ipv6.addresses:
 ipv6.gateway:                           --
 ipv6.routes:
 ipv6.route-metric:                      -1
 ipv6.ignore-auto-routes:                no
 ipv6.ignore-auto-dns:                   no
 ipv6.never-default:                     no
 ipv6.may-fail:                          yes
 ipv6.ip6-privacy:                       0 (disabled)
 ipv6.addr-gen-mode:                     stable-privacy
 ipv6.dhcp-send-hostname:                yes
 ipv6.dhcp-hostname:                     --
 GENERAL.NAME:                           Wired connection 1
 GENERAL.UUID:                           1a2d9768-104d-3714-814c-57ea2faff63b
 GENERAL.DEVICES:                        eno1
 GENERAL.STATE:                          activated
 GENERAL.DEFAULT:                        yes
 GENERAL.DEFAULT6:                       no
 GENERAL.VPN:                            no
 GENERAL.ZONE:                           --
 GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/14
 GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/1
 GENERAL.SPEC-OBJECT:                    /
 GENERAL.MASTER-PATH:                    --
  IP4.ADDRESS[1]:                         192.168.1.18/24
 IP4.GATEWAY:                            192.168.1.17
 IP4.ROUTE[1]:                           dst = 169.254.0.0/16, nh = 0.0.0.0, mt = 1000
 IP4.DNS[1]:                             192.168.1.17
 IP4.DNS[2]:                             8.8.8.8
 IP6.ADDRESS[1]:                         fe80::7abb:ec07:22dc:c7bd/64
 IP6.GATEWAY:

Looking at the ZTE WiFi Hotspot as seen from the Optiplex-790

erick@OptiPlex-790 ~ $ nmcli connection show b85d60f9-5875-4eaa-a0c0-df43d174c869
 connection.id:                          SprintHotspot2.4-BA3A
 connection.uuid:                        b85d60f9-5875-4eaa-a0c0-df43d174c869
 connection.interface-name:              --
 connection.type:                        802-11-wireless
 connection.autoconnect:                 yes
 connection.autoconnect-priority:        0
 connection.timestamp:                   1513128371
 connection.read-only:                   no
 connection.permissions:
 connection.zone:                        --
 connection.master:                      --
 connection.slave-type:                  --
 connection.autoconnect-slaves:          -1 (default)
 connection.secondaries:
 connection.gateway-ping-timeout:        0
 connection.metered:                     unknown
 connection.lldp:                        -1 (default)
 802-11-wireless.ssid:                   SprintHotspot2.4-B838

802-11-wireless.mode:                   infrastructure
 802-11-wireless.band:                   --
 802-11-wireless.channel:                0
 802-11-wireless.bssid:                  --
 802-11-wireless.rate:                   0
 802-11-wireless.tx-power:               0
 802-11-wireless.mac-address:            08:86:3B:04:85:88
 802-11-wireless.cloned-mac-address:     --
 802-11-wireless.mac-address-blacklist:
 802-11-wireless.mac-address-randomization:default
 802-11-wireless.mtu:                    auto
 802-11-wireless.seen-bssids:            34:69:87:BB:B8:38
 802-11-wireless.hidden:                 no
 802-11-wireless.powersave:              default (0)
 802-11-wireless-security.key-mgmt:      wpa-psk
 802-11-wireless-security.wep-tx-keyidx: 0
 802-11-wireless-security.auth-alg:      --
 802-11-wireless-security.proto:
 802-11-wireless-security.pairwise:
 802-11-wireless-security.group:
 802-11-wireless-security.leap-username: --
 802-11-wireless-security.wep-key0:      <hidden>
 802-11-wireless-security.wep-key1:      <hidden>
 802-11-wireless-security.wep-key2:      <hidden>
 802-11-wireless-security.wep-key3:      <hidden>
 802-11-wireless-security.wep-key-flags: 0 (none)
 802-11-wireless-security.wep-key-type:  0 (unknown)
 802-11-wireless-security.psk:           <hidden>
 802-11-wireless-security.psk-flags:     0 (none)
 802-11-wireless-security.leap-password: <hidden>
 802-11-wireless-security.leap-password-flags:0 (none)
 ipv4.method:                            auto
 ipv4.dns:                               8.8.8.8,8.8.4.4
 ipv4.dns-search:
 ipv4.dns-options:                       (default)
 ipv4.dns-priority:                      0
 ipv4.addresses:
 ipv4.gateway:                           --
 ipv4.routes:
 ipv4.route-metric:                      -1
 ipv4.ignore-auto-routes:                no
 ipv4.ignore-auto-dns:                   no
 ipv4.dhcp-client-id:                    --
 ipv4.dhcp-timeout:                      0
 ipv4.dhcp-send-hostname:                yes
 ipv4.dhcp-hostname:                     --
 ipv4.dhcp-fqdn:                         --
 ipv4.never-default:                     no
 ipv4.may-fail:                          yes
 ipv4.dad-timeout:                       -1 (default)
 ipv6.method:                            auto
 ipv6.dns:
 ipv6.dns-search:
 ipv6.dns-options:                       (default)
 ipv6.dns-priority:                      0
 ipv6.addresses:
 ipv6.gateway:                           --
 ipv6.routes:
 ipv6.route-metric:                      -1
 ipv6.ignore-auto-routes:                no
 ipv6.ignore-auto-dns:                   no
 ipv6.never-default:                     no
 ipv6.may-fail:                          yes
 ipv6.ip6-privacy:                       0 (disabled)
 ipv6.addr-gen-mode:                     stable-privacy
 ipv6.dhcp-send-hostname:                yes
 ipv6.dhcp-hostname:                     --


Note: Needed to Add a Route on a Machine Connected to Raspberry Pi via Ethernet

I needed to add a route to the rpi to get to the 192.168.1.0/24 network.

I THOUGHT that this had worked automatically initially. It seemed that I could at least get to the pi at http://raspberrypi and 192.168.1.17.

But really a route is needed to the 192.168.1.0/24 network via the raspberrypi.rputer on 192.168.128.X

sudo route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.128.46

there is a helper file for this in ~/bin add-route-to-192.168.1.0.sh

erick@media-pc ~/Music $ route
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 default         192.168.128.1   0.0.0.0         UG    600    0        0 wlx08863b02838f
 link-local      *               255.255.0.0     U     1000   0        0 wlx08863b02838f
 192.168.1.0     raspberrypi.rou 255.255.255.0   UG    0      0        0 wlx08863b02838f
 192.168.128.0   *               255.255.255.0   U     600    0        0 wlx08863b02838f

using mtr to a device on the 192.168.1.0 network confirms hw packets are routed through!

————————————————————————————–

NETSTAT VERY HELPFUL

netstat –help
usage: netstat [-vWeenNcCF] [<Af>] -r         netstat {-V|–version|-h|–help}
netstat [-vWnNcaeol] [<Socket> …]
netstat { [-vWeenNac] -i | [-cWnNe] -M | -s }

-r, –route              display routing table
-i, –interfaces         display interface table
-g, –groups             display multicast group memberships
-s, –statistics         display networking statistics (like SNMP)
-M, –masquerade         display masqueraded connections

-v, –verbose            be verbose
-W, –wide               don’t truncate IP addresses
-n, –numeric            don’t resolve names
–numeric-hosts          don’t resolve host names
–numeric-ports          don’t resolve port names
–numeric-users          don’t resolve user names
-N, –symbolic           resolve hardware names
-e, –extend             display other/more information
-p, –programs           display PID/Program name for sockets
-c, –continuous         continuous listing

-l, –listening          display listening server sockets
-a, –all, –listening   display all sockets (default: connected)
-o, –timers             display timers
-F, –fib                display Forwarding Information Base (default)
-C, –cache              display routing cache instead of FIB

<Socket>={-t|–tcp} {-u|–udp} {-w|–raw} {-x|–unix} –ax25 –ipx –netrom
<AF>=Use ‘-6|-4’ or ‘-A <af>’ or ‘–<af>’; default: inet
List of possible address families (which support routing):
inet (DARPA Internet) inet6 (IPv6) ax25 (AMPR AX.25)
netrom (AMPR NET/ROM) ipx (Novell IPX) ddp (Appletalk DDP)
x25 (CCITT X.25)

—————————————————————————————

APPENDIX 1:

https://raspberrypi.stackexchange.com/questions/8010/internet-access-via-1-of-2-network-interfaces

You need to verify that you DO NOT have a default gateway set on your ETH0 interface. It has probably been assigned by DHCP, so you may have to address it statically, edit your router configuration. You will also need to verify that you have a default gateway on your WLAN interface.

Post the output of ip route show when both interfaces are connected for more detailed response.

You also need to ensure that your two routers are on different subnets. For example, the network connected to ETH0 could be 192.168.1.0 255.255.255.0, and WLAN0 could be 192.168.0.0 255.255.255.0, but they have to be on different networks. More on subnet mask

Finally you may want to read up on local routing for Debian systems.

Sorry I can’t be more specific, but there could be a book written to explain this topic. If you get stuck, or have a more specific question after doing a little reading, please let me know. I would be happy to help.

EDIT: Based on the added ip route show, you need to re-address one of your networks so the pi knows they are not connected. You may still have gateway issues, but that is where you need to start.
shareimprove this answer

edited Jun 18 ’13 at 16:13

answered Jun 17 ’13 at 22:38
Butters
1,339522

add a comment
up vote
3
down vote

eth0 is always preferred interface over wireless, you will need to issue command route -n to see your routes and then probably change default routing using:

$ sudo route add default gw 192.168.1.1 wlan0

just use correct address for your wireless router.

Raspberry Pi WiFi via USB device

Setting WiFi up via the command line on Raspberry Pi, using a USB Wireless Adapter

These are notes on how to setup WiFi on the Raspberry Pi. The R-Pi is a model 2 running Raspbian 4.1.19+.

In my case and in the example that follows, the Raspberry Pi is connected to an Ethernet network, static IP at 192.168.1.17. This is the address that  I am logging in via SSH to get to the command line to configure the USB WiFi. Adapter.

USB WiFi Adapters

Two USB Wifi’s were tried a Belkin N300 and an Edimax EW7811Un. Both use the Realtek RTL8192CU chip and work well with the R-pi. Initial testing was with the Belkin and the output from this device is used in this post for the command line examples.

Edimax EW7811Un 150Mbps 11n WiFi USB Adapter, it is nano size and has a blue activity light. It works well, can’t imagine how small the antenna is in there and how they get RF to work out OK with these sub wavelength antennas!

NOTE: Originally the adapters were tried by plugging them into a powered USB hub which plugged into the R-pi. This allows for hot-plugging. If a device is hot-plugged directly into the R-Pi it will force a reboot, at least on the one that I have (R-Pi Model 2B). This is probably due to an inrush current that pulls down the power bus momentarily, I am guessing. The powered USB hub isolates the R-Pi from the devices connected to it as far as power is concerned and things will hot plug fine using it. I did realize later on that when I plugged the USB WiFi adapter directly into the R-Pi, I got more stable behavior with it. As in less strange dropouts of the WiFi from the network. It maintains a network connection better for me at least, plugged in directly.

Initial Testing

The first steps involve checking to see that the adapter is detected, registered and the correct device driver is loaded. They are just confirmation that all is well. They can be skipped and then ran later if problems arise and troubleshooting is needed.

Plug in USB WiFi adapter and run a lsusb and dmesg…

lsusb
erick@raspberrypi ~ $ lsusb
Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. 
Bus 001 Device 004: ID 2101:8500 ActionStar 
Bus 001 Device 005: ID 2101:8501 ActionStar 
Bus 001 Device 006: ID 154b:005b PNY 
Bus 001 Device 008: ID 050d:2103 Belkin Components F7D2102 802.11n N300 Micro Wireless Adapter v3000 [Realtek RTL8192CU]
Bus 001 Device 007: ID 174c:1153 ASMedia Technology Inc.

The device shows up fine using lsusb. Now on to dmesg to see if the correct driver loaded…

[156238.694964] usb 1-1.3.3: new high-speed USB device number 8 using dwc_otg
[156238.797146] usb 1-1.3.3: New USB device found, idVendor=050d, idProduct=2103
[156238.797188] usb 1-1.3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[156238.797207] usb 1-1.3.3: Product: Belkin Wireless Adapter
[156238.797225] usb 1-1.3.3: Manufacturer: Realtek
[156238.797242] usb 1-1.3.3: SerialNumber: 00e04c000001
[156239.201673] usbcore: registered new interface driver rtl8192cu

Kernel driver rtl8192cu is loaded, all should be well with the adapter!

lsmod

The following lsmod was run and it confirms the kernel module is loaded for the 8192cu driver. It is just added confirmation that all is well.

 

erick@raspberrypi ~ $ lsmod
 Module                  Size  Used by
 xt_state                1434  1
 ipt_MASQUERADE          1220  1
 nf_nat_masquerade_ipv4     2814  1 ipt_MASQUERADE
 iptable_nat             2120  1
 nf_nat_ipv4             6162  1 iptable_nat
 nf_nat                 17132  2 nf_nat_ipv4,nf_nat_masquerade_ipv4
 8192cu                556175  0
 nfsd                  263815  11
 nf_conntrack_ipv4      14388  3
 nf_defrag_ipv4          1766  1 nf_conntrack_ipv4
 xt_conntrack            3420  1
 nf_conntrack           95316  6 nf_nat,xt_state,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4
 iptable_filter          1698  1
 ip_tables              12362  2 iptable_filter,iptable_nat
 x_tables               18590  5 ip_tables,ipt_MASQUERADE,xt_state,xt_conntrack,iptable_filter
 i2c_dev                 6386  4
 snd_bcm2835            22502  0
 snd_pcm                92861  1 snd_bcm2835
 snd_seq                58152  0
 snd_seq_device          5142  1 snd_seq
 snd_timer              22156  2 snd_pcm,snd_seq
 snd                    67534  5 snd_bcm2835,snd_timer,snd_pcm,snd_seq,snd_seq_device
 sg                     20575  0
 i2c_bcm2708             5988  0
 bcm2835_gpiomem         3703  0
 bcm2835_rng             2207  0
 uio_pdrv_genirq         3526  0
 uio                    10078  1 uio_pdrv_genirq

 

Setup the WiFi Connection

In this example this is a WPA type of security. I know the SSID and password and just put them into the wpa_supplicant configuration file. If you need to see what WiFi nodes are available on the network consider using the scan …

( IF Needed )

sudo iwlist wlan0 scan

Check /etc/network/interfaces

Check to see that the section exists in the file that will allow the USB apapter to hot plug and also the wpa-roam line that points to the wpa_supplicant.conf file where the SSID and password will be stored in the next step.

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

…any changes to the interfaces file will require a reboot or a restart of networking to take effect, via….

 sudo service networking restart

If you are running the R-Pi headless it will disconnect from SSH and will require a re-login. If a mistake is made in the interfaces file, it might not come back and require connecting a keyboard and monitor to reconnect. The good news about having both a running eth0 and wlan0 is that if you make a mistake in only one of them it will be possible to connect via the other one. Less chances of being totally locked out by a small typo in the interfaces file. Sometimes a restart of network services will cause a non-recoverable dropout which requires a reboot to get an SSH connection going again.

Config wpa_supplicant
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Go to the bottom of the file and add the following type of entry, putting in the correct SSID and pwd:

network={
 ssid="SprintHotspot2.4-example"
 psk="thepassword"
 }

Save and Exit

Then execute the follow to apply the new configuration….
wpa_cli -i wlan0 reconfigure

TEST IT:

ifconfig wlan0

 

Results show that the interface is up and and running

erick@raspberrypi ~/Music/music-test $ ifconfig wlan0
wlan0     Link encap:Ethernet  HWaddr 74:df:e0:e0:0e:e0  
          inet addr:192.168.128.46  Bcast:192.168.128.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4023854 errors:0 dropped:1664207 overruns:0 frame:0
          TX packets:2955774 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3003299485 (2.7 GiB)  TX bytes:1956045437 (1.8 GiB)

It should be working at this stage. Trying to reach the net using something like ping google.com should give good results. If not more troubleshooting is required. I had to do the next step to get it to reach the net on the wireless network as it was trying to use the Ethernet connection to the router to get out, set as the default gateway, which was not hooked up to the web at all. Just a router at 192.168.1.1 and no WAN port connection.

Additional Step

The following bit may or may not apply for everyone. But, I am adding here as it was not obvious at the moment I got WiFi up. I had to think on it a bit! Basically a default route has to exist that takes it to a gateway to the Internet.

IN ORDER TO GET PI OUT ON INTERNET NEEDED TO DO A

sudo route add default gw 192.168.128.1 wlan0

As there was no route out and it must pick eth0 by priority! Needs a route to the Sprint Box that I am connected to the net on the 192.168.128.0 network.

THE ABOVE WOULD HAVE TO HAPPEN ON EVERY REBOOT OR NETWORK REFRESH! Or just get rid of default gateway on ETH0 and it might just pick up the gateway on the wlan0 all of the time! If both are dhcp the eth0 gateway will always  be treated as preferred, so going static will get rid of it as I was planning on using this R-pi as a bridge from LAN to WLAN! This required editing /etc/network/interfaces to remove the 192.168.1.1 router as a gateway. The router was reconfigured by turning off DHCP on it, effectively making it an access point for WiFi. Essentially it becomes another path to the Internet along with the WiFi hotspot.

erick@raspberrypi ~ $ route -n
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
 192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
 192.168.128.0   0.0.0.0         255.255.255.0   U     0      0        0 wlan0
 erick@raspberrypi ~ $ sudo route add default gw 192.168.128.1 wlan0
 erick@raspberrypi ~ $ route -n
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 0.0.0.0         192.168.128.1   0.0.0.0         UG    0      0        0 wlan0
 0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
 192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
 192.168.128.0   0.0.0.0         255.255.255.0   U     0      0        0 wlan0
SHOWS UP IN /etc/resolv.conf as well….
erick@raspberrypi ~ $ cat /etc/resolv.conf
 domain router
 search router
 nameserver 192.168.128.1

 

ip show route as well…
erick@raspberrypi ~ $ ip route show
 default via 192.168.128.1 dev wlan0
 default via 192.168.1.1 dev eth0
 192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.17
 192.168.128.0/24 dev wlan0  proto kernel  scope link  src 192.168.128.46

 /etc/network/interfaces

The following /etc/network/interfaces file was edited to make the wlan0 connection which connected to the Internet work for the R-Pi.

Note the eth0 connection is setup static on the 192.168.1.0 wired network. The wlan0 connection is set for dhcp on the 192.168.128.0. The default gateway at 192.168.1.1 ( the router ) is commented out to allow the default to fall on the 192.168.128.1 WiFi router, which is a ZTE WiFi hotspot, basically a repeater from 4G LTE cell to WiFi.

Note the wpa-roam points to the wpa_supplicant file that has the SSID and password entered earlier in this post to get the WiFi going.

erick@raspberrypi ~/Music/music-test $ sudo cat /etc/network/interfaces
[sudo] password for erick: 
auto lo

iface lo inet loopback
#iface eth0 inet dhcp

iface eth0 inet static
address 192.168.1.17
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
# Remove gateway to see if it fails-over to wlan0 gateway on 192.168.128.1 12242017
# gateway 192.168.1.1
# nameservers
dns-nameservers 8.8.8.8 8.8.4.4

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

ALL UP AND RUNNING OK

Running ip a shows all interfaces. Note that the R-pi is connected to wlan0 on the 192.168.128.46 address and eth0 is connected at 192.168.1.17, both networks are now available.

erick@raspberrypi ~ $ ip a
 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
 valid_lft forever preferred_lft forever
 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
 link/ether b8:2a:eb:2a:a4:2a brd ff:ff:ff:ff:ff:ff
 inet 192.168.1.17/24 brd 192.168.1.255 scope global eth0
 valid_lft forever preferred_lft forever
 3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
 link/ether 08:8a:3c:ba:83:8a
 brd ff:ff:ff:ff:ff:ff
 inet 192.168.128.46/24 brd 192.168.128.255 scope global wlan0
 valid_lft forever preferred_lft forever

The connection to the WiFi node can be confirmed via iwconfig wlan0…

erick@raspberrypi ~ $ iwconfig wlan0    

 IEEE 802.11bg  ESSID:"SprintHotspot2.4-example"  Nickname:"<WIFI@REALTEK>"
 Mode:Managed  Frequency:2.452 GHz  Access Point: 34:3A:87:3A:BA:3A
 Bit Rate:54 Mb/s   Sensitivity:0/0
 Retry:off   RTS thr:off   Fragment thr:off
 Power Management:off
 Link Quality=100/100  Signal level=63/100  Noise level=0/100
 Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
 Tx excessive retries:0  Invalid misc:0   Missed beacon:0

lo        no wireless extensions.

eth0      no wireless extensions.

 

Script to check and re-connect WiFi on Raspberry Pi

After having occasional dropouts of the Raspberry Pi Wifi ( after I started using it as a bridge) and having to ifup the wlan0 manually, I considered making a script to automate it. But, a little searching online found this elegant solution to the problem.

mharizanov/WiFiCheck

https://gist.github.com/mharizanov/5325450

Temperature, Humidity and Barometric Pressure Monitoring with Raspberry Pi

Brief notes on hooking up sensors for temperature, humidity and barometric sensors. These notes are kind of an outline of what I used and went through to get the sensors up an running. There are plenty of other detailed notes out there on the web, this might help fill in some blanks. Plus I figured, I took all these notes down and they might as well be published to do some more good.

Raspberry Pi Pinout

Model B IO Pins

DHT22 / AM2302

https://www.adafruit.com/products/385?&main_page=product_info&products_id=385

https://www.adafruit.com/products/393?&main_page=product_info&products_id=393

I followed this example to get the DHT22 sensor up and running all the way to executing the example code

https://learn.adafruit.com/downloads/pdf/dht-humidity-sensing-on-raspberry-pi-with-gdocs-logging.pdf

Raspberry Pi Weather Dashboard

I put all if this together to make a dashboard page for the sensors.

http://erick.mynetgear.com

 

 

USB Stick Fix for NTFS filesystem on Raspberry Pi

I’ve used USB sticks on the Raspberry Pi before, many times. So I plugged one in for some extra storage. It let me write once and then went into r/o mode. What the hell! Pulled it, plugged it into the PC, no problem. Plugged it into a Windows 7 machine, it complained about drive errors and fixed it. Try again, no luck. So I gave up for a while, it was full of Music that I play through the Sockso Music Server, so no big deal, but I really do want to use it for external storage as the SD card will fill up eventually. Plus I would like to use it as more storage for ftp as well and possibly for Sparkleshare which I intend to install.

One more look

It turns out that at some point while fiddling with the USB drive, I formatted it NTFS. I believe that I tried to format it ext4 and then ext2, neither worked. So I used a WIndows machine to make it NTFS, so it would work.

The issue is that the Rasp Pi does not come with NTFS support out of the box. So I needed to  install ntfsprogs to get it to work OK….

sudo apt-get install nfsprogs

Otherwise it immediately goes to read only. I have not tested but I assume nfsprogs also installs tools to check NTFS file-systems.

To mount my USB stick, I Need to execute …

sudo mount -t ntfs -o rw,uid=erick,gid=erick /dev/sda1 /media/sda/

Symlinks for Sockso Music Server

I have a symlink for music in my Music dir (/home/erick/Music. Set it to, main-collection -> /media/sda/Music/ , in this manner it points to the music collection on the usb stick and Sockso happily finds the music. Sockso does not like spaces in filenames!

 

 

Using mount with bind to access usb drive via vsFTP

I have a USB stick plugged into my Raspberry Pi for external storage, mostly to put music on for the Sockso Music Server to get at. But I wanted to use it a bit more for generic storage. FTP is great, you can get to it from any machine and the command line for it is the same on Win or Linux. So I can walk up to any machine, not have to install a thing and reach into a folder with FTP.

For instance, I have an infected Windows Machine, I don’t dare stick a USB stick in it. Instead I go to the command line, ftp to the Raspberry Pi and grab the tools I need from there.

The Issue

The issue was that I tried to symlink from the ftp directory to the USB drive. vsFTP will not follow symlinks for security reasons.

The Solution

Mount the directory you want under the FTP directory using bind. /media/sda is the USB stick mount point and the whole thing gets mounted under the FTP dir using…

sudo mount --bind /media/sda/ /home/ftpuser/usb-drive/

Resources

FTP on Raspberry Pi. An easy way to make shared folders

Alternatives to FTP

https://radu.cotescu.com/vsftpd-and-symbolic-links/

sSMTP Installing and Configuration and Use Tips

Recently I was looking at creating a method of sending a warning email when ever my house temperature went below a threshold. I remembered that sSMTP was a simple way to send automated emails and CRON emails. I have some simple notes on what I did.

Installation

Very easy, just use apt-get from the command line…

sudo apt-get install ssmtp

Configuring

The configuration file (/etc/ssmtp/ssmtp.conf) can be edited using any test editor you typically use.

 

Config at /etc/ssmtp/ssmtp.conf

Below is my config file with the critical info blocked out. Lines in Red are what I modded to get ssmtp working for me.

The key pieces to get it working for me at least were…

hostname = My ISP’s domain

root = my complete email that I use at the ISP

mailhub = I looked it up in Thunderbird, it is the smtp.myispsdomain.net part.

AuthUser=my complete email that I use at the ISP. It might be different for you. Years ago it used to be just the user name part of email without the domain.

AuthPass = The password that goes along with my email.

I commented out the defaults for the ones that existed in the code.

The config file is a bit ugly after I touched it but I was trying to get this up and running quick and didn’t clean it up. But, hey it works!

 

#
 # Config file for sSMTP sendmail
 #
 # The person who gets all mail for userids < 1000
 # Make this empty to disable rewriting.
 #root=postmaster  <--- comment out
 
# The place where the mail goes. The actual machine name is required no
 # MX records are consulted. Commonly mailhosts are named mail.domain.com
  #mailhub=mail <-- comment out
 
# Where will the mail seem to come from?
 #rewriteDomain=
# The full hostname
 #hostname=raspberrypi <--- I was testing and kill this, failed to work
 # hostname has to be the mail domain! Or else it complains about
  # the raspberrypi part! The STMP server at frontier does that is.
  hostname=myispdomain.net
# Are users allowed to set their own From: address?
 # YES - Allow the user to specify their own From: address
 # NO - Use the system generated From: address
 #FromLineOverride=YES <-- Commented out and set below, I was testing!
# New Code put here 11302015
  root=me@myispdomain.net
  mailhub=smtp.myispdomain.net
AuthUser=me@myispdomain.net
AuthPass=myemailpassword
FromLineOverride=YES
#UseSTARTTLS=YES <-- Tried this, I didn't need it for my ISP.

CRON Email

Once installed if you or root on the machine have any CRON jobs, you will start to get email from them. You can stop this by appending …

> /dev/null 2>&1

to the end of the commands that are being run by CRON. Which will cut back on the emails that you will receive.

 Testing

I installed mail utils to allow sending simple messages…

sudo apt-get install mailutils

Then I sent a message via the command line…

echo "Test" | mail -s "Test Subject" me@myispsdomain.net

…and I was able to see it work OK.

Send files via email

If you want to send files you have to install mpack.

sudo apt-get install mpack

 

Then you can send files to your email like this…

mpack -s "Test" /tmp/web/log.txt me@myispsdomain.net

 Command Line Usage

If you execute ssmtp with an email address it will let you create an email from the command line. Which is good for quick emails to for example remind yourself of something, or send a snippet of code to yourself. You edit the email in the form of the example below and hit Ctrl-D when done and then it will send out.
ssmtp recipient_email@example.com
The following is an example right off the command line. Note the one line of space after the Subject, this is a must have…
erick@raspberrypi ~ $ ssmtp me@myispdomain.net
To:me@myispdomain.net
From:me@myispdomain.net
Subject:This is a test of ssmtp from the command line!

Hello there this is a test of the ssmtp from the command line tool. It could be used to send a reminder or small snips of code. Use Ctrl-D when you are done.

It is called up by using ssmtp emailtosendto@domain.com

Bye,
Me

Example of Sending CPU Temp Warning Emails

When I am away from home I can infer if my house is running to cold, which may indicate a problem with the furnace. The Raspberry Pi is light loaded, usually just idling, so the CPU temperature tracks the room temperature, with an offset. When I am away, I set the house thermostat at 47 degrees F. If it drops below this value the CPU temperature of the Raspberry Pi will drop below 34 degrees Celsius. So I can just have it send me an email if this happens. Then I can double check a log that is created of the temperature reading to see what is going on. Also I run a webcam pointed at an actual thermometer for a sanity check, this is logged by using fswebcam to take an hourly snapshot. So I have my bases covered for the most part. Obviously if the power is out, I am in the dark about the temperature, because the whole thing is down! Solving that is a future project.

Below is the snippet of code from a shell script that sits in /etc/cron.hourly that handles the warning emails that are sent to 2 addresses. variables mailaddr and mailaddr2.

temp is the CPU temperature in Celsius as an integer stripped using cut from the thermal_zone0 reading.

minimum and maximum are my temperature thresholds. I don’t care much about maximum but I have it set at 65 Deg. C. just in case.

# Read the temp and cut it to grab leftmost 2 characters, integer Temp
temp="`cat /sys/class/thermal/thermal_zone0/temp | cut -c1-2`"
#echo $temp

# Mail if about or below the limits
if (( $temp > $maximum )); then
   #echo "above"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi HIGH CPU Temp > $maximum" $mailaddr2

elif (( $temp < $minimum )); then
   #echo "below"
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr
   echo "Rasp Pi CPU Temp = $temp. " | mail -s "Rasp Pi LOW CPU Temp < $minimum" $mailaddr2

fi

Boot Email

I want to know if an when the Raspberry Pi I run 24/7 ever reboots due to a power outage, so I have it send me an email. The line of code below handles it and is in the root crontab. I have it sleep for 180 seconds first, then send the email. This allows the cascaded routers which I have the Pi connected to and the cable modem, time to come on line.

@reboot sleep 180 && echo "Rasp Pi Rebooted" | mail -s "Rasp Pi Reboot!" me@myispsdomain.net

I also log boots in a file that I can view online, just to keep track in one record.

@reboot date >> /var/www/bootlog.txt

Keeping track of boots helps for instance if I am away from home and the power goes out. If I get the email that the Pi rebooted, I can check to see how long the power was down and what the temperature of the house is to see if all is well.

Every hour I take a time/date stamped webcam snapshot of a thermometer so I can just look to see how many are missing and have a rough estimate of how long the power was out and how cold the house got and verify that it is getting warmer because the furnace is on!

In the future I will connect a BME280 sensor to the Raspberry Pi that will be able to read ambient room temperature directly, along with humidity and barometric pressure. So I won’t have to infer the house temperature via the CPU temperature.

Resources

This is the page I used to configure ssmtp on the Rasp Pi.

http://www.raspberry-projects.com/pi/software_utilities/email/ssmtp-to-send-emails

Raspberry Pi

Reduce writes to the Raspberry Pi SD card

After 5 months of solid up-time for my Raspberry Pi server, which has been running great. It has been taking a picture every hour and from them creating a creating a timelapse video every day. Also it is being used as a place to drop files to periodically from other place on the network, a little bit of file storage. Eventually I will add more storage space to it to use it even more for network storage.

Recently, I started to think about the potential wear of the SD card as I came across several articles online dealing with the topic. I decided to make a few changes to the Raspberry Pi configuration to reduce the amount of writing to the SD card.

Write Saving #1: Using a tmpfs

I editted /etc/default/tmpfs. In it the comments state  that /run, /run/lock and /run/shm are already mounted as tmpfs on the Pi by default. Which I have observed. This was a change made a while ago for the Pi according to the buzz online. I additionally set RAMTMP=Yes to add /tmp to the directories put on the tmpfs. This sets up access to /tmp with rwx-rwx-rwx permissions. There was a suggestion that I saw online to limit the sizes of the various directories, I added that as well.

# These were recommended by http://raspberrypi.stackexchange.com/questions/169/how-can-i-extend-the-life-of-my-sd-card
# 07262015, mods for using less of the SD card, RAM optimization.
TMPFS_SIZE=10%VM
RUN_SIZE=10M
LOCK_SIZE=5M
SHM_SIZE=10M
TMP_SIZE=25M

The OS and some programs will use /tmp. But so do I. I created a /tmp/web folder under it when the Raspberry Pi boots. Into this folder files go such as the hourly photo and the daily video that scripts create for the webcam that is attached. I have reduced 3 hourly writes to just one photo. I keep only one on the SD card as I don’t want to risk losing a bunch of them taken during the day if I totally relied on the tmpfs. If I was using a UPS, I would have no problem saving all of them on the tmpfs and occasionally backing up to the SD or another device. The big saver is the daily timelapse.avi for the web that is created daily from all of the hourly captured photos. It is many megs in size gets written daily and it doesn’t matter if I lose it. It can be recreated from the photos at will. So it is the perfect kind of file to throw on a RAM file system.

I also store the hourly and daily logs that I create using the cron driven logcreate script that I run. The logcreate script creates an hourly log that is concatenated into a daily log on the tmpfs then every day the daily log will concatenate into a full log, that is rotated, on the SD card, so I have a permanent record. Need to put the link for this here!

What is a tmpfs?

It is a RAM Disk, a.k.a. RAM Drive that allows RAM to be used as a hard drive. Obviously when the power goes out, it goes away. So we don’t want anything important to go there. But for things like files that I make such as the hourly photos that my Web Cam takes and the video it makes daily and logs, it is perfectly fine for usage. It is not a really big deal if the power went out and I lost this information as it will be recreated shortly anyways.

Caution

The only issue that I see with having logs on a tmpfs would be a situation where the Pi got in a state of weirdness where it started rebooting itself and then you had no logs to track down the problem. Then I suppose, it would be just a matter of changing the /etc/fstab file to revert to putting the logs back onto the SD card for a while to track down the problem. But, for a Raspberry Pi like mine that is running stable and I am not doing many experiments with right now, having the logs in volatile memory is not something I worry about. Plus it is easy to make a script to backup the logs to the SD card or another computer, if you manually reboot it, so you can save them if you like when you have control of the reboots.

Write Saving #2: Turning off swap

If the Raspberry Pi runs out of RAM, not likely if it is a server set up for light duty usage, it will start to use swap which is on the SD card, causing writes to the swap file. Mine rarely touches swap. I would rather tune the thing for better memory use than have it use swap.

It is possible to turn off swap usage using the command…

sudo dphys-swapfile swapoff

This is not persistant and needs to be done on every boot. It could be put into the root crontab by editting it using sudo crontab -e and adding the line. Or creating a script for it along with other items that are to be run at startup.

@boot dphys-swapfile swapoff

Online, people said that there was another way to turn it off by reducing the swap file size to zero, a config file for swap, can’t remember the name. But it is claimed that when it reboots it just overrides that and makes a default 100M swap file.

Write Saving #3: Moving /var/log to a tmpfs

One of the biggest offenders as far as writing to files periodically is the logs that live under /var/log and it’s sub-directories. You can create an entry in /etc/fstab that will create a tmpfs for /var/log. The only caution here is daemons, like Apache that require a directory to exist under /var/log or else they will not start. Apt also has a directory under /var/log, but it creates itself when apt runs for the first time so that is no problem. The apt directory has logs that keep track of what apt installs or uninstalls, good info to know about. News seems to work fine creating a directory for itself too. So for me only Apache is a problem.

  1.  Put an entry in /etc/fstab…
     tmpfs /var/log tmpfs defaults,noatime,mode=0755 0 0
  2.  Found out that news and apt folders create themselves when these things run.
  3. Apache is the one thing that does not like a missing folder so made a Kludge for now using ~/bin/setup-tmp.sh where I create /var/log/apache2 and chmod it 750. Then I restart apache using apachehup.sh, which just restarts it. Apache was failing to load when I pointed the log dir to /tmp in /etc/apache2/envvars under the export APACHE_LOG_DIR directive.

Write Saving #4: noatime

As you can see above one of the options used in the /etc/fstab file is the noatime option. By default the Raspberry Pi uses this option for the mount of the SD card. If you add mount points of your own to the card, make sure noatime is used. Without it Linux makes a small write each time a file is read to keep track of when it was last accessed, this obviously causes writes. It is possible to use it for the writes to the tmpfs as I am doing above. It saves a bit of time as the system does not have to do a write when a file is just being read.

Another good use of noatime is for drives connected across the network. For example on NFS mounts noatime is a really good choice. The network is generally slower than devices attached to a PC and having to send a write across every time a file is read, slows things down a bit when moving many files.

 


Been running this setup with the RAM savings for a few months now with no problems. I hardly ever see the ACT light blinking on the Pi anymore.

 

The LEDs have the following meanings :

  • ACT – D5 (Green) – SD Card Access
  • PWR – D6 (Red) – 3.3 V Power is present
  • FDX – D7 (Green) – Full Duplex (LAN) connected
  • LNK – D8(Green) – Link/Activity (LAN)
  • 100 – D9(Yellow) – 100Mbit (LAN) connected

From http://www.raspberrypi-spy.co.uk/2013/02/raspberry-pi-status-leds-explained/

FTP on Raspberry Pi. An easy way to make shared folders

The idea with FTP is to have folders that can be reachable between Linux and Windows, locally and remotely and easily. FTP is not secure, but it can be made secure, that info can be found on the web. For now I am covering the basics of FTP here.

For most things that I need to do, I don’t need the files to be secure anyways, 90% of the time nothing critical is going back and forth across remotely. If it is I would use a secure method of sending files via SSH via SFTP or an SSHFS.

FTP is an old protocol but it just plain works and is compatible with Windows, Linux and Mac. I have tried WebDAV in the past but it is compatible to only a degree with various Windows operating systems. I have had a hard time getting it working correctly on versions of Windows beyond XP, resorting in installing patches to Windows and etc. Generally not easy to implement.

I was also looking at FTP as a native tool typical of server installs. I have experimented with cloud setups such as OwnCloud and Sparkleshare, but with FTP I was looking for something simple and quick to setup, no special software, no mySQL databases running on the Raspberry Pi, no special software on client PCs, that sort of thing.

vsFTP

sudo apt-get install vsftpd

Edit the configuration file

Back it up first then do an edit.

sudo cp /etc/vsftpd.conf /etc/vsftpd.orig
sudo nano /etc/vsftpd.conf

uncomment local_enable = YES

uncomment write_enable = YES

Find this and check that it is set this way…

local_umask=022

Enabling PASV

I have read online that enabling the PASV capability for FTP is a good idea. Frequently when I have FTP’d to various ISP’s sites I have seen them operate in PASV mode. So it stands to reason that if the pro’s are have it set up that way it may have it’s advantages.

Add the following lines to the /etc/vsftp.conf file.

pasv_enable= Yes
pasv_min_port=40000
pasv_max_port=40100

There is nothing magic about the numbers of the port range other than they should be unused by anything else that your setup might require and generally I have seen high numbers used commonly. To work out side of your local network you must enable port forwarding of the range of port numbers through your router configuration.

Changes to vsFTP

With the newer versions of vsFTP there is a change that has occurred since I wrote my previous post about vsFTP (  http://oils-of-life.com/blog/linux/server/additional-utilities-for-a-linux-server/ )

The change has to do with the fact that the root directory of the user has to be non-writable and I have read online that it is best to make it owned by root as well. This is covered below, after the section on adding a user. You need to have a user first before modifying their permissions!

FTP User

To create an FTP user, create it in a way that it does not have a login shell. So that someone who can log in to the FTP account can’t execute shell commands. The line /sbin/nologin may not be in the /etc/shell file and in that case it needs to be added in there. The user basically has to be jailed in their directory and has to have no login shell.

sudo useradd -m -s /sbin/nologin -d /home/user user

I added Documents, public_html directories to the /home/user as well. Then made the users root folder /home/user, owned by root and nonwritable.

cd /home/user
chown user:user Documents
chown user:user public_html

chown root:root /home/user
Make Root of user non writable
sudo chmod a-w /home/user



FTPing on the PC

Now that ftp is set up on the server you will want to be able to connect to it!

Options for connecting…

Command Line, WIndows and Linux

ftp yoursite.com

That gets you into FTP via the command line. The command prompt will now start with ftp> ,that is how you know that you are within the ftp command shell.

It is archaic, but worth knowing when you have to stick a file up or pull it down right at the command line. The commands the ftp prompt accepts are basic, but good enough to get most work done. Type help at the prompt to get a list of commands.

Via Folders

Linux

Just enter the location of the ftp server right into the top of the directory folder and you will be prompted for a password and taken there.

Windows
Windows7/Vista:
  1. Open Computer by clicking the “Start” button, and then clicking Computer.
  2. Right-click anywhere in the folder, and then click Add a Network Location.
  3. In the wizard, select Choose a custom network location, and then click Next.
  4. To use a name and password, clear the Log on anonymously check box.

From: https://www.google.com/search?q=connect+to+ftp+windows+7&ie=utf-8&oe=utf-8

 

 

Automatic Server Status Page Creation Update

In January 2015 I created a post about automatically creating a status page for a Linux server that I have. Typically this is put under a restricted directory and allows you to see a snapshot of what is happening with the server. I run it by putting the scripts in the /etc/cron.hourly directory on a Linux PC and a Raspberry Pi running Linux.

It serves as a simple way to check up on the server without having to use a tool such as Webmin that requires a login. It also keeps a trail of log files that get rotated on a monthly basis, so there is always a few old ones around to track down any problems and patterns in the operation.

I have found this information useful when I have traced down malfunctions that can occur when setting up a server and also when I was trying to get a webcam up and running and had the USB bus hang up a few times when the cam was overloaded with too much light.

In the new script file I fixed a bug by adding parenthesis around a line that I was trying to echo and I added code to run the w command to show a quick picture on who is logged in, how long the server has been up and running and the values for the average load on the server at the 1, 5 and 15 minute marks.

Logcreate Script

#!/bin/dash
# Remove old log
rm /var/www/status/log.txt
# Print logged outputs into new log.txt
date >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
tail /var/log/syslog >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
free >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
df -h >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> /var/www/status/log.txt
ps aux | sort -nrk 4 | head >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Logged in User info using w command
w >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
echo >> /var/www/status/log.txt
# Copy log.txt into the full log that is collected
cat /var/www/status/log.txt >> /var/www/status/fulllog.txt
# Create a free standind copy of the process tree
pstree > /var/www/status/pstree.txt

Alternate Version

I also created a version of the script for a desktop Linux PC that does not have Apache installed.  In it I use a DIR variable to contain the directory that I want the log.txt file stored.

 #!/bin/dash

# User defined variables
# No trailing / on DIR!
DIR=/home/erick/status

# Remove old log
rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
#ps aux | sort -nk +4 | tail >> log.txt
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt
ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/fulllog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

Rotation of Log

In the /etc/cron.monthly directory I have created a file that is called status-log-rotate and it will save backup copies of 2 months worth of the full concatenated server status logs.

#! /bin/bash
DIR=/home/erick/status
mv $DIR/fulllog.txt.1 $DIR/fulllog.txt.2
mv $DIR/fulllog.txt $DIR/fulllog.txt.1

Tweaks for Raspberry Pi

For the Raspberry Pi which has an SD card that I am trying to be conscious of writing to often. I have recently made some modifications to put the /tmp folder onto RAM using tmpfs. I create the hourly log underneath a folder there. Daily via a script it cron.hourly it gets concatenated into a daily log which is under a status folder that has restricted access. This gets appended once per day to the fulllog which actually lives on the SD card. The end result, no multiple hourly writes to the log file, just one append to the full log per day. The only downside is if the power drops and then some log entries will be lost for the day.

Logcreate runs from /etc/cron.hourly for Raspberry Pi

#!/bin/dash
# Set DIR, on Pi this is a temp location for log
DIR=/tmp/web

# Set fixed DIR FIXDIR for files that have to be stored on SD card
# Nevermind, just make a daily log and then copy that to the full log daily.
#FIXDIR=/var/www/status

# Remove old log

rm $DIR/log.txt
# Print logged outputs into new log.txt
date >> $DIR/log.txt
echo >> $DIR/log.txt
tail /var/log/syslog >> $DIR/log.txt
echo >> $DIR/log.txt
free >> $DIR/log.txt
echo >> $DIR/log.txt
df -h >> $DIR/log.txt
echo >> $DIR/log.txt
# Top memory using processes http://www.commandlinefu.com/commands/view/3/display-the-top-ten-running-processes-sorted-by-memory-usage
echo "USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND" >> $DIR/log.txt

ps aux | sort -nrk 4 | head >> $DIR/log.txt
echo >> $DIR/log.txt
# Logged in User info using w command
w >> $DIR/log.txt
echo >> $DIR/log.txt
echo >> $DIR/log.txt
# Copy log.txt into the full log that is collected
cat $DIR/log.txt >> $DIR/dailylog.txt
# Create a free standing copy of the process tree
pstree > $DIR/pstree.txt

dailylog-to-fulllog script, runs from /etc/cron.daily

#! /bin/bash

DIR=/tmp/web
FIXDIR=/var/www/status

echo "----------------------------------------------" >> $DIR/dailylog.txt
date >> $DIR/dailylog.txt
echo "----------------------------------------------" >> $DIR/dailylog.txt
cat $DIR/dailylog.txt >> $FIXDIR/fulllog.txt
rm $DIR/dailylog.txt

Logcreate Output from Raspberry Pi

Below is what the logcreate script will output to the log.txt file on a Raspberry Pi that I have running as a web server.

Sun Jul 12 14:17:01 EDT 2015

Jul 12 13:47:51 raspberrypi dhclient: DHCPACK from 192.168.1.1
Jul 12 13:47:52 raspberrypi dhclient: bound to 192.168.1.17 -- renewal in 40673 seconds.
Jul 12 13:59:01 raspberrypi /USR/SBIN/CRON[28010]: (erick) CMD (aplay /opt/sonic-pi/etc/samples/guit_e_fifths.wav)
Jul 12 13:59:07 raspberrypi /USR/SBIN/CRON[28009]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:00:01 raspberrypi /USR/SBIN/CRON[28013]: (erick) CMD (/home/erick/fswebcam/cron-timelapse.sh >> timelapse.log)
Jul 12 14:00:23 raspberrypi /USR/SBIN/CRON[28012]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:01:01 raspberrypi /USR/SBIN/CRON[28022]: (root) CMD (/home/erick/bin/usbreset /dev/bus/usb/001/004)
Jul 12 14:01:02 raspberrypi /USR/SBIN/CRON[28021]: (CRON) info (No MTA installed, discarding output)
Jul 12 14:09:01 raspberrypi /USR/SBIN/CRON[28053]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Jul 12 14:17:01 raspberrypi /USR/SBIN/CRON[28064]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

             total       used       free     shared    buffers     cached
Mem:        445804     424488      21316          0     106768     260516
-/+ buffers/cache:      57204     388600
Swap:       102396          0     102396

Filesystem      Size  Used Avail Use% Mounted on
rootfs          6.3G  3.1G  3.0G  51% /
/dev/root       6.3G  3.1G  3.0G  51% /
devtmpfs        214M     0  214M   0% /dev
tmpfs            44M  240K   44M   1% /run
tmpfs           5.0M  8.0K  5.0M   1% /run/lock
tmpfs            88M     0   88M   0% /run/shm
/dev/mmcblk0p5   60M   19M   41M  32% /boot

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root      2071  0.0  3.0  24896 13652 ?        Ss   Jun28   2:24 /usr/sbin/apache2 -k start
www-data 27745  0.0  1.5  25412  7084 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27744  0.0  1.5  24960  6760 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27743  0.0  1.5  25428  7116 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27742  0.0  1.5  25396  7036 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27538  0.0  1.5  25396  7032 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27502  0.0  1.5  25404  7036 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27501  0.0  1.5  25396  7044 ?        S    06:25   0:00 /usr/sbin/apache2 -k start
www-data 27747  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start
www-data 27746  0.0  1.3  24936  6188 ?        S    09:58   0:00 /usr/sbin/apache2 -k start

 14:17:02 up 14 days, 12:56,  1 user,  load average: 0.00, 0.01, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
erick    pts/0    192.168.1.5      14:04   10:39   1.70s  1.70s -bash