All posts by erickclasen

About erickclasen

This is my Blog for writing about technical items and my other thoughts. I also use it to experiment with WordPress.

Most Starred Python Projects on Github

Info Links, , , ,

I have been working through the Python Crash Course book by Eric Matthes. I got to Chapter 17, Working With API’s  and ran the example code python_repos_updated.py that generates an SVG graph of the most starred Github Python projects. The thing that stands out to me is the amount of machine learning related resources that show up on this hot list. The interactive graph is generated by using Pygal which works well and is easy to use.

Interactive SVG Graph of Most Starred Github Python Projects

Out of curiosity I looked at the most starred C projects on Github as well with a simple modification of the code.

Interactive SVG Graph of the Most Starred Github C Projects

Python Crash Course

The book has been a good resource to develop programming skills in Python, my background is in C primarily and this book was an excellent starting point to officially learn Python. What I mean by that, is that I have been using Python by modding other peoples code when I need some piece of code to use for some application. For a few years I was learning it that way on an as needed basis (plus using the Internet to look up things that I might get suck on) which works to a point. When I started to get into machine learning a bit deeper, I could see that having a more formal introduction to Python would be helpful. Having a book that goes through the process formally helped to quickly build skills in my opinion. In a matter of a few months I have gone through a decent amount of the book and have been able to branch off some of the examples and create free standing code that was interesting to learn from.

Cloning Linux Mint Setups

Linux, My Thoughts, Scripts, , , , , , , , , , ,

Recently I swapped in an SSD to be the new primary drive on my media center PC which was running Linux Mint 18.0 on the spinning SATA drive.

This post is basically a brief documentation of the basic steps involved in cloning or upgrading/cloning Linux Mint. Most likely this works fine for Ubuntu as well as Debian as they share a common ancestry. There are most likely limits to this scheme. I imagine things would break badly trying to do this across 17.3 and 18 for example. The base on those is a different version of Ubuntu, 14.04 vs 16.04. I might try to do a clone when the next whole number version of Mint comes along. Just pop in a drive that I don’t care about, or do it on a VM, such as Virtualbox for an experiment.

Plans

The plan is to relieve some of the storage duties of the spinning drive which was filling up. Plus a speed increase as the SSD can move data 4x faster than the spinning drive but more importantly with no moving parts the access time is minute in comparison the the spinning drive. Applications open much faster, boot time is cut by 75%, etc. If the machine needs to use swap it won’t grind down to a halt as well with a fast disk. This machine is a bit older, SATA II, but a Solid State Drive (SSD) still makes a big difference.

The idea is to clone over the home folder but exclude large data such as  the ~/Music folder and leave that on the old drive and mount the drive as additional storage and use a symlink to it.

Old Setup 160GB Spinning Drive
Old Setup: 160GB Spinning Drive
New Setup: 80GB Primary SSD
New Setup: 80GB Primary SSD

Goal

The goal of this post’s example is to  do an upgrade to Linux Mint 18.3 from 18, clone over my user settings and reinstall all programs. Over the past year and a half that the machine has been in use there have been quite a few programs that have been installed on this machine. Many of them run from the command line or are libraries related to some of the machine learning code that gets run in the background on the machine. Needless to say it would be very hard to remember them and a lot of little things would be broken.

Step 1: Install Linux Mint from USB Stick or DVD

This step is pretty basic and is covered elsewhere on the web…

Linux Mint,  Ubuntu , Debian

But needless to say you want to create a user that has the same name and User ID (UID) and Group ID (GID) as on the drive that you will be cloning from.

Step 2: Login on the new machine/drive setup kill your home directory and rsync the old one over

Mount the old drive, doing this from the GUI folder is a fine way to do it. Make note of where it mounts. You can always execute df from the command line to find where it mounted as well

It sounds crazy but, it does work. Login, open a terminal and execute…

rm -rf /home/yourusername

Everything is in memory that the OS needs right now to get through the next step, so nothing will crash and this will give you a blank slate to work with.

Next rsync over your home folder from the old drive ( /dev/sda in my case) making sure that you use the archive option. Using the v and h options as well is helpful as well to produce a lot of output in case you have to trace through a problem.

-v : verbose
-a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
# -h : human-readable, output numbers in a human-readable format

Example:

For me it went something like this…

rsync -avh /media/erick/B0B807B9-E4FC-499E-81AD-CDD246817F16/home/erick /home/

Next log out and then back in. Almost like magic everything should look familiar. The wallpaper on the desktop should look like whatever it was on the old setup, fonts and other desktop sizing customizations should be there. Open the browser and it should be like you left it in the old setup. It is almost like you hibernated the old setup and woke it up after teleporting it’s soul into the new drive.

But, wait the software needs attention

Step 3: Bring over the software too, sort of…

More like apt-get install it over is closer to the truth. I tried following a post on how to do this (https://askubuntu.com/questions/25633/how-to-migrate-user-settings-and-data-to-new-machine) but, what was described in it did not lead to success. The suggestion was the following…

oldmachine$ sudo dpkg --get-selections > installedsoftware
newmachine$ sudo dpkg --set-selections < installedsoftware
newmachine$ sudo apt-get --show-upgraded dselect-upgrade

It didn’t work but, it at least did the collection part. So I wound up using the first part…

oldmachine$ sudo dpkg --get-selections > installedsoftware

…and then brute forced an install by doing some grep,rev,cut,rev again on the input file. Which basically flips every line in the file and removes the word “install” which is now at the beginning and backwards then flips it back over line by line.

The next line with the awk command prepends sudo apt-get install to the front of each line and saves the output to reinstall-software.sh

 installedsoftware-to-apt-get-install.sh
 #!/bin/bash
 cat installedsoftware | grep "install" | rev | cut -c 12- | rev > cleaned-installed-software
 awk '{printf "sudo apt-get install "$0"\n"}' cleaned-installed-software > reinstall-software.sh

Run the reinstall-software.sh script and it will do just what it says, install all of the software that was on the old setup. I believe there is an option for apt-get to preanswer Yes when it comes up and asks you the yes or no question about installing. I left it off so that I could review all the larger sized software being loaded. A few times I hit no by accident so had to re-run the script, no big deal.

Reboot is best now to avoid side-effects

Before going much further a reboot is probably in order as so much has changed on the machine.

For me, during the software install process, I was presented with a question about picking LightDM or another X- windows manager. I picked LightDM because I think that is what I had been using. After I was all done, I put the machine in suspend and it had a bit of trouble coming out of it, having a temporary error related to the X-windows manager. A blue screen came up and had a message about removing a temporary file. Just rebooting the machine cleared this up as the /tmp directory is flushed. Apparently this was something that was set before the upgrade, clone and software install process and did not get unset. Other than that I have seen no side effects from the process of upgrade, clone, software install.

Other Items

If you had files configured outside of the home directory such as /etc/hosts, you will obviously have to copy that over. Also, if you have any /etc/cron.hourly,weekly,monthlies that you put on the old machine. Also, it pays to do a dump of crontab’s using crontab -l > crontab-dump.txt on the old setup so they can be reconfigured to the same settings.

Cloning old to new box

This entire process can be used to clone one computer setup to another, old box to new one for example. Which brings us to…

Final Thoughts: Twin Machines

It is entirely possible to keep two machines in sync using the methods outlined above. I have not tried this but I am tempted to test it out at least. What I am thinking of is a laptop and desktop for instance. The desktop with it’s ability to hold multiple drives with ease works nice here It has one drive with the same OS as the “twin” laptop and is setup as multi OS boot. The steps above are executed, cloning the laptop setup and data to the desktop. It is entirely possible to keep cloning the home folder contents back and forth between the two to keep them sync’d. Even the software can be kept in sync using the method used above to re-install it.

It is possible to do this directly between them, both on at the same time. Or, through a server where they both get backed up to. The only caveat is overwriting and deletions. Such as care when using the –delete option with rsync. There is a potential for a race condition of sorts if settings and files get changed and then clobbered by a sync operation. If I were to try this I would start with a one direction sync. One device is the master and the other the slave. Deletions and settings changes get cloned from master to slave automatically only.

Raspberry Pi WiFi via USB device

Linux, Raspberry Pi, Raspberry Pi Server, Unpolished Gems, , , , , , , , , , , ,

Setting WiFi up via the command line on Raspberry Pi, using a USB Wireless Adapter

These are notes on how to setup WiFi on the Raspberry Pi. The R-Pi is a model 2 running Raspbian 4.1.19+.

In my case and in the example that follows, the Raspberry Pi is connected to an Ethernet network, static IP at 192.168.1.17. This is the address that  I am logging in via SSH to get to the command line to configure the USB WiFi. Adapter.

USB WiFi Adapters

Two USB Wifi’s were tried a Belkin N300 and an Edimax EW7811Un. Both use the Realtek RTL8192CU chip and work well with the R-pi. Initial testing was with the Belkin and the output from this device is used in this post for the command line examples.

Edimax EW7811Un 150Mbps 11n WiFi USB Adapter, it is nano size and has a blue activity light. It works well, can’t imagine how small the antenna is in there and how they get RF to work out OK with these sub wavelength antennas!

NOTE: Originally the adapters were tried by plugging them into a powered USB hub which plugged into the R-pi. This allows for hot-plugging. If a device is hot-plugged directly into the R-Pi it will force a reboot, at least on the one that I have (R-Pi Model 2B). This is probably due to an inrush current that pulls down the power bus momentarily, I am guessing. The powered USB hub isolates the R-Pi from the devices connected to it as far as power is concerned and things will hot plug fine using it. I did realize later on that when I plugged the USB WiFi adapter directly into the R-Pi, I got more stable behavior with it. As in less strange dropouts of the WiFi from the network. It maintains a network connection better for me at least, plugged in directly.

Initial Testing

The first steps involve checking to see that the adapter is detected, registered and the correct device driver is loaded. They are just confirmation that all is well. They can be skipped and then ran later if problems arise and troubleshooting is needed.

Plug in USB WiFi adapter and run a lsusb and dmesg…

lsusb
erick@raspberrypi ~ $ lsusb
Bus 001 Device 002: ID 0424:9512 Standard Microsystems Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. 
Bus 001 Device 004: ID 2101:8500 ActionStar 
Bus 001 Device 005: ID 2101:8501 ActionStar 
Bus 001 Device 006: ID 154b:005b PNY 
Bus 001 Device 008: ID 050d:2103 Belkin Components F7D2102 802.11n N300 Micro Wireless Adapter v3000 [Realtek RTL8192CU]
Bus 001 Device 007: ID 174c:1153 ASMedia Technology Inc.

The device shows up fine using lsusb. Now on to dmesg to see if the correct driver loaded…

[156238.694964] usb 1-1.3.3: new high-speed USB device number 8 using dwc_otg
[156238.797146] usb 1-1.3.3: New USB device found, idVendor=050d, idProduct=2103
[156238.797188] usb 1-1.3.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[156238.797207] usb 1-1.3.3: Product: Belkin Wireless Adapter
[156238.797225] usb 1-1.3.3: Manufacturer: Realtek
[156238.797242] usb 1-1.3.3: SerialNumber: 00e04c000001
[156239.201673] usbcore: registered new interface driver rtl8192cu

Kernel driver rtl8192cu is loaded, all should be well with the adapter!

lsmod

The following lsmod was run and it confirms the kernel module is loaded for the 8192cu driver. It is just added confirmation that all is well.

 

erick@raspberrypi ~ $ lsmod
 Module                  Size  Used by
 xt_state                1434  1
 ipt_MASQUERADE          1220  1
 nf_nat_masquerade_ipv4     2814  1 ipt_MASQUERADE
 iptable_nat             2120  1
 nf_nat_ipv4             6162  1 iptable_nat
 nf_nat                 17132  2 nf_nat_ipv4,nf_nat_masquerade_ipv4
 8192cu                556175  0
 nfsd                  263815  11
 nf_conntrack_ipv4      14388  3
 nf_defrag_ipv4          1766  1 nf_conntrack_ipv4
 xt_conntrack            3420  1
 nf_conntrack           95316  6 nf_nat,xt_state,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_ipv4
 iptable_filter          1698  1
 ip_tables              12362  2 iptable_filter,iptable_nat
 x_tables               18590  5 ip_tables,ipt_MASQUERADE,xt_state,xt_conntrack,iptable_filter
 i2c_dev                 6386  4
 snd_bcm2835            22502  0
 snd_pcm                92861  1 snd_bcm2835
 snd_seq                58152  0
 snd_seq_device          5142  1 snd_seq
 snd_timer              22156  2 snd_pcm,snd_seq
 snd                    67534  5 snd_bcm2835,snd_timer,snd_pcm,snd_seq,snd_seq_device
 sg                     20575  0
 i2c_bcm2708             5988  0
 bcm2835_gpiomem         3703  0
 bcm2835_rng             2207  0
 uio_pdrv_genirq         3526  0
 uio                    10078  1 uio_pdrv_genirq

 

Setup the WiFi Connection

In this example this is a WPA type of security. I know the SSID and password and just put them into the wpa_supplicant configuration file. If you need to see what WiFi nodes are available on the network consider using the scan …

( IF Needed )

sudo iwlist wlan0 scan

Check /etc/network/interfaces

Check to see that the section exists in the file that will allow the USB apapter to hot plug and also the wpa-roam line that points to the wpa_supplicant.conf file where the SSID and password will be stored in the next step.

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

…any changes to the interfaces file will require a reboot or a restart of networking to take effect, via….

 sudo service networking restart

If you are running the R-Pi headless it will disconnect from SSH and will require a re-login. If a mistake is made in the interfaces file, it might not come back and require connecting a keyboard and monitor to reconnect. The good news about having both a running eth0 and wlan0 is that if you make a mistake in only one of them it will be possible to connect via the other one. Less chances of being totally locked out by a small typo in the interfaces file. Sometimes a restart of network services will cause a non-recoverable dropout which requires a reboot to get an SSH connection going again.

Config wpa_supplicant
sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

Go to the bottom of the file and add the following type of entry, putting in the correct SSID and pwd:

network={
 ssid="SprintHotspot2.4-example"
 psk="thepassword"
 }

Save and Exit

Then execute the follow to apply the new configuration….
wpa_cli -i wlan0 reconfigure

TEST IT:

ifconfig wlan0

 

Results show that the interface is up and and running

erick@raspberrypi ~/Music/music-test $ ifconfig wlan0
wlan0     Link encap:Ethernet  HWaddr 74:df:e0:e0:0e:e0  
          inet addr:192.168.128.46  Bcast:192.168.128.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4023854 errors:0 dropped:1664207 overruns:0 frame:0
          TX packets:2955774 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:3003299485 (2.7 GiB)  TX bytes:1956045437 (1.8 GiB)

It should be working at this stage. Trying to reach the net using something like ping google.com should give good results. If not more troubleshooting is required. I had to do the next step to get it to reach the net on the wireless network as it was trying to use the Ethernet connection to the router to get out, set as the default gateway, which was not hooked up to the web at all. Just a router at 192.168.1.1 and no WAN port connection.

Additional Step

The following bit may or may not apply for everyone. But, I am adding here as it was not obvious at the moment I got WiFi up. I had to think on it a bit! Basically a default route has to exist that takes it to a gateway to the Internet.

IN ORDER TO GET PI OUT ON INTERNET NEEDED TO DO A

sudo route add default gw 192.168.128.1 wlan0

As there was no route out and it must pick eth0 by priority! Needs a route to the Sprint Box that I am connected to the net on the 192.168.128.0 network.

THE ABOVE WOULD HAVE TO HAPPEN ON EVERY REBOOT OR NETWORK REFRESH! Or just get rid of default gateway on ETH0 and it might just pick up the gateway on the wlan0 all of the time! If both are dhcp the eth0 gateway will always  be treated as preferred, so going static will get rid of it as I was planning on using this R-pi as a bridge from LAN to WLAN! This required editing /etc/network/interfaces to remove the 192.168.1.1 router as a gateway. The router was reconfigured by turning off DHCP on it, effectively making it an access point for WiFi. Essentially it becomes another path to the Internet along with the WiFi hotspot.

erick@raspberrypi ~ $ route -n
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
 192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
 192.168.128.0   0.0.0.0         255.255.255.0   U     0      0        0 wlan0
 erick@raspberrypi ~ $ sudo route add default gw 192.168.128.1 wlan0
 erick@raspberrypi ~ $ route -n
 Kernel IP routing table
 Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
 0.0.0.0         192.168.128.1   0.0.0.0         UG    0      0        0 wlan0
 0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
 192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
 192.168.128.0   0.0.0.0         255.255.255.0   U     0      0        0 wlan0
SHOWS UP IN /etc/resolv.conf as well….
erick@raspberrypi ~ $ cat /etc/resolv.conf
 domain router
 search router
 nameserver 192.168.128.1

 

ip show route as well…
erick@raspberrypi ~ $ ip route show
 default via 192.168.128.1 dev wlan0
 default via 192.168.1.1 dev eth0
 192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.17
 192.168.128.0/24 dev wlan0  proto kernel  scope link  src 192.168.128.46

 /etc/network/interfaces

The following /etc/network/interfaces file was edited to make the wlan0 connection which connected to the Internet work for the R-Pi.

Note the eth0 connection is setup static on the 192.168.1.0 wired network. The wlan0 connection is set for dhcp on the 192.168.128.0. The default gateway at 192.168.1.1 ( the router ) is commented out to allow the default to fall on the 192.168.128.1 WiFi router, which is a ZTE WiFi hotspot, basically a repeater from 4G LTE cell to WiFi.

Note the wpa-roam points to the wpa_supplicant file that has the SSID and password entered earlier in this post to get the WiFi going.

erick@raspberrypi ~/Music/music-test $ sudo cat /etc/network/interfaces
[sudo] password for erick: 
auto lo

iface lo inet loopback
#iface eth0 inet dhcp

iface eth0 inet static
address 192.168.1.17
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
# Remove gateway to see if it fails-over to wlan0 gateway on 192.168.128.1 12242017
# gateway 192.168.1.1
# nameservers
dns-nameservers 8.8.8.8 8.8.4.4

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

ALL UP AND RUNNING OK

Running ip a shows all interfaces. Note that the R-pi is connected to wlan0 on the 192.168.128.46 address and eth0 is connected at 192.168.1.17, both networks are now available.

erick@raspberrypi ~ $ ip a
 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
 valid_lft forever preferred_lft forever
 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
 link/ether b8:2a:eb:2a:a4:2a brd ff:ff:ff:ff:ff:ff
 inet 192.168.1.17/24 brd 192.168.1.255 scope global eth0
 valid_lft forever preferred_lft forever
 3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
 link/ether 08:8a:3c:ba:83:8a
 brd ff:ff:ff:ff:ff:ff
 inet 192.168.128.46/24 brd 192.168.128.255 scope global wlan0
 valid_lft forever preferred_lft forever

The connection to the WiFi node can be confirmed via iwconfig wlan0…

erick@raspberrypi ~ $ iwconfig wlan0    

 IEEE 802.11bg  ESSID:"SprintHotspot2.4-example"  Nickname:"<WIFI@REALTEK>"
 Mode:Managed  Frequency:2.452 GHz  Access Point: 34:3A:87:3A:BA:3A
 Bit Rate:54 Mb/s   Sensitivity:0/0
 Retry:off   RTS thr:off   Fragment thr:off
 Power Management:off
 Link Quality=100/100  Signal level=63/100  Noise level=0/100
 Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
 Tx excessive retries:0  Invalid misc:0   Missed beacon:0

lo        no wireless extensions.

eth0      no wireless extensions.

 

Create a hidden WordPress page using bash on the command line

Linux, Server, WordPress, , , , , ,

Recently I was searching around looking for a way to create a hidden page on a WordPress site. It is a hosted site, not on wordpress.com. It is on a Linux server to which I have shell access.

Initially I tried using a plugin that I found that hides pages and posts. Plugins, you got to love or hate them. Love then when they work great right out of the box, hate them when they take a long time to troubleshoot.

Rather than waste too much time with the plugin, I went straight to the command line.

Screenshot_2018-04-03_18-28-35-shows-making-hidden-page

It turns out that if you publish a page and then log into the hosting server, make a directory somewhere under your public_html, change directory into it and execute…

 wget -x -nH your-page-url-to-hide-here

 

Screenshot_2018-04-03_18-38-33-draft-or-private-wp-setting
Set to Draft or Private

…then go back it and make the page a draft or under review, so it “disappears” from the menu structure. It will still work as a “cached” HTML page that has been downloaded to the folder that you have created. It will work, pictures and what not that you have loaded in it will be fully functional.

Example of a hidden page

http://erick.heart-centered-living.org/hidden/i-am-a-hidden-page/

Once the original page is put into draft/under review or private mode, it is gone…

http://erick.heart-centered-living.org/i-am-a-hidden-page/

Caveat

I have noticed that caching can get in the way. If your server caches pages, wget may not see the page updated when you make changes. A quick remedy is to set the page to draft/pending review or private, delete the hidden page. I usually use rm -rf from the directory above it and then force it to download the “404” page. Then  you can publish the page re-run wget and it will force it to get the fresh version. Keep note of the size of the file as a hint that it is getting the right one.

Upcoming: Do this with a CGI Script

In an upcoming post, I will cover how to make a CGI script that will allow you to create a hidden page easily without having to use SSH to login to the server.

 

wget options used in this example, from the man page

-x
–force-directories
The opposite of -nd—create a hierarchy of directories, even if
one would not have been created otherwise.  E.g. wget -x
http://fly.srk.fer.hr/robots.txt will save the downloaded file to
fly.srk.fer.hr/robots.txt.

-nH
–no-host-directories
Disable generation of host-prefixed directories.  By default,
invoking Wget with -r http://fly.srk.fer.hr/ will create a
structure of directories beginning with fly.srk.fer.hr/.  This
option disables such behavior.

Wget Resources

https://www.lifewire.com/uses-of-command-wget-2201085

https://www.labnol.org/software/wget-command-examples/28750/

The Ultimate Wget Download Guide With 15 Awesome Examples

http://www.linuxjournal.com/content/downloading-entire-web-site-wget

Script to check and re-connect WiFi on Raspberry Pi

Info Links, Linux, Raspberry Pi, Raspberry Pi Server, , , , ,

After having occasional dropouts of the Raspberry Pi Wifi ( after I started using it as a bridge) and having to ifup the wlan0 manually, I considered making a script to automate it. But, a little searching online found this elegant solution to the problem.

mharizanov/WiFiCheck

https://gist.github.com/mharizanov/5325450

Install Slackware on a VM

Info Links, Linux, , , , , , , , , , , , ,

Easy to follow tutorial on installing Slackware Linux onto a Virtual Machine

I have been interested in trying out Slackware for some time now. The Slackware Linux Essentials ( aka Slackbook) is an excellent review of Slackware and Linux in general. I went through it one winter a few years ago and was impressed as it was a great refresher course on Linux. After a while I tend to forget some of the tricks on the command line that I do not use on a regular basis. Going over a manual like this is a good brush up. Reading the book convinced me that I would have to try out Slackware someday.

Tutorial

I had no trouble following the tutorial and getting Slackware up and running on a VirtualBox VM. The current version 14.2 (February 2018) is similar enough to the 13.0 install in the guide that the few differences are not a problem. The one difference that I noticed is that when the disk is partitioned the option for bootable did not appear for me as it did in the tutorial. I just went ahead and wrote the disk and it was fine. The tool might have some logic built in to decide what to do and does not required you to tell it that it has to be set as bootable anymore.

http://archive.bnetweb.org/index.php?topic=7370.0

Slackware DVD ISO Torrent Page

http://www.slackware.com/getslack/torrents.php

Slackware Live DVD/USB Stick

Live DVD/USB Stick installs are relatively new for Slackware. In case you want to just go ahead and try it on a Live CD or USB stick it is now available as a download.
http://bear.alienbase.nl/mirrors/slackware/slackware-live/

Linux not booting, incorrect drive mapping, try hacking grub.cfg

Linux, , , ,

Hacking /boot/grub/grub.cfg is not usually the first thing that comes to mind. But, I ran into a problem when installing Kali Linux on a plug in USB Hard Drive. I wanted to try to put it on a USB hard drive by using the USB boot stick. All went well with the install. On reboot, guess what, the USB stick and it’s multiple partitions was no longer “parked” on /dev/sdc-sdg and when Kali had installed it thought it was going to be at /dev/sdh. Pulling the USB stick put the USB Hard Drive at /dev/sdc on the next boot! So naturally when grub wanted to boot the machine it drops to a command prompt after it times out trying to find where root is on /dev/sdh which has disappeared. When this happened, I puzzled on it for a few minutes before digging into grub.cfg, it was not my first thought, but it was the only thing that I could think of that could doing this.

When the machine can boot and drops to the basic command line find out what drives it thinks it has by running…

ls /dev/sd[a-z]2

…this will show all the sda2,sdb3 and etc. Usually but not always the 2nd partition is the Linux File System, this is where the /boot directory lives with all of the GRUB files. Using a USB Stick with a live install or the DVD and booting is helpful at this point as you can use sudo fdisk -l to list what drives are available and you will need to use and editor to modify the grub.cfg file.

Hacking grub.cfg

The hack is to reboot the machine either with the USB stick/Live DVD or off of the hard drive resident in the machine and then….

chmod 644 /boot/grub/grub.cfg

…as it is read only…

-r--r--r-- 1 root root 14791 Jan  7 16:29 /boot/grub/grub.cfg

…remember to chmod it back to read only after editing using….

chmod 444 /boot/grub/grub.cfg

Make a backup copy of it first before editing!

Editing grub.cfg

Once you have it in read/write state open it in an editor, emacs or something, even nano would work.

Yes it complains not to edit it, but when you can’t boot at all, because it can’t find where the root is, it’s worth a try!

 

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

In a terminal window find out where the drive is really mapped using…

sudo fdisk -l

Example of second bootable disk at /dev/sdb…

Disk /dev/sdb: 149 GiB, 160000000000 bytes, 312500000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0007e21b

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdb1  *         2048 279187455 279185408 133.1G 83 Linux
/dev/sdb2       279189502 312498175  33308674  15.9G  5 Extended
/dev/sdb5       279189504 312498175  33308672  15.9G 82 Linux swap / Solaris

The trick is to make the lines for the drive the OS is on that you are trying to boot and that is failing line up in grub.cfg. In grub.cfg search for lines like this…

set root='hd0,msdos1'

If it says hd0 it better be the sda drive for instance. In my case it showed hd7=sdh and I needed to edit hd7 to hd2. This was done via a search and replace.

Also for good measure and to keep all the commenting straight correct anything by searching around for /dev/sd and make sure all the menuentrys line up with the hd* values in set root. This is just so that you don’t get confused later if set root is changed and the menus don’t line up with reality.

menuentry 'Linux Mint 17.3 Xfce 64-bit, with Linux 3.19.0-32-generic (on /dev/sdb1)

After this correction, it booted fine. Now I will just have to pay attention to what happens with drive mapping in case I plug it into another machine. But, it is just an experiment for now, nothing critical on the drive so no worries!

Another note on Boot Repair

If grub gets totally screwed somehow, boot-repair will fix it.

For instance I once had grub lose it’s brains on an EFI boot drive. The symptom was the error about not finding a bootable medium from the BIOS. Use boot-repair. Boot using a live CD or usb stick with exact same version as you are trying to fix! Run the repair and let it automatically fix it.

sudo add-apt-repository ppa:yannubuntu/boot-repair 

sudo apt-get update 

sudo apt-get install -y boot-repair && boot-repair

 

from: https://help.ubuntu.com/community/Boot-Repair

 

Additional Resources

Fix for Linux boot failure and grub rescue prompt following a normal system update

http://giantdorks.org/alain/fix-for-linux-boot-failure-and-grub-rescue-prompt-following-a-normal-system-update/

How do I run update-grub from a LiveCD?

Introduction to fstab

https://help.ubuntu.com/community/Fstab

7490 TTL Chip Digital Clock Schematics

TTL Chip Clock, , , , , , ,

Below are the schematics for the 7490 TTL Chip Clock. After 10 years of not doing anything with the clock, I had to spend some time reverse engineering it a bit to understand it’s design. The original hand drawn schematics ( not published on this site ) were a bit off target as I deviated from them. These new ones that are published are accurate. They were hand drawn then scanned and imported into Gimp for cleanup.

Clicking on the images on this page will bring up a full sized version.

Gimp Drawing Cleanup

In Gimp basically the process is to use threshold to clean the “noise” out from the scan. Then cut more noise out using the cut tool and eraser. Then go over the lines with the pencil tool where they are faint. Finally, add text in. Yes,  there are some CAD tools that could do this easier but, since I don’t do this kind of thing to often,  Gimp works out nicely as I have had a lot of hours on it and can work quickly in it.

Slave Oscillator and Prescaler, dividing from 10MHz to 1kHz on the right hand board

This schematic shows all connections, including power and ground and is laid out to represent the component placement on the board. Looking at the 7490’s the pattern of the circuit is just like the example from How it Works, dividing frequency by 10 every time. The difference between the clock that I built and theirs is that I used a 10MHz source and they used 60Hz line frequency. Therefore I needed to use more divider stages. There is a shortcut, the 74390 chip. It is a BCD dual divider/counter. It could eliminate half of the stages as it divides by 100. I happened to have quite a few 7490s at the time that I built this clock and went with that approach rather than order 74390s. I would recommend using the 74390s though, less stages, less soldering, less power consumption being the benefits.

 

7490-clock-10Mhz-to-1kHz-schematic
7490-clock-10Mhz-to-1kHz-schematic

1kHz Divided down to Hours, Minutes and Seconds on the left hand side board of the clock

This schematic DOES NOT show all connections, power and ground are omitted, also no connects on the chips are omitted, plus the 7447 chips are there to show there location but the connections are not shown. Refer to the page that covers the How it Works information on how to connect 7490s to 7447s. It is laid out to represent the component placement on the board. However the board is inverted on the clock and the digit LEDs sit on the bottom of the board but inverted in order to read them.

7490-clock-1kHz-to-HHMMSS-schematic
7490-clock-1kHz-to-HHMMSS-schematic

Autoshutdown Code Modded to hybrid-sleep and allow required restarts

Linux, Server, , , , ,

Hybrid Sleep Code

I decided to use hybrid sleep instead of a suspend. I have been using the code for autoshutdown as both autoshutdown, using the shutdown command and suspending. A server that I have been using for a year now supports suspend and I have used systemctl suspend successfully with it. But, if the power goes out, the next time it is as if it was shut down and gets a fresh boot. The way around that is to use systemctl hybrid sleep which puts the RAM content into swap and then suspends. This way if the power goes out it will just resume from hibernate.

Reboot code

After setting up the machine with hybrid sleep. I realized that the mechine needs a reboot once and a while after unattended updates and thought that it would be nice to automate that process. I looked on line and found a piece of code that will reboot the machine if a reboot is required. This is done via detecting the presence of the reboot-required file. So far testing once today 8/12/2017, OK so far!

https://muffinresearch.co.uk/how-do-i-know-if-my-ubuntu-server-needs-a-restart/

Snippit of code added to autosuspend a.k.a autoshutdown code that was covered in the original post on this topic …

# If the reboot-required file is present, restart and l$
 if [ -f /var/run/reboot-required ]
 then
 logit "RESTART REQUIRED"
 echo 'Restart required' >> /var/www/html/shutdown.txt
 date >> /var/www/html/shutdown.txt
 echo "------------------------------------" >> /var/www$
 systemctl reboot
 fi

# Fall through and hybrid-sleep it!

systemctl hybrid-sleep
 # Switched to hybrid-sleep 08122017 systemctl suspend
Shadowbox for 7490 Clock

7490 TTL Digital Clock Housing

Projects, TTL Chip Clock, , , , , , ,

After a few months of testing ( April – June 2017 ) the 7490 TTL decade counter chip based clock out with the newly added OCXO it was ready to be mounted in a housing.

7490 TTL Clock with OCXO under test
7490 TTL Clock with OCXO under test

Monitoring it against an accurate timebase ( Linux PC with NTP ) while piled up loosely on a small table in the office, it was running good and steady. No flaky counts, no flaky digits that were not lighting or staying lit when they should not be.

 

 

 

Consolidating Power

After the photo was taken above,  the power supplies for the clock were consolidated. Under testing it was powered by 2 wall supplies, 5V for the TTL chips and 12V for the OCXO. For the final version I used a 7805 regulator on the right hand side board to down regulate 12V from a Radio Shack 12V/2.3A power supply to 5V for the TTL chips. I got lucky and spotted the power supply during the last few days that Radio Shack was still in business in April 2017.

Wood Housing

Not having a good set of tools on hand to build a metal housing, a good option was a  wooden one. Micheal’s craft store had both the base that the boards were mounted to and the shadowbox. Both were unfinished and reasonably priced, I believe $5 for the board and $12 for the shadowbox in 2017.

7490 TTL Clock Circuit Board Base
7490 TTL Clock Circuit Board Base
7490 TTL Clock Shadowbox, cover removed
7490 TTL Clock Shadowbox, cover removed. A bit larger than is needed but, it works fine. The glass cover was removed in preparation for sanding and the application of linseed oil.

The base was drilled out to recess fit 10mm standoffs. Holes were drilled for the M3 screws and then countersunk with a larger bit to accommodate the 10mm standoffs. Once again I made out good at the Radio Shack closeout and got a bunch of standoffs.

Subassembly

7490 TTL Clock Mounted on Base
7490 TTL Clock Mounted on Base

The clock mounted on the base in a state of readiness to drop into the shadowbox as a subassembly. This trick is a bit that I learned from working in manufacturing in the past. To make things efficient and serviceable it was common to mount boards on sub assemblies and then mount the subassembly into a piece of machinery.

 

 

 

7490 TTL Clock Mounted in Shadowbox
7490 TTL Clock Mounted in Shadowbox

 

Finally the subassembly gets dropped into the shadowbox and is held in place by four small wood screws that are on the back.

Previous post in this series

Next Post in this series

7490 TTL Clock Final Housing
7490 TTL Clock, Final Housing