All posts by erickclasen

About erickclasen

This is my Blog for writing about technical items and my other thoughts. I also use it to experiment with WordPress.

Apple Notepad

Termbin

Info Links, Linux, Unpolished Gems, , , ,

I’d like to make sure everybody is at home when they are on vacation.

http://termbin.com/
An easy to use Pastebin like tool that allows “pasting” from the command line.

Requires that netcat is installed on the PC, which is by default on Linux and can be installed on a Windows computer as well.

.bashrc

I recommended adding an alias to your .bashrc in Linux to make a shorthand to post to termbin…

alias tb='nc termbin.com 9999'

 

Posts stay active for a month as an example I posted the link to this post online via termbin.

Privatebin

Other Bins that are install-able
https://privatebin.net/
https://github.com/PrivateBin/PrivateBin/wiki/Installation

How to Setup A Hastebin Server

http://sergiogervacio.com/host-hastebin-server/

 

termbin.com is powered by Fiche – open source command line pastebin server. There is a link to github repository: https://github.com/solusipse/fiche.

LeelaZero Go Game Analysis

Go, , , ,

You need to think beyond yourself. You know how to look, how to act, how to eat, how to live. And you have to know that it’s the soul in you, not the external world in you.

 

Go is famous for it’s games between masters and now it seems like machine learning is a new master after playing Lee Sedol in 2016. One of the most famous games is covered in the book by Yasunari Kawabata, “The Master of Go”, it is the historic 1938 match between the reigning master, Honinbo Shusai and Kitani Minoru, referred to as The Meijin’s Retirement Game. The SGF of the game is available for downloading.

LeelaZero Analysis

LeelaZero can even analyze a previously saved game that was stored in the SGF format. Opening the file in an editor we can see some details in the header of the SGF file, italics are my notes, not in the file…

PB[Kitani Minoru]  Player Black
PW[Honinbo Shusai] Player White
BR[7p] Black Rank 7p, 7 Professional Dan
WR[9p] White Rank 9p, 9 Professional Dan
RE[B+5] Black Wins By 5 Points
EV[The Meijin’s Retirement Game]
PC[Koyokan, Tokyo (2 sessions), Naraya, Hakone (8 sessions), Dankoen, Ito (5 sessions)]
SO[Yasunari Kawabata, “The Master of Go”;

Below is an analysis of the famous Honinbo Shusai and Kitani Minoru game at move 52, not looking good for black as can be seen from the descending green line on the left.

The Meijin's Retirement Game 52
The Meijin’s Retirement Game, Move 52
Penguin

Pastepile

Linux, Unpolished Gems, Web Hosting, , , , , , , , ,

Future development depends on the determination of future generations to contribute towards building their own society.

Pastepile is a fork of guestbook.cgi – a simple guestbook script, written by John Callender in 1999. It is the last part of his Beginner’s Guide to CGI Scripting with Perl, Running a Guestbook.
This new version, a.k.a Pastepile, is a dirt simple Perl CGI program that creates a blog like pasting tool to be able to paste notes online or on a local server. Allows grabbing snippets of text or code for later use on whether on the local machine or another on the network. I made it sometime after studying John Callender’s tutorial when I was digging into learning about CGI and Perl several years ago.

History

Pastepile originated as a way to paste snippets of code and write short notes on configuration changes that I make on servers, including a Raspberry Pi that I run. The only requirement is the installation of a web server such as Apache. This little Pastepile tool made it easy to keep track of information related to the servers in one place  and search-able via a browser. After using it in the is mode for a while, I cloned it and started to use it as a general place to paste info and write short notes. This makes it easy to save something while on one machine on the network and be able to retrieve it later on that machine or another.

Example of What it Looks Like

Subject: Running a form-to-email gateway at 19:04:44, Sat Feb 3, 2018 from 192.168.1.7

http://www.resoo.org/docs/perl/begperl/form_to_email.html

Subject: Slackware Linux Essentials at 12:10:03, Thu Feb 1, 2018 from 192.168.1.179

www.slackbook.org/html/index.html

Additions:Remote Address,Time,Reversed Order

Besides stripping down guestbook.cgi to remove code specifc to it’s guestbook function and cleaning up the code to reflect it’s new use, three functional code changes were made. One is to add a timestamp to keep track of when the entry was created. Another change made was showing the IP of the machine the post originates from by reading the environment variable $REMOTE_ADDR to keep track of where the post originates from. Many machines on my network have static IP’s so this is handy for me at least. The final change was to reverse the order of the listing. In guestbook.cgi a First In On Top ordering is the layout of the guestbook posts. For Pastepile I reversed it so the most recent and not the oldest post is on top of the list, Last In On Top ordering. This made sense as the most recently written notes are more likely to be important and I want those to be close to the top of the pile and let the file get long with the aging information at the bottom.

/p/

For ease of access via a shorter link a simple index.html redirector page was place in /var/www/p/, following a 4chan-esqe style of folder naming and a nice short link to get to the pastepile.cgi script, which lives in the /usr/lib/cgi-bin/ directory. The /p/ directory also holds the pastepile.html file that is created by pastepile.cgi as it’s data page.

 Redirector Example for /var/www/p/index.html

<HTML>
<HEAD>
<meta http-equiv="refresh" content="0; url='http://192.168.1.17/cgi-bin/pastepile.cgi'"/>
</HEAD>
<BODY>
<p><a href="http://192.168.1.17/cgi-bin/pastepile.cgi">Redirect</a></p>

</BODY>
</HTML>

paste-pile-mover.sh

A helper script called pastepile-file-mover.sh, moves the pastepile.html, the data file created by pastepile.cgi from it’s default location to a date stamped file in a location set in the script BASEDIR/dir/filename. Where BASEDIR is your choice ( I use /var/www/p/), dir is the current year and filename is YYYYMMDD.html, so that that there is a  year and date hierarchy. I allow this script to run at the start of the month via root CRON to “clean” out the Pastepile and archive the old one, much the same way that log are rotated.

0 0 1 * * /home/erick/bin/pastepile-file-mover.sh

pastepile-file-mover.sh

#!/bin/bash

# Move the pastepile.html file from it's default location to a date stamped
# file in a location BASEDIR/dir/filename
# So that it has year and date heirarchy

# For now, Monthly, Move pastepile over to year dir and datestamped HTML file
#0 0 1 * * /home/erick/bin/pastepile-file-mover.sh



BASEDIR=/var/www/p
# Testing
#BASEDIR=/tmp

dir=$(date +"%Y")
#echo $dir

#File name timestamped
filename=$(date +"%Y%m%d").html
#echo $filename

# Make the YEAR dir if it dow not exist.
if [ ! -d "$BASEDIR/$dir" ]; then
  # Control will enter here if $DIRECTORY doesn't exist.
  mkdir $BASEDIR/$dir
fi

# Do the move to the YEAR directory with the YYYYMMDD.html filename.
mv $BASEDIR/pastepile.html $BASEDIR/$dir/$filename

# This is needed make a new pastepile.html and chmod 666
# else pastepile.cgi does not work, it can't make it's own output file.
touch $BASEDIR/pastepile.html
chmod 666 $BASEDIR/pastepile.html

Last but not least pastepile.cgi

#!/usr/bin/perl -Tw

# pastepile.cgi a fork of...
# guestbook.cgi - a simple guestbook script

# This program is copyright 1999 by John Callender.

# This program is free and open software. You may use, copy, modify,
# distribute and sell this program (and any modified variants) in any way
# you wish, provided you do not restrict others from doing the same.

# pastepile.cgi - guestbook.cgi, modded to become # a pastepile program. Erick Clasen Jan 25,2018
# This new version is a dirt simple CGI program that creates a blog like
# pasting tool to be able to paste notes online or on a local server.
# allows grabbing snippets of text or code for later use on whether
# on the local machine or another on the network.

$data_file = '/var/www/p/pastepile.html';

$max_entries = 0; # how many guestbook entries to save?
                   # set to '0' (zero) for infinite entries...

use CGI;
use Fcntl;
$query = new CGI;

unless ($action = $query->param('action')) {
    $action = 'none';
}

print <<"EndOfText";
Content-type: text/html

<HTML>
<HEAD>
<TITLE>Raspberry Pi Server Paste Pile</TITLE>
</HEAD>
<BODY>
<H1>Raspberry Pi Server Paste Pile</H1> 
<P><EM>$ENV{DOCUMENT_ROOT}/p/</EM></P>



<A HREF="../status/index.html">Back to Status Index</A>&nbsp;
<A HREF="../p/2018">2018 Paste Archive</A>

<P>You can <A HREF="#form">add your own subject and entry</A> using the form at the bottom of the page. Here we is has your pastes...</P>

<HR>
EndOfText

# Input action to add a new entry. ----------------------
if ($action eq 'Add entry') {

    # process the form submission
    # and assemble the guestbook entry


    # Input the subect and the paste which is called a comment here in this code.
    $subject = $query->param('subject');

    $comment = $query->param('comment');

    # clean up and fiddle with $subject
 unless ($subject) {
        $subject = 'No Subject';
   if (length($subject) > 50) {
        $subject = 'Subject line too long >50 chars';
    }
# End Input action to add a new entry. ----------------------



    }

    # Add a time stamp, put in variable theTime. This allows the paste to be timestamped.
    
    @months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
    @weekDays = qw(Sun Mon Tue Wed Thu Fri Sat Sun);
    @digits = qw(00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 59 59);
    ($second, $minute, $hour, $dayOfMonth, $month, $yearOffset, $dayOfWeek) = localtime();
    $year = 1900 + $yearOffset;
    $theTime = "$digits[$hour]:$digits[$minute]:$digits[$second], $weekDays[$dayOfWeek] $months[$month] $dayOfMonth, $year";

    # untaint variable
    unless ($theTime =~ /^([^<]*)$/) {
        die "couldn't untaint name: $theTime\n";
    }
    $theTime = $1;

    

    # clean up and fiddle with $subject--------------------------------

    $subject_clean = "$subject";
    $subject_clean =~ s/, , /, /;        # remove duplicate ', '
    $subject_clean =~ s/^, //;           # remove initial ', '
    $subject_clean =~ s/, $//;           # remove final ', '
    if ($subject_clean =~ /^[,\s]+$/) {
        # nothing but commas and whitespace
        $subject_clean = 'Subject format wrong!!';
    }
    
    if (length($subject_clean) > 75) {
        $subject_clean = 'Subject too long.';
    }

    # disable HTML tags
    $subject_clean =~ s/</&lt;/g;

    # untaint variable
    unless ($subject_clean =~ /^([^<]*)$/) {
        die "couldn't untaint subject_clean: $subject_clean\n";
    }
    $subject_clean = $1;

    

    # clean up and fiddle with $comment----------------------------

    if (length($comment) > 32768) {
        $comment = '...overloaded blog buffer chars > 32768.';
    }
    unless ($comment) {
        $comment = '...nothing to speak of.';
    }

    # fix line-endings
    $comment =~ s/\r\n?/\n/g;

    # lose HTML tags
    $comment =~ s/</&lt;/g;

    # untaint variable
    unless ($comment =~ /^([^<]*)$/) {
        die "couldn't untaint comment: $comment\n";
    }
    $comment = $1;
    # end cleanup #comment-----------------------------------------

    

    # assemble finished guestbook entry -- anything after this line until EndofText will show in the post!!!!!

    # Enviroment variable for REMOTE_ADDR is grabbed and printed directly. No untainting, but probably safe to do.

    $entry = <<"EndOfText";

<P><STRONG> Subject: $subject_clean </STRONG> at <EM>$theTime </EM> from <EM>$ENV{REMOTE_ADDR} </EM> <BR>
<BLOCKQUOTE>$comment</BLOCKQUOTE></P>
<HR>
EndOfText

    # open non-destructively, read old entries, write out new
   $all_entries .= $entry;
    sysopen(ENTRIES, "$data_file", O_RDWR)
                             or die "can't open $data_file: $!";
    flock(ENTRIES, 2)        or die "can't LOCK_EX $data_file: $!";
    while(<ENTRIES>) {
        $all_entries .= $_;
    }

 if ($max_entries) {

          # lop the tail off the guestbook, if necessary

          @all_entries = split(/<HR>/i, $all_entries);
          $entry_count = @all_entries - 1;

          while ($entry_count > $max_entries) {
              pop @all_entries;
              $entry_count = @all_entries - 1;
          }

          $all_entries = join('<HR>', @all_entries);

      }


   

    # now write out to $data_file

    seek(ENTRIES, 0, 0)        or die "can't rewind $data_file: $!";
    truncate(ENTRIES, 0)       or die "can't truncate $data_file: $!";
    print ENTRIES $all_entries or die "can't print to $data_file: $!";
    close(ENTRIES)             or die "can't close $data_file: $!";

}

# display the guestbook a.k.a pastepile.html

open (IN, "$data_file") or die "Can't open $data_file for reading: $!";
flock(IN, 1)            or die "Can't get LOCK_SH on $data_file: $!";
while (<IN>) {
    print;
}
close IN                or die "Can't close $data_file: $!";

# display the form    

print <<"EndOfText";
<A NAME="form"><H2>Add a comment/entry to the paste pile (no HTML):</H2></A>

<FORM METHOD="POST" ACTION="pastepile.cgi">
<TABLE>



<TR>
<TD ALIGN="right"><STRONG>Subject: </STRONG></TD>
<TD><INPUT NAME="subject" SIZE=30></TD>
</TR>

<TR>
<TD ALIGN="right"><STRONG>Entry :</STRONG></TD>
<TD>
<TEXTAREA NAME="comment" ROWS=5 COLS=30 WRAP="virtual"></TEXTAREA>
</TD>
</TR>

<TR><TD COLSPAN=2> </TD></TR>
<TR>
<TD> </TD>
<TD><INPUT TYPE="submit" NAME="action" VALUE="Add entry"></TD>
</TR>
</TABLE>

</FORM>
</BODY>
</HTML>
EndOfText

 

fb-submit

Meta Response

My Thoughts, , , , , , , , , ,

Does the interaction of social and emotional data with the power of social and emotional data translate into positive results?

Recently I was given a Meta Survey via Facebook and I replied with this…

What is your biggest concern about Meta?

I work with ML/AI and I know it is possible to steer people with it to get a given outcome in very subtle ways. My concern is that companies like Meta are steering people towards content that produces a psychological reaction that then produces addictive behavior in order to keep them engaged with the goal of gathering personal data for the purposes of sale of such data and targeting people with advertising that encourages them to consume more than they would under normal circumstances. Generally with social media the corporations running it are extracting a resource, as in data, time and attention from people in a way that tilts the scales in favor the social media platform providers. To improve this would require more transparency with regards to the data that is collected and an ‘open book’ policy where individuals must give consent to all elements of data harvesting and are allowed to access the records that are kept on their personal data. Also, to have tools built into the platforms that allow people to block things as simple as using keywords to black/whitelist content and optimize the user experience in a way that allows them to override algorithmic control of the content that they are presented with, including advertising. AI/ML works best when it is collaborating with the user and not in the role of an invisible hand that exercises control over their experience from beyond their control.

What do you value most about Meta?

The fact that there has been some open source release of code such as PyTorch which I use on a regular basis.

Meta Survey
Meta Survey

 

Extremism and Mental Illness and Social Media

There is something that is pushing extremes and actually it is in both directions. This seems to have happened ever so slowly, it has crept in. Someone I heard on a podcast said once “Almost everyone is in Wokeistan or Trumpistan and there are like 7 people left on Earth that are like WTF???” It is an exaggeration but makes a point. There are disturbing things in the realm of mental illness, one of the worst is the fact that there is now a catagory for suicide in pre teens. There was no such category at one point as it was too rare of an outlier to keep track of, since 2007 there is. What happened around that time? Smartphones with social media apps, correlation is not always causation but, it does make you wonder. If anything the issue pushing extremes can be found in the media, both traditional as it grapples with loss of market and pushes more extreme ‘bait’ to keep viewers. And, lastly you guessed it, social media in which the invisible hand of algorithms feed us content determined by large corporate interests. I think that it will, and this is the hope, that it will selfcorrect, the pendulum when it gets to an extreme will eventually go in the other direction. This is the blessing of free, democratic countries, change is possible. Democracies are a mess but, they are the best we have as of now,considering the alternatives have generally turned out worse for people.

sellbuy-o-meter

Trading, , , , ,

We are governed by laws, not by laws, but by people. Government is by people.

We all need a sellbuy-o-meter

  • Get the timing just right every time.

  • No more missing the tops and bottoms.

  • Maximize profits right from the start.

  • Be a winner every time.

Hey there, it’s your favorite lawyer, Saul Goodman, and I’ve got some advice for all you traders out there. You need a sellbuy-o-meter, plain and simple. This little gadget tells you when to buy and sell, so you can get the timing just right every time. No more missing the tops and bottoms of the market, my friends. You’ll be maximizing your profits right from the start and be a winner every time.

With this sellbuy-o-meter, who needs stop losses? You’ll be making maximum gains with no losses. Your eyes will thank you, too, because there’ll be no more chart reading voodoo and magic. Put away your lucky charms and get ready to use some 10x or more leverage.

And let me tell you, you’ll sleep like a baby at night knowing you’ll never get a trade wrong. Risk management will only entail replacing the batteries in the meter. (Batteries not included, unfortunately.)

So what are you waiting for? Order your sellbuy-o-meter today and get a free spouse-o-meter! This little beauty tells you what mood your significant other is in – mad, sad, or glad – we’ve got them all covered. Don’t hesitate – order now!

sellbuy-o-meter
sellbuy-o-meter

Sometimes these Clickbait con-tra-prenur social media ads just get under my skin. It feels a little bit like I am being sold a scam by Slippin Jimmy a.k.a. Saul Goodman.

Daytrading: Response to Clickbait Ad

I responded to an clickbait ad on social media with the following.

Daytrading, I would not recommend it for newbies. It’s not for everyone and it’s not something to jump into blindly. With trading in general, start very small, like 1% of your net worth, learn and then build up your position sizes slowly as a reward for success. Other than your ‘play’ money, while learning just buy and hold with the bulk of you invest able $. Daytrading is something to do only when a level of mastery of trading on higher time frames using small money. Trading will cost you money and time until you gain experience, these are the facts, plain and simple. It is part skill, part art, timing and luck. The shorter the time frame, the higher the market noise relative to the signal that you are trying to capture and make trades on, this combined with fees and slippage makes daytrading difficult to begin with (Super quick trades, noise, fees and slippage will eat your $ up.). Is it possible to make money daytrading ,yes, but you will be strapped to a computer to do it, is that something that will be enjoyable in the long run, seriously think about it. Do you really want to sit in front of the computer watching 1,5,15m candles with a finger hovering on the mouse all day. That’s for machines to excel at, really, that’s how the big firms can even make money at all, besides low fees and getting paid for order flow. Being able to code algorithms to trade for you via an API, then you just have to babysit the system to make sure it is working right, less work than sitting in front of charts on PC all day, requires math,coding and fintech type skill however. That might be a path to daytrading for the committed in the long run. If none of the preceding jargon makes sense, time to study up. Knowledge of probabilities and statistics go a long way, I had to relearn quite a bit of this, yeah it sucked a bit, should have paid more attention in those classes. Understand what the Kelly Criterion is, “Fortune’s Formula” the book, covers it. That way you can get a grasp on risk management and don’t blow up your account. A so-so trader with good risk management will make money. A good trader with poor risk management will fail, the account will go up and down, maybe most and more down,boom and bust and potentially fully bust. Works by Ed Thorp, Elder Alexander, Perry Kaufmann, John Murphy are a good foundation to understand trading. Reminiscences of a Stock Operator Book by Edwin Lefèvre Zen, they hand this out at Goldman when you start there. Get to understand the Psychology of trading and money (Morgan Housel), the ups and downs are rough. If you don’t get a grasp on how it impacts you emotionally it will be painful in the gut. Position sizes, watch those, you want to be able to sleep at night, once again we are talking risk management. If you’ve studied the foundations and it still doesn’t make sense keep studying, it might keep sinking in while you practice trade with small money and put a chunk in buy+hold. As you gain experience some of the material you study will start to make more sense, time to reread. If after all this, it still doesn’t work, find a mentor who will help you, someone who has “made it” and is willing to give back to help the next generation. Ideally buddy up with someone from the start, find a club. There is too much hype and charlatanism out there waiting to mislead and take advantage of newbies, pushing a lot of nonsense.

Another Similar Response to an Ad on how to figure out Bull/Bear conditions

You can tell if it is bullish or bearish in a few minutes, why it is bullish or bearish, well that can take much longer to figure out. But, you can learn this yourself or find a mentor. Hang around the right people and you will learn, find a mentor who will help you, someone who has “made it” and is willing to give back to help the next generation. Ideally buddy up with someone from the start, find a club. There is too much hype and charlatanism out there waiting to mislead and take advantage of newbies, pushing a lot of nonsense………But for now, practical wise, an example of figuring out bull/bear bias of an asset. Ichimoku chart is good to use and learn, on it, is the Tenken above the Kijun? Is the price above both of those? Is the price above the cloud? Also is the asset in a golden cross 50 day MA > 200 day MA. Then look at the macro picture, over all market up? Is the sector up? These are Bull signals, the more the better. Better yet, figure out your own method and come up with a checklist. Also, price can always change on a whim, you can’t predict it. Use risk mgmt to mitigate against loss, tomorrow there could be a banking crisis, political crisis, war, flash crash, protect against those as well as you can.

laptop

Scraping a WordPress site for text to use with char-rnn

Linux, Scripts, WordPress, , ,

As with anything else in life, it is possible to change nothing but yourself.
The first step toward making change is simply to change yourself.

In this quick example the gist of scraping a WordPress website using Linux and Lynx will be shown. Wget is great at scraping from the web but, I have found out that it does not always work well with WordPress sites.

Grab some text

Step one of the process is to grab some text to work with. In the example I tried I grabbed all of the text of the posts of this blog by scraping it with wget. I also used the text of the US Constitution to see what the tools would do with it as well. Generally the more text, the better the machine learning code will be at generating something interesting.

Scraping the posts

Using the command line web browser lynx in a script I was able to download the text of the posts on this site. Initially I thought to use wget. But, I remembered that wget will do a good job downloading static sites and sometimes will not do so well with ones like this one that is created in WordPress.

There is probably a way to loop this code in bash, and increment a counter for the pages. But, being that this is a one time thing, I opted for a quick approach instead of thinking too hard on making a loop.

#!/bin/bash
lynx -dump -nolist http://erick.heart-centered-living.org/ > my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/2/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/3/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/4/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/5/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/6/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/7/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/8/ >> my-posts.txt
lynx -dump -nolist http://erick.heart-centered-living.org/page/9/ >> my-posts.txt

This code will output a file that contains all of the text from the posts on this site. Up to Fall of 2018 when I ran it.

Wave Styled by ML

Modifying a image with a style using machine learning

Machine Learning, , , , , , , , , , ,

The first ingredient to happiness in happiness is an attitude of openness.

Fun With Machine Learning

I have been sharing the following images around. They were created with machine learning code that I came across and modified a bit while examining how it operates.

See the following for more info… How to Generate Art Demo Command Line Version

The code basically takes an image and imposes a style from another image upon it. It is rather computational expensive as it takes around 12 Gigabytes of RAM to “work” on a 1024×1024 pixel image and about 2 hours of compute time to run the 20 iterations required to complete an image. Machine learning is a fascinating new frontier in technology that I have been spending some time since the spring of 2018 getting to understand on a deeper level. I’ve seen a lot of technologies come and go but, this field has stunned me, is moving forward very fast and is here to stay.

Prior to 2018 I had some exposure to machine learning in the sense of using adaptive control systems in industry. I also worked on a research project that involved a type of fuzzy logic and cellular automata for a learning system that would be used in a control loop. I also developed code that used a tractrix curve as the main element of a control system. But, that is kind of simple machine learning as compared to what is going on today.

This link is to the original paper on this topic, there are some more images in it as well,

https://arxiv.org/pdf/1508.06576.pdf?fbclid=IwAR19_eqx3SmHMyqnAnirbfyAWDqLYkqMp98C0XZ6GEfeXZRxnDPaagZ5B_E

iPhone Mounting

Linux, Unpolished Gems,

The beauty of love is not in your beauty but in your ability to make it manifest.

Mostly because I have had to look up how to mount an iPhone on Linux Mint 18.0 a few times and had to look it up,I made a helper script. – iPhone-mount.sh

#!/bin/bash
 # Allow non-root users to mount with ifuse!
 #sudo nano /etc/fuse.conf
 # Allow non-root users to specify the allow_other or allow_root mount op$
 # UNCOMMENT user_allow_other

# Pair the iPhone
 idevicepair pair
 sleep 5
 # Mount it
 ifuse /home/erick/iPhone

Allow users to mount via ifuse

In /etc/fuse.conf uncomment user_allow_other to let users mount devices. This makes it easy to mount the device in your home directory for instance with full access to it.

Mint and Mounting

The iPhone mounts fine on Mint 18.0. My desktop has a newer version and I had to do work to it before mounting the iPhone worked. Apparently something got lost between versions.

https://www.dedoimedo.com/computers/linux-iphone-6.html

RNN Text Generation Using Tensorflow

Machine Learning, Text Generation, , , ,

Imagination is the power to make a difference in yourself.

After trying a few RNNa and LSTMs for text generation that rely on numpy alone it is interesting to see the performance of Tensorflow based code that is closer to the cutting edge of what is possible to do with machine learning.

I found a good and easy to use set of code in the following Github archive…

https://github.com/spiglerg/RNN_Text_Generation_Tensorflow

The requirements are simple…

numpy==1.13.3
tensorflow==1.4.0

I was running it on Conda Python 3.6 environment but, this is not a requirement. The code uses a saved folder where it can save training checkpoints, so it is possible to interrupt and resume training and also use it in a generate or “talk” mode after the model has been trained. The caveat that I learned quickly when training on a few types of files is when training each type of file that is trained into requires it’s own set of checkpoints, which is pretty obvious. So it is best to either wipe out the saved dir contents after a run on a specific corpus. OR, better yet make a subdir for the training checkpoints.

Training is basically sending it the following command…

python rnn_tf.py --input_file=data/us-constitution.txt --ckpt_file="saved/model.ckpt" --mode=train

Once trained it can be fed another command…

python rnn_tf.py --input_file=us-const-lstm/us-constitution.txt --ckpt_file="saved/model.ckpt" --test_prefix="The " --mode=talk

or if the checkpoint files have been moved to their own directory then you can use something like this…

python rnn_tf.py --input_file=us-const-lstm/us-constitution.txt --ckpt_file="saved/us-const-trained/model.ckpt" --test_prefix="The " --mode=talk

In the structure for the commands the location of the file is listed and the location of the checkpoint file as well. The generate mode allows priming with a word or phrase such as “The”.

The US Constitution is not a big corpus and I am sure this code like others would benefit from training against a larger corpus. My intent in the future for an experiment is to train it against a file containing all the posts on this site to see what it can do on that corpus.

-rw-r–r– 1 erick erick 1115394 Apr 26 12:01 shakespeare.txt
-rw-r–r– 1 erick erick   45120 Apr 26 12:59 us-constitution.txt
-rw-r–r– 1 erick erick  374605 Apr 26 13:52 my-posts.txt

When trained on the US Constitution it does very well at producing coherent text. Besides the lack of capitalization it seems to be actually to the point of memorizing parts of the text. This might be because it is a small corpus and it is overfitting.

The Senators and Representatives before mentioned, and the Members of the
several State Legislatures, and all executive and judicial Officers, both of
the United States and of the several States, shall be bound by Oath or
Affirmation, to support this Constitution; but no religious Test shall ever be
required as a Qualification to any Office or public Trust under the United
States.

Article 7.

The Ratification of the Conventions of nine States, shall be sufficient for the
Establishment of this Constitution between the States so ratifying the Same.

Sentence:
the several states, shall be bound by oath or
affirmation, to support this constitution; but no religious test shall ever be
required as a qualification to any office or public trust under the united
states.

article 7.

the ratification of the conventions of nine states, shall be sufficient for the
establishment of this constitution between the states so ratifying the same.

done in convention by the unanimous consent of the states present the
seventeenth day of september in the year of our lord on

 

 

the Case of a Bill.

Section 8
The Congress shall have Power To lay and collect Taxes, Duties, Imposts and
Excises, to pay the Debts and provide for the common Defence and general
Welfare of the United States; but all Duties, Imposts and Excises shall be
uniform throughout the United States;

To borrow money on the credit of the United States;

To regulate Commerce with foreign Nations, and among the several States, and
with the Indian Tribes;

To establish an uniform Rule of Naturalization, and un

Sentence:
the case of a bill.

section 8
the congress shall have power to lay and collect taxes, duties, imposts and
excises, to pay the debts and provide for the common defence and general
welfare of the united states; but all duties, imposts and excises shall be
uniform throughout the united states;

to borrow money on the credit of the united states;

to regulate commerce with foreign nations, and among the several states, and
with the indian tribes;

to establish an uniform rule of naturalization, and un

 

Training

Training against the corpus of blog posts on this site produced output like this and took about 4 hours of compute time.

batch: 0  loss: 4.492201328277588  speed: 121.8853488969507 batches / s
batch: 100  loss: 3.214789628982544  speed: 1.3747759497226923 batches / s
batch: 200  loss: 3.0983948707580566  speed: 1.4065962415903654 batches / s
batch: 300  loss: 2.8669371604919434  speed: 1.4141226357348917 batches / s
batch: 400  loss: 2.359729051589966  speed: 1.416853411853437 batches / s
batch: 500  loss: 2.0080957412719727  speed: 1.4160802277642834 batches / s

…

batch: 19500  loss: 0.22069120407104492  speed: 1.4188681716674931 batches / s
batch: 19600  loss: 0.21757778525352478  speed: 1.4218841226396346 batches / s
batch: 19700  loss: 0.2309599369764328  speed: 1.362554971973392 batches / s
batch: 19800  loss: 0.23969298601150513  speed: 1.3983937654375616 batches / s
batch: 19900  loss: 0.23989509046077728  speed: 1.3854887855619515 batches / s

The following is some samples of the output it generates. It definitely could use more training to help it. The fact that the posts contain some code, numbers and jargon probably doesn’t help either.

Sentence:
the installed.
 display install wiflinut for ray run process queue every monday, wednesday and friday ran
   1000000 ractine resitely and configure a firewall to only allow certain ip numbers a
   connection to show that the board is
   powered. there are a concatenated version of the log.txt
cacking out of the full -ho 1 than i could have it may be set the
   command which just restart the “how ther have up suncals
   regulator, frequency valies.
   more data. i sho, vift…
sudo selond

   below is

Sentence:
the whole hmad can noid through the server and logged
in a while later and the shutdown script had recorded failed pings into
systemctl.

i was not ne rewent when it shuts down.

for a help afout shourd entire (but mean most looking a series of
for clean ubuntu server install will prompt for a username and password to access folders as
well, especially if the users and password is needed autosuspend should oright. it level, no 62 defanly 34-fermentation crontab, still radio
shar

 

 

 

I is has Cheezburger

Greens

Food Preparation, Unpolished Gems, ,

Greens, always good with a cheezburger.

Greens, collard or kale or both

Cut up a medium onion, medium chop. Put this into a pan in which you have melted coconut oil or bacon fat plus a bit of olive oil. Plus about 1 tbsp of sesame oil

Cover and cook slowly, incorporate some dried curry, a few tbsp. Stir occasionally

Meanwhile cut up 4 garlic cloves, dice well and place the garlic on top of the onions in the pan and allow it to cook this way for a few minutes before stirring.

Stir the mixture, be careful with the heat once the garlic is in, burning it is bad.

Meanwhile prepare the greens, Cut them de-stem, all the prep gets done while slowly cooking the Onion, garlic & curry.

Collard Greens, Swiss
Fill the pan with the greens; kind of overfill it a bit. Add some broth or stock, vegetable, chicken, whatever you have on hand. This flavors and produces steam.

Put on a lid. Increase to medium heat for a while. Stirring occasionally, add in some Teriyaki or soy or both. Plus a bit of Worcestershire sauce. A few tbsp of each.

Next cut up some ginger, dice finely.

Push the greens to one side of the pan and put the ginger in the open area with a bit of oil, olive oil works good. Cook for a few minutes.

Stir the entire mixture, add more broth or stock, and reduce heat to low. Lid back on.

Cook for up to one hour on low, check occasionally and add more broth or stock.

When the greens are fully wilted and have soaked up the flavors, they are done.