Monthly Archives: March 2020

Bull market 2009 to 2020

Trading Resources Books

Speculation in stocks will never disappear. It isn’t desirable that it should. It cannot be checked by warnings as to its dangers. You cannot prevent people from guessing wrong no matter how able or how experienced they may be. Carefully laid plans will miscarry because the unexpected and even the unexpectable will happen. Disaster may come from a convulsion of nature or from the weather, from your own greed or from some man’s vanity; from fear or from uncontrolled hope. – Reminiscences of a Stock Operator


This post is a trading resource dump on books for trading 

I have some good books in the list. The ones that cover the psychological aspects of trading are worth the time to read if you have gotten past the basics of trading. It helps if you have traded a little at least to have some hands on experience with it. Best to start out with a few hundred dollars and build up the account as you get the hang of trading. Adding to the account slowly. I wish I had read these books  in the beginning, just when I was trying the first half dozen trades, at the same time that I was learning about the technical aspects of trading.

 

Subject: Road less traveled M. Scott Peck

https://www.apnamba.com/Ebooks-pdf/The%20Road%20Less%20Traveled.pdf

The first section on discipline is a good read for the psychology of trading. Think discipline in terms of trading.


Subject: Technical Analysis (TA) of financial markets John Murphy

Classic TA book. This along with Perry J. Kaufman’s book New Trading Systems and Methods will be good references for the nuts and bolts of trading.

http://194.145.209.129/bk/wp-content/uploads/2020/02/download.php?file=Technical-Analysis-of-the-Financial-Markets-8freebooks.net_.pdf

Subject: Another TA Book, Open Source

This book is a good reference book. It compiles a lot of information that is in the public domain, mostly via Wikipedia in one place. At the end of each chapter there are good notes, references and further reading. It is good to have this on hand when you want to look something up quick or for beginners to get an overview of technical analysis.

http://www.mrao.cam.ac.uk/~mph/Technical_Analysis.pdf

In the Markets: Confessions of a Samurai Trader Edward Alan Toppel

Worth a read, especially for anyone that has been exposed to Asian culture. It still makes sense if you’re not familiar with the culture but, if you are it’s going to be a deeper read.

Confessions of a Samurai Trader Ebook https://www.forexfactory.com/attachment.php/2566801?attachmentid=2566801&d=1510983772

Trading for a living by Elder Alexander

Very good, Elder Alexander is a Psychologist turned trader. He is also the inventor of the Triple Screen Trading System, which is outlined in the book.

This is in download PDF http://www.saham-indonesia.com/Ebooks/Technical%20Analysis/Elder%20Alexander%20-%20Trading%20For%20A%20Living.pdf

 Reminiscences of a Stock Operator Book by Edwin Lefèvre Zen

There is nothing like losing all you have in the world for teaching you what not to do. And when you know what not to do in order not to lose money, you begin to learn what to do in order to win. Did you get that? You begin to learn!

Very good, a must read! This book is supposedly handed out to new employees at Goldman Sachs to read as a first assignment. I can see why, it really is an eyeopener. It is a book I wished that I had read right in the beginning of starting to consider trading myself.

https://ia803009.us.archive.org/4/items/JesseLivermoreReminiscencesOfAStockOperator/Jesse%20Livermore%20Reminiscences%20Of%20A%20Stock%20Operator.pdf

General Economics and Investing

While not necessarily for trading, it’s helpful to have a broad selection of different types of knowledge in your latticework of the mind as Charlie Munger calls it. In case you didn’t know, Charlie Munger is Warren Buffet’s right hand man at Berkshire Hathaway.

Investing , the last liberal art by Robert G. Hagstrom

This table of contents is what got my attention with this book. I was a bit caught of guard at first with the topics and the connection they might make to investing. But, a short skim confirmed the author was right on target with all of the topics covered.

Table of Contents: Investing: The last Liberal Art
Table of Contents: Investing: The last Liberal Art

I didnInvesting, the last liberal art’t know what to think of this book when I first saw it. But, it’s different from most investing books in a good way. It’s more about building a mindset, a latticework in your mind to pull from to better think about investing. Clearly, knowing other subjects beyond economics and finance are helpful to have a background of general knowledge to be able to pull ideas from. For me, I agree as this has been my own experience. This book went from one that I was skeptical about to a favorite after a few chapters.Initially the chapter titles caught my attention, as they were unlike any ones that I have seen in other investing books.

 

Principles of Economics by Alfred Marshall

Originally from 1890, it’s a classic. It’s still used in some college curriculums today. This book can provide a background on economics. With trading, you need to be able to understand the view from above as well. Understanding how economics works provides a high altitude view from far above the landscape of fundamentals and technical analysis. It’s important to a least have some understanding of the bigger picture, a macro view of economics from the beginning when you trade.

This is the 8th edition of what is regarded to be the first “modern” economics textbook, leading in various editions from the 19th into the 20th century. The final 8th edition was Marshall’s most-used and most-cited.

http://files.libertyfund.org/files/1676/Marshall_0197_EBk_v6.0.pdf

Crypto Compares Big Gains Q1 2019

Trading Resource Videos

There are many thousands of people who buy and sell stocks speculatively but the number of those who speculate profitably is small. As the public always is “in” the market to some extent, it follows that there are losses by the public all the time. The speculator’s deadly enemies are: Ignorance, greed, fear and hope. All the statute books in the world and all the rules of all the Exchanges on earth cannot eliminate these from the human animal. Accidents which knock carefully conceived plans skyhigh also are beyond regulation by bodies of coldblooded economists or warm-hearted philanthropists.– Reminiscences of a Stock Operator

 

Trading Resource Videos

The following post contains video resources. Some of this material especially the material from Francis Hunt and David Paul on the psychology and mindset of trading is quite important. It is best to learn about the psychology and mindset of trading while learning other typical trading topics such as technical analysis and fundamentals. It is just as important in the long run and having discipline and the right mindset goes a long way to preventing losses and maximizing gains over the long haul.

Channels

Josh Olszewicz: Crypto + Technical Analysis, Building your trading toolbelt

Josh Olszewicz produces market updates on a regular basis cover BTC and ETH primarily. Plenty to learn from and in my opinion, reliable insights.

https://www.youtube.com/channel/UC587BAG9cLTYtJ7Q4CqcOnw

Francis Hunt: The Crypto Sniper

Francis Hunt produces periodic crypto market updates and actually trades in legacy markets as well and provides useful insights from both realms and also currencies plus gold and silver.

https://www.youtube.com/channel/UCdC4a2KquFV1F4O21mW3K7g

Aswath Damodaran: Fundamentals and Value Investing in Stocks.

He is a professor, so get ready to study. Many videos on his channel. A few updates here and there but , most of it is in depth college level training on topics such as valuation and analyzing the fundamentals of companies. Good information for a background on the stock market. Plenty of videos to pick and choose from as you want to dive into a specific topic.

https://www.youtube.com/channel/UCLvnJL8htRR1T9cbSccaoVw

 

UKspreadbetting

Basics. Well done videos, most of them nice and short and covering specific topics. Everything that applies to spreadbetting can be applied to trading as well so don’t let the name fool you.

https://www.youtube.com/channel/UCnKPQUoCRb1Vu-qWwWituGQ

Trading 212

Basics. Watched a few videos off of this channel a bit of overlap with UKspreadbetting but short concise videos. Worth looking at.

https://www.youtube.com/channel/UCfWQuGYMfhZk9qq5x00sb9w


Series Worth Watching

Francis Hunt Interview Series on UK Spread Betting

Especially these two in the series are very important…
Trading Secrets of the Mind Master the Emotional Side of Trading
Improving the Mindset Game in our Trading

David Paul on UK Spread Betting

More David Paul Videos

 

https://www.youtube.com/watch?v=MGglyvc8d58&list=WL&index=113&t=0s

Subject: David Paul Tips from video

Asset above 89 day MA. Market above 21 day MA And rising , will get hit rate to 80 Probability matrix Pattern to finesse entry . 1-2 pct at risk per trade. Plan and repeated perfect execution. Build neural pathways after 8-30 trades…….. William O’Neil How to make money in stocks……from The Psychology of trading and investing

Audio book: Trading For a Living, Psychology,Trading Tactics, Money Management

 

Jim Simons

Not trading information specifically but, he is a legend. Mathematician, code breaker  turned trading genius. He co-founded Renaissance Technologies in East Setauket, New York, a town away from where I grew up. Renaissance Technologies, Ren Tech, makes huge yearly gains in it’s Medallion Fund. The company does a lot of quant work and hires PhD scientists, mathematicians and etc. Working for this company would be amazing to say the least. So much to learn there working with the talented people he has helped to pick and shape over the years.

Renaissance’s flagship Medallion fund, which is run mostly for fund employees,[8] is famed for the best track record on Wall Street, returning more than 66 percent annualized before fees and 39 percent after fees over a 30-year span from 1988 to 2018.[9][10]

DMAC BTC Trading Example With Tuning

I got this example originally called simple-strat.py from a Datacamp article . I have my changes to the code up on Github.

Tunable DMAC

Basically the modifications to the original code incorporates a Dual Moving Average Crossover (DMAC) strategy that adjusts the averages to pick optimum values . DMAC is a very basic trend trading strategy. Other types of strategies would include, mean reversion, hedging such as pairs trading, arbitrage or buying and selling across the bid-ask spread as a market maker, so called earning the spread.

DMAC Basics

A dual moving crossover is exactly what the name says. There are two moving averages, one short and one long. When the short one rises above the long one, it is a buy signal. When they cross the other way, guess what, sell. This is almost about as dirt simple as a strategy can get. Single moving average, where price crosses one average would be the simplest moving average strategy. DMAC used in the example is not  fancy at all, no stops, no screening, no failure mode protections. I have found that about 10% of designing a good algorithmic strategy is in the strategy, another 10-20% is in the tuning and generalizing, backtesting. The rest is protecting the strategy from itself, such as screening out bad trades. The absolute worst is price shocks, which can’t be predicted.

Tuning

Initially it was coded to work with stocks but, I revised it to take in BTC instead. I also took it one level up and had it automatically tune for the best combinations of the short and long average for the DMAC stratagy. This is a bit better than sitting there and fiddling the parameters by hand. The pitfall is that the model can be overfit to the backtest data and then perform poorly on new data. It is best to have a model that generalizes a bit. Sometimes this can be accomplished by actually dumbing the process down a bit or using a stochastic approach. More parameters in the model the more likely overfit will occur as well.

Aside on Ensemble Backtesting

One approach that I typically use when tuning an algorithmic models parameters is to use an ensemble approach. I have used this successfully with genetic algos, such as differential evolution. I will have the backtest tuning take random slices of the time series from random points within the time series. From this I will run backtests on each one and then average the results together to form the ensemble. This helps to generalize but, won’t work for every kind of model. This has to do with the nature of local maxima and minima that the model might rise up  to or fall into. With some models averaging may just average to the plain and not the peaks and valleys. This has to be looked at carefully. But, I am getting off topic here.

 DMAC versus Buy and Hold

The idea here is to do some Monday morning quarterbacking. When I was in industry this was always a thing on Mondays. No matter what the sport, everyone threw around the coulda, woulda, shouldas. Probably more so if there was money riding on the game. Well if you are a trader or investor, there is real money riding on the game so sometimes it is worth looking back and seeing how a trading strategy would have worked against the classic long term buy and hold. a.k.a HODL…hold on for dear life.

If any of the terminology or details above seem fuzzy, check out the Datacamp article.

Fantasy Account

If we bought BTC on 1/1/2019 and sold on 12/31/2019 and had a magic wand to make the perfect DMAC tuned model, would we have done better than buy and hold?

We start with $100K on 1/1/2019 and cash out on 12/31. What do we have?

DMAC with a 29 day short and 59 day long average…

Short AVG,Long AVG,Best Bank 29 59 902908.9599609375

$900K, the clear winner.

Buy and Hold comes in at $182K

HODL Gains: 1.8258754145331861

Obviously, this is perfect trades, like a Carnot cycle engine without friction, it’s not realistic . In the real world there are fees, slippages, price shocks, picking bad parameters and other gotchas.

But, even if you threw a dart at pairs of numbers for the averages, at least for 2019, odds are the DMAC would have beat buy and hold for the year. This is not always the case, as in years past a buy and hold would have beat the DMAC. See the printout here for a 2010-2017 run.

BTC Price with Average and Buy and Sell points
2019 BTC Price with Average and Buy and Sell points Up Arrow = BUY, Down Arrow = SELL
Amount of cash made on paper
2019: Amount of cash made on paper

output from simple-strat-loop-backtest-2019.py

Get Data
Run Model
Short AVG,Long AVG,Best Bank 1 1 100000.0
Short AVG,Long AVG,Best Bank 1 2 611522.802734375
Short AVG,Long AVG,Best Bank 1 4 646585.4736328125
Short AVG,Long AVG,Best Bank 1 16 652523.6328125
Short AVG,Long AVG,Best Bank 1 17 689456.2744140625
Short AVG,Long AVG,Best Bank 3 89 690951.904296875
Short AVG,Long AVG,Best Bank 4 90 696133.9599609375
Short AVG,Long AVG,Best Bank 6 86 707596.8994140625
Short AVG,Long AVG,Best Bank 9 82 709683.1787109375
Short AVG,Long AVG,Best Bank 9 83 732913.4521484375
Short AVG,Long AVG,Best Bank 10 83 742904.150390625
Short AVG,Long AVG,Best Bank 13 17 746094.677734375
Short AVG,Long AVG,Best Bank 14 81 762164.5263671875
Short AVG,Long AVG,Best Bank 22 95 781311.2548828125
Short AVG,Long AVG,Best Bank 27 55 814494.62890625
Short AVG,Long AVG,Best Bank 29 56 820056.7626953125
Short AVG,Long AVG,Best Bank 29 57 886127.4658203125
Short AVG,Long AVG,Best Bank 29 59 902908.9599609375
signal   short_mavg    long_mavg  positions
Date
2019-01-01     0.0  3843.520020  3843.520020        NaN
2019-01-02     0.0  3893.464722  3893.464722        0.0
2019-01-03     0.0  3874.556885  3874.556885        0.0
2019-01-04     0.0  3870.347046  3870.347046        0.0
2019-01-05     0.0  3865.316553  3865.316553        0.0
2019-01-06     0.0  3900.535889  3900.535889        0.0
2019-01-07     0.0  3918.351946  3918.351946        0.0
2019-01-08     0.0  3932.413940  3932.413940        0.0
2019-01-09     0.0  3943.845323  3943.845323        0.0
2019-01-10     0.0  3917.353247  3917.353247        0.0
2019-01-11     0.0  3896.445268  3896.445268        0.0
2019-01-12     0.0  3876.849915  3876.849915        0.0
2019-01-13     0.0  3851.934777  3851.934777        0.0
2019-01-14     0.0  3841.514596  3841.514596        0.0
2019-01-15     0.0  3827.458643  3827.458643        0.0
2019-01-16     0.0  3816.680405  3816.680405        0.0
2019-01-17     0.0  3808.555908  3808.555908        0.0
2019-01-18     0.0  3800.182766  3800.182766        0.0
2019-01-19     0.0  3796.413587  3796.413587        0.0
2019-01-20     0.0  3786.643591  3786.643591        0.0
2019-01-21     0.0  3776.614490  3776.614490        0.0
2019-01-22     0.0  3768.794611  3768.794611        0.0
2019-01-23     0.0  3760.808891  3760.808891        0.0
2019-01-24     0.0  3754.144582  3754.144582        0.0
2019-01-25     0.0  3747.969434  3747.969434        0.0
2019-01-26     0.0  3742.372934  3742.372934        0.0
2019-01-27     0.0  3736.506004  3736.506004        0.0
2019-01-28     0.0  3727.004020  3727.004020        0.0
2019-01-29     0.0  3717.387224  3717.387224        0.0
2019-01-30     0.0  3705.065211  3709.680371        0.0
…            …          …          …        …
2019-12-03     0.0  8139.303778  8350.672074        0.0
2019-12-04     0.0  8067.217824  8338.195445        0.0
2019-12-05     0.0  8001.267056  8324.681624        0.0
2019-12-06     0.0  7941.937231  8313.125919        0.0
2019-12-07     0.0  7898.880573  8295.507233        0.0
2019-12-08     0.0  7855.803442  8278.183023        0.0
2019-12-09     0.0  7798.747340  8262.575270        0.0
2019-12-10     0.0  7747.724289  8244.635676        0.0
2019-12-11     0.0  7692.612742  8225.930954        0.0
2019-12-12     0.0  7638.642797  8206.752102        0.0
2019-12-13     0.0  7589.042447  8190.893042        0.0
2019-12-14     0.0  7541.893538  8175.251465        0.0
2019-12-15     0.0  7493.670814  8159.122492        0.0
2019-12-16     0.0  7436.929603  8141.483051        0.0
2019-12-17     0.0  7379.385776  8118.634824        0.0
2019-12-18     0.0  7347.339473  8102.613207        0.0
2019-12-19     0.0  7318.911065  8084.971233        0.0
2019-12-20     0.0  7304.292666  8070.405356        0.0
2019-12-21     0.0  7300.657530  8064.922082        0.0
2019-12-22     0.0  7304.581391  8065.228863        0.0
2019-12-23     0.0  7315.192130  8043.109003        0.0
2019-12-24     0.0  7321.274835  8010.525266        0.0
2019-12-25     0.0  7323.232927  7971.939519        0.0
2019-12-26     0.0  7313.139935  7937.750000        0.0
2019-12-27     0.0  7307.173811  7901.519506        0.0
2019-12-28     0.0  7291.889211  7869.523975        0.0
2019-12-29     0.0  7286.821037  7839.406482        0.0
2019-12-30     0.0  7282.293541  7806.048696        0.0
2019-12-31     0.0  7277.866329  7769.928041        0.0
2020-01-01     0.0  7273.729391  7735.433461        0.0

[366 rows x 4 columns]
BTC-USD  holdings      cash     total  returns
Date
2019-01-01      0.0       0.0  100000.0  100000.0      NaN
2019-01-02      0.0       0.0  100000.0  100000.0      0.0
2019-01-03      0.0       0.0  100000.0  100000.0      0.0
2019-01-04      0.0       0.0  100000.0  100000.0      0.0
2019-01-05      0.0       0.0  100000.0  100000.0      0.0
BTC-USD  holdings           cash          total  returns
Date
2019-12-28      0.0       0.0  902908.959961  902908.959961      0.0
2019-12-29      0.0       0.0  902908.959961  902908.959961      0.0
2019-12-30      0.0       0.0  902908.959961  902908.959961      0.0
2019-12-31      0.0       0.0  902908.959961  902908.959961      0.0
2020-01-01      0.0       0.0  902908.959961  902908.959961      0.0
Portfolio Plot
Sharpe Ratio 2.2316038135122973
Compound Annual Growth Rate (CAGR) 0.8258754145331861
HODL Gains: 1.8258754145331861

 

Maximum Drawdown for the Period 1/1/2019-12/31/2019
Maximum Drawdown for the Period 1/1/2019-12/31/2019

2018

Followup, I was curious about the rough 2018 year for BTC. 2018 was quite bearish, with only a few small rallies up. It had awful buy and hold performance.

HODL Gains: 0.2565408133534354

Short AVG,Long AVG,Best Bank 1 4 498855.46875

Buy and hold would have been a 4x loss and DMAC would have traded in and out quite often, racking up some fees and slippage of course but, made a 5X gain.

2018 BTC Price with Average and Buy and Sell points Up Arrow = BUY, Down Arrow = SELL
2018 BTC Price with Average and Buy and Sell points Up Arrow = BUY, Down Arrow = SELL
 2019: Amount of cash made on paper
2018: Amount of cash made on paper
2018 Drawdown
2018 Drawdown

2018 Results Details on Github

Github repo

https://github.com/erickclasen/DMAC-BTC-Trading-Example-With-Tuning

wave-gallery

How to Generate Art Demo Command Line Version

Red black circle
Red black circle Test

This post covers code that I have played with that applies a style to an image. The code is derived from Siraj Raval’s work. I was interested in having a version that would run outside of a Python Jupyter notebook. The idea is to have a version that can be run from the command line. This allows for it to be run ‘production’ style. This means have it set with images and parameters and run right from the command line. I found a way to convert the code from Jupyter notebook code to straight Python. The exact method I can’t remember right now. The code also produces intermediate frames while running, one per iteration. These are stored in the  iters directory. They can be used to monitor the models progress and optionally generate an animated GIF showing the unfolding of the process.

 

I have conda installed on my machine, so the dependency aka, requirements text file was used to load dependencies, following setup of the environment. This virtual environment makes sure that there are no collisions with other code environments, including the base. I did not want to upset anything that I depend on over an experiment. It can also be put under a virtual environment in Python. This is not a requirement but, good practice when developing code. It helps to keep the code in its own confined space and makes things work in a repeatable way. Making it easy to replicate the work of others in a controlled way. Sometimes it helps to be able to troubleshoot issues by switching between environments too.

Conda Environment Setup

The best resource for learning to work with conda is the cheat sheet for conda

I set mine up to work like this …

#
# To activate this environment, use
#
#     $ conda activate art-demo
#
# To deactivate an active environment, use
#
#     $ conda deactivate
Python Virtual environment example , setup and activate…
virtualenv -p python $HOME/tmp/art-demo-venv/
 cd How-to-Generate-Art-Demo/
 source $HOME/tmp/art-demo-venv/bin/activate

To setup dependency list

After setting up the optional environment, one of the above, install dependencies.

 pip install -r requirements.txt

 

Not sure about this, might have been in place already, the following that is…

If it doesn’t exist, create a file called ~/.keras/keras.json and make sure it looks like the following:

{
    "image_dim_ordering": "tf",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "tensorflow"
}

 

Then the demo code can be run from the command line.

demo-256.py 2019-05-23 16:40 16K
demo-512.py 2019-06-07 11:42 16K
demo-1024.py

Memory

Bear in mind that bigger resolution values will take longer than small. The memory requirements increase fast with resolution increases. I was only able to run 256 resolution on a machine with 4GB. I also tried to make thumbnail size images but, the model seems to have issues with smaller images, it produced distorted results. With a machine that has 16GB of RAM, it was easy to create 512 and 1024 pixel square images. When in doubt, monitor memory usage via the top command. The giveaway is swap gets used heavily with a large image. Even with a decent SSD and it having swap mounted on it, a machine might grind down to terrible lags in response, if swap is used dynamically on a heavy basis . Then the option is just to kill the process and make sure RAM is de allocated. This is really the only issue with running this code, memory use. If you have enough, it’s load up the dependencies and go.

Code

The code for this project is in this directory…

http://erick.heart-centered-living.org/upload/code/How-to-Generate-Art-Demo/

It is currently ‘live’. What I mean by that is that it is a symlink to the working directory on my main machine that is rsync’d ( rsync options -avhLK) to it’s location online. So, it is subject to change, you may see things get added in, I may break links accidentally too, hopefully not. If something is obviously broken, shout out in a comment and I can fix it. I might eventually load code to Github as a static repository.

Note the images directory. Also under it is the styles directory. I have added a few more examples beyond what Siraj Raval had. Mostly downloaded from 4chan while scrolling through wallpaper and some random images that were collected on my machine from wherever. Plus a few that I made for doing references against a simple image, such as a red dot , red and black dot and grid lines.

Plain is a directory that contains the original Siraj Raval code for reference.

Images and styles, contain a sampling of images to try. In the code the image location for initial image and style image are hard coded. Look around line 37 to line 52 for…

content_image_path

and

style_image_path

and you will see examples that can be switched in and out via mask commenting the lines out. Change it as needed.Plus try your own for fun.

Adjust parameters if needed. Look for them around line 152 under the line with …

# In[51]:

Also around line 305 is the iterations variable which can be adjusted.

There are some samples of output  that were generated in the top level directory. This is where the output is dumped as output.png. I just kept renaming the interesting images to useful names as I kept playing with various image and style combinations. You have to rename output.png as it will be overwritten when a new run occurs. It is possible to run rm ??output.png to delete them. It is possible to put this line in the script and activate. It is in the script but I have it commented out because I did not want to blast all of the output.png’s away when the gif is made.

iters directory contains animated gifs of the images and a BASH script that is used to make the animated gifs.

notes.txt was a sketch of of the names for the virtual environment and conda environment, setups on two different machines. I might add other info and issues I find in this file.

Example of styling an image

Real Person to style with anime
Real 3D Person 512×512 as input from images/anime
2D Anime person to style the image
Anime 2D Person for the styling image, from images/style/anime

To get my bearings straight on this code again, I ran an example to make sure I understand it, it still runs. Hopefully others can gain insight into what a proper run does.

Input Images

Above I have two images, one the initial image and the second the style. The initial and styling image are both cropped to be 512×512 pixels, re-scaling as needed. Re-scaling is not necessary, except for the fact that the program will re-scale and if the proportions are not square, the resultant image will be stretched or squished.

On the first one there was a bit of a haze of blue to the right that I bucket filled to white so it wouldn’t get styled as well. From extending the photo to a larger canvas to make square this blue haze was an obvious line of demarcation. Plus her shoulder was truncated, so I filled that in a bit to make things look better. The anime image for styling was just cropped square and re-scaled too and flipped. Not sure if it makes a difference but, I looked for images that were close-ish in look and pose and set the same orientation.

Memory Usage and Performance

My machine, an i5, Sandy Bridge OptiPlex-790, can be see taking around 500-700 seconds per epoch. It was also running other code at the time, tuning parameters for algo trading code, so all the cores were not focused on making the images. Plus, all the miscellaneous stuff running, mail, browser with too many tabs open, it is using 1/2 core just doing that. But, 20 loops through the code is not a long time, really. You can see below that a fair amount of memory is being used, including some swap. But, the machine is not slowing down at all. I think some of the swap is just used up on other things and may not have been de-allocated over time, so it’s more or less static swap now.

top - 11:49:25 up 118 days, 19:01, 13 users,  load average: 8.51, 8.57, 8.12
Tasks: 304 total,   3 running, 300 sleeping,   0 stopped,   1 zombie
%Cpu0  : 78.7 us,  9.3 sy,  0.0 ni, 12.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  : 73.0 us,  9.7 sy,  0.0 ni, 17.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  : 97.3 us,  2.7 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  : 80.9 us, 10.4 sy,  0.0 ni,  8.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 16309180 total,  6623860 free,  6149876 used,  3535444 buff/cache
KiB Swap: 16654332 total,  8917700 free,  7736632 used.  9330380 avail Mem

Iter Directory and Animated GIFs

While this code is running it will be copying png’s of the process into the iters directory. This lets you get a peek into the models progress and it is possible to use these later in conjunction with the make-gif-.sh in that directory to make an animated GIF of the model ‘building’ the final image. The copies starting from 00output.png and so on up to the last epoch number (The prefixing and order matter for the script to generate an animation) are a clue as to if things are going right. If after a few images, things look way off base or you question what direction the results are going, viewing the intermediate images give you a chance to abort early. There is a line in the file at the end that will optionally remove the output.png’s using rm ??output.png.

iterations of how to generate art demo
Iters Directory Showing Intermediate Copies

Command Line Output

From my command line this is what I see when it runs. Change to the directory, activate conda environment for the art-demo and fire off python demo-512.py. (I created another called demo-512-anime+human.py to replicate the example in this post ) Soon it spins up, well the fans on the machine do as well, Python grabs all the CPU power it can on all available cores so soon you will hear the power of machine learning as the fans kick to max RPM. Time goes by as the iterations of epochs run, 20 in this case. As long as the very high loss value is coming down, work is being done. As can be seen there is a point of diminishing returns beyond which not much is to be gained. This also can be assessed by looking at the progression of images in the iters directory. After it runs, it quits and a final output.png will appear in the top level, for me the ~/ml/How-to-Generate-Art-Demo/ directory, same directory the code is run from. Nothing special about ~/ml other than it is the place I keep all the machine learning code.

Note the warnings: I did not notice this right away but the Tensorflow install was not compiled to use SSE3,4.X and AVX. I retried a few loops of this example at 256×256 in my base install where Tensorflow seems to be compiled with these features and it yields 40% speed improvement.

(base) erick@OptiPlex-790 ~ $ cd ~/ml/How-to-Generate-Art-Demo/
(base) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo $ conda activate art-demo
(art-demo) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo $ python demo-512.py 
Using TensorFlow backend.
(1, 512, 512, 3)
(1, 512, 512, 3)
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
Start of iteration 0
Current loss value: 1.68822e+11
Iteration 0 completed in 511s
Start of iteration 1
Current loss value: 1.05267e+11
Iteration 1 completed in 480s
Start of iteration 2
Current loss value: 7.54607e+10
Iteration 2 completed in 459s
Start of iteration 3
Current loss value: 5.67734e+10
Iteration 3 completed in 489s
Start of iteration 4
Current loss value: 4.71848e+10
Iteration 4 completed in 518s
Start of iteration 5
Current loss value: 4.19099e+10
Iteration 5 completed in 513s
Start of iteration 6
Current loss value: 3.82277e+10
Iteration 6 completed in 533s
Start of iteration 7
Current loss value: 3.56423e+10
Iteration 7 completed in 508s
Start of iteration 8
Current loss value: 3.36591e+10
Iteration 8 completed in 498s
Start of iteration 9
Current loss value: 3.21125e+10
Iteration 9 completed in 441s
Start of iteration 10
Current loss value: 3.08934e+10
Iteration 10 completed in 610s
Start of iteration 11
Current loss value: 2.9882e+10
Iteration 11 completed in 516s
Start of iteration 12
Current loss value: 2.90331e+10
Iteration 12 completed in 495s
Start of iteration 13
Current loss value: 2.82984e+10
Iteration 13 completed in 499s
Start of iteration 14
Current loss value: 2.76652e+10
Iteration 14 completed in 495s
Start of iteration 15
Current loss value: 2.70985e+10
Iteration 15 completed in 530s
Start of iteration 16
Current loss value: 2.66107e+10
Iteration 16 completed in 86288s
Start of iteration 17
Current loss value: 2.61579e+10
Iteration 17 completed in 526s
Start of iteration 18
Current loss value: 2.57533e+10
Iteration 18 completed in 642s
Start of iteration 19
Current loss value: 2.53972e+10
Iteration 19 completed in 717s
(art-demo) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo $

The output file appears in the same directory

(art-demo) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo $ ls output.png 
output.png

Final Result

Final after 20 iterations Human + Anime (style image)
Final after 20 iterations Human + Anime (style image) = Humanime?

Animated GIF

To make an animated GIF from the output files in the iters directory. (Note, in Linux the program Convert must be installed if it is not already installed in the distribution that is being used). Go to the iters  directory, make sure that there are no extraneous output.png files from previous runs. This can happen if a previous run was made that was longer, more iterations and generated numbered outputs above the last iteration. Then run…

(art-demo) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo $ cd iters
(art-demo) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo/iters $

(art-demo) erick@OptiPlex-790 ~/ml/How-to-Generate-Art-Demo/iters $ ./make-gif.sh

In a few seconds a file called animation.gif will appear. Rename the file if you want to keep it as any future work may write over it.

The code is well commented and easy to tweak, thanks to Siraj Raval. I will end with one part of the conclusion , it’s appropriate as a final word.

# It's now your turn to play! Try changing the input images, their sizes, the weights of the different loss functions, the features used to construct them and enjoy different sorts of output. If you end up creating something you truly wish to share, [please do so](https://twitter.com/copingbear)!

Resources

Original work is on GitHub provided by Siraj Raval

How-To-Generate-Art-Demo

Video Tutorial that accompanies GitHub Code

Original Paper

https://arxiv.org/pdf/1508.06576.pdf