Daily Archives: March 29, 2019

The Meijin's Retirement Game 52

AlphaGo: Machine Learning and the game Go

This post is basically a list of good resources on AlphaGo and the game Go. There are many fine tutorials out there on the Internet that  I have read through to understand more about machine learning and about how AlphaGo functions. I have collected what are in my opinion some of the best out there and published them in this post.

Additionally there is (as of June 2018) an open source version of AlphaGoZero (The zero means it started from zero, as in  there was no priming with data from human played games, it is programmed with  the rules to Go and just plays against itself repeatedly) called LeelaZero which was built by following the paper published by DeepMind that covers the research and development of AlphaGoZero. It is a formidable player indeed, I can’t even get one point against the monster. As an experiment, I played it against GnuGo to see how GnuGo would fair. It still gets beat by LeelaZero  at a slower rate than I do but, is able to score some points against LeelaZero on occasion.

GnuGo versus LeelaZero

  Alpha Go how and why it works

The post by Tim Wheeler is hands down one of the clearest explanations I have seen. Tim Wheeler not only does a great job with this post, he has many other quality posts  on his site http://tim.hibal.org

tim.hibal.org/blog/alpha-zero-how-and-why-it-works/

While you are looking at Tim’s post consider viewing the Alpha Go Cheatsheet as well, keep them both open and flip between them, a great way to learn.

hi res AlphaGo Cheatsheet

https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png

Other resources

One Diagram AlphaGoZero

https://medium.com/applied-data-science/alphago-zero-explained-in-one-diagram-365f5abf67e0

The Wikipedia article on Monte Carlo Tree Search is worth a skim if you are not already familiar with Monte Carlo Tree Search which is used in game playing code, both machine learning driven game algorithms and what I would call pre-machine learning types. Previous to machine learning it was successful mostly for games that have a lower branching factor such as Chess. It is also used in GnuGo in a mode that plays on a smaller than standard board (9×9 and smaller). It is probably limited to a small board by the branching factor which gets huge as the board size increases. The number of board configurations is 3^n^2, n being intersections. A 9×9 board has 10^38 and a 19×19 10^170 legal positions according to a Wikipedia article that I read.

Background

For a good background and a brief history of machine learning, deep reinforcement learning in particular. Well worth the read…

Andrej Karpathy Deep Reinforcement Learning: Pong from Pixels

Hands on Exercise

There is an article on Medium that is worth a read, Teach a machine to learn Connect4 strategy through self-play and deep learning
Plus it lets you follow along and build the code to get a nice hands on experience.

LeelaZero: Basically an open source Alpha Go Zero and  uses a JAVA GUI (Lizzie) to play it

LeelaZero, the Go engine  is an easy to download and compile program, at least on Linux where I had it up and running in about 5 minutes. It uses a companion interface Lizzie written in Java for it’s GUI. LeelaZero is interesting and fun to try out. The pondering mode is cool. You hit the space bar and it ponders and shows the next moves probabilities of winning and depths of search. You can see from the 1.7%, it is beating me pretty bad after 70 moves as it is almost certain to win.

LeelaZero PonderingHovering the mouse pointer over a specific move shows projections of the next moves for both players labeled with numbers up to it’s maximum forward game play estimates.

LeelaZero Future Moves

 

Released Code for LeelaZero

https://github.com/featurecat/lizzie/releases/tag/0.6

https://github.com/leela-zero/leela-zero

Main LeelaZero Page

zero.sjeng.org/

An interesting discussion on LeelaZero

https://lifein19x19.com/forum/viewtopic.php?f=18&t=15631

  GnuGo in EMACS

GnuGo Can be played within Emacs, which is handy. This is what I did when I played GNUGo against LeelaZero. I forced LeelaZero to play black so it went first and then mirrored it’s move into the GnuGo board and GnuGo’s move back to the Lizzle/LeelaZero board. When I looked at the projects code it looks like development stopped around 2009. At the time it was a fairly good computer Go game but since then others have outpaced it’s strength. From what I recall it is in the 900 elo range for strength.

If you have Emacs installed the GUI version and GnuGo installed, then with Emacs open pressing Alt-X and entering gnugo in the Emacs command buffer will open GnuGo within Emacs. The benefits of this is that you can use the mouse or up and down arrows to play instead of entering coordinates at the command line.

www.gnu.org/software/gnugo/gnugo_3.html#SEC27

General Go Resources

https://en.wikipedia.org/wiki/Book:Go:_The_Board_Game

https://en.m.wikipedia.org/wiki/Computer_Go

https://en.m.wikipedia.org/wiki/Rules_of_Go

https://en.m.wikipedia.org/wiki/John_Horton_Conway