Tag Archives: artificial intelligence

Buddigar

Buddigar: Stable Confusion

Image Generation, Machine Learning, My Thoughts, , , , , , , , , , , , , , , ,

Humor is the weapon of self-control.

Budgierigar
Budgierigar

While playing around with Stable Diffusion, I realized that I must have misspelled “Budgerigar” and was surprised to see the monstrous images that it generated. Perhaps I accidentally typed “Buddigar” or some other variation, resulting in the strange images below. Some of them resemble monsters, while others appear to be stores from a distant land outside of the USA.

These images fall into the uncanny valley of creepiness. For instance, I heard on a Lex Fridman podcast that image generators sometimes struggle with creating realistic facial expressions. It’s fascinating to observe what Stable Diffusion produces when prompted to generate an image of a person with a surprised expression. Although the results can be unsettling and bizarre, they’re also quite amusing, like this one….

OMG, a Buddigar!!!!!
OMG,I spot a Buddigar!!!!!

Buddigars

The first one is my favorite in terms of shock and awe.

Buddigar: It's like Golem's Pet.
Buddigar: It’s like Golem’s Pet.

Buddigar tale by ChatGPT (mostly, just with a bit of prompting and edits)

There is a mischievous creature named Buddigar. It’s small and furry, with sharp claws and bright, beady eyes that seemed to sparkle with mischief. Buddigar loves nothing more than causing trouble, and it has a talent for doing so in the most unexpected ways.

If you can’t find your keys, Buddigar is probably to blame. It has a habit of sneaking into people’s pockets and snatching whatever shiny object catches its eye. It wasn’t that Buddigar has any use for these trinkets – it simply enjoys hoarding them, like a dragon with its treasure.

And if your computer is running slow, you could bet that Buddigar had something to do with it. It has a knack for hiding files, moving icons, and generally wreaking havoc on any device it comes across. Even the most tech-savvy of users are no match for Buddigar’s cunning ways.

But Buddigar’s mischief isn’t limited to material possessions. It has a strange talent for causing physical ailments, too. If you have a pain in your hip, for instance, you could be sure that Buddigar has something to do with it. It’s not that Buddigar means to hurt anyone – it simply enjoys the chaos that its actions cause.

The funny thing is, Buddigar doesn’t try to hide its misdeeds. When caught in the act, it simply grunts and declares its mischievous intent proudly. “Buddigar!” it exclaims, with a mischievous twinkle in its eye. “Buddigar take keys! Buddigar slow computer! Buddigar make hip hurt!” And then it scampers away, leaving behind a trail of chaos and confusion.

Despite its mischievous nature, however, Buddigar is impossible to hate. There is just something endearing about its playful antics, and the way it seemed to revel in the chaos it caused. Even Gollum, the famously twisted and corrupted creature from Lord of the Rings, would have adored Buddigar’s mischievous ways. It is so ugly it’s almost gone full circle to cute in an uncanny way.

And so, Buddigar continues to roam the land, causing trouble wherever it goes. Some curse its name, while others simply laugh and shake their heads in amusement. But love it or hate it, there was no denying that Buddigar is one of a kind – a little creature with  a talent for mischief that it’s unmatched by any other.

 

More Buddigars…

makemore

Makemore

Machine Learning, Text Generation, , , , , , , , , , , , , , , , , , ,

The best way to predict a future is to look for it. We live in the moment where the world is a vast machine for predicting the future.

This summer, I spent a significant amount of time contemplating large language models and delving deeper into their research. My first encounter with GPT-2 was back in 2019, where I explored its code and experimented with it. During this period, I became curious about transfer learning and its applications. Additionally, I had some prior knowledge about transformers, but it wasn’t as comprehensive as my understanding of LSTMs and RNNs. I couldn’t confidently explain what they did, for example.

While researching transfer learning with smaller models like GPT-2, I stumbled upon Gwern Branwen’s website (https://gwern.net/) and, in particular, his TWDNE Project (https://gwern.net/twdne). I found it clever because it combined a generative model for both images and text. I decided to focus on the text side of the project, as the image aspect was already well-addressed by applications like Stable Diffusion….

Misato Katsuragi as a Math Teacher
Misato Katsuragi as a Math Teacher

I might revisit the image style transfer aspect in the future, as I had previously explored it to some extent. You can find more about this in my “How to Generate Art Demo Followup.”

Before this, I had predominantly explored machine learning with code from the ground up using Python (PMLC). I have used ML practically in the form of genetic algorithms for tuning parameters on investing models for years, non-differentiable, so no chain rule! An offshoot was a project called gen-gen-algo, a generic genetic algorithm. Now, finally after all these side quests, I was ready to tackle something more complex and cutting-edge using GPT.

I found excellent resources on GitHub and in video format from Andrej Karpathy (https://github.com/karpathy). The following repositories were particularly helpful in my learning journey. The first one, “nn-zero-to-hero,” features a series of videos that provided a solid foundation in understanding transformers.

The second repository, “makemore,” served as my warm-up exercise to get back into working with transformers and Large Language Models (LLMs) after a period of dormancy in the field. You can access these repositories here:

1. “nn-zero-to-hero”: https://github.com/karpathy/nn-zero-to-hero
2. “makemore”: https://github.com/karpathy/makemore

Fork of makemore

My experience with “makemore” went beyond the basic examples provided in the original repository, which generated new names based on a dataset of names. Initially, my goal was to apply “makemore” to various datasets other than “names.txt.” I experimented with larger and smaller datasets, including those with extensive collections of English words, numbers for addition, square roots, and a substantial dataset of quotes containing nearly 10 million entries, some of which had lines as long as 505 characters. By using scripts and modifications to “makemore.py,” I conducted a grid search to optimize hyperparameters, including constraints on model size. Output from “makemore.py” was saved to a CSV file, along with hexadecimal hash values for easy tracking and analysis during the tuning process.

To further enhance the code, I introduced a grid search optimization method using a Bash script. This allowed for exploring the hyperparameter space while maintaining a ceiling on the model size. Without such constraints, optimization typically led to increasingly larger models that resulted in the lowest loss.

I also introduced the concept of assigning a random hexadecimal tag to the output of “makemore.py.” This tagging system facilitated the easy identification of the best loss and the associated set of hyperparameters that produced it. Additionally, I incorporated an early stopping mechanism into the “makemore.py” code.

If you’re interested in exploring my fork of Andrej Karpathy’s “makemore” code, you can find it here:

https://github.com/erickclasen/makemore

For a more detailed understanding, I’ve created a comprehensive “verbose-readme.pdf” that provides additional information:

Version on this site, opens in browser:

verbose-readme

GitHub Version requires downloading:

https://github.com/erickclasen/makemore/blob/master/verbose-readme.pdf