The page below generates random quotes every hour from a distil-GPT-2 model trained on a 8MB database of quotes.
http://erick.heart-centered-living.org/quotes/
To learn more see the following PDF, specifically Appendix 4 which is reproduced below, followed by the Python code that produces the quotes page.
Quotes Generator Info PDF: See Appendix 4
Appendix 4
The following was generated by training the distil-gpt2 model on the quotes csv file. In this case I used what I refer to as q-inference where the code creates 10 inferences and picks the best one. I plan on putting this code on GitHub when it is ready, TBD.
python3 q-inference.py quotes Using model path: quotes Prompt: The 0 -13.700541496276855 1 -26.85333251953125 2 -27.804710388183594 3 -31.28477668762207 4 -17.877981185913086 5 -23.032310485839844 6 -32.729827880859375 7 -24.180086135864258 8 -27.836820602416992 9 -24.09499740600586 The world is never divided into the bad and the good, while everyone in the rich is in the good. Thus, the good and the bad are always a combination. I think the idea that anyone can become the best doctor in the world is insane. The good news is, that Barack Obama, a brilliant political speechwriter, is less of an inspirational man than a charming, charming politician. I believe that being the best husband in the world is the greatest gift. An artist is never too old to paint what he does. The thing that is called an artist is to know the best of what he is doing. That is called an artist because he is always engaged in listening. The greatest gift is the gift. The best way to give is to make others to see who is giving. Be honest, give what you think. Be true to yourself, be good to yourself, make others know what you are. Be true to yourself, make others great to yourself, make others proud, make your own everything else look better, make yourself considerate, make everything else make what you are. Be strong to yourself, make what you are. Be true to yourself try to your best, make other people beautiful, make all else do it. Be true to yourself good to yourself, make others feel that you would be the best, make yourself so that you are. Be true to yourself a masterpiece. Time: 37.74
The model was trained for 51 epochs, 1 to test and see timing, 10 for another test overnight and finally 40 more to bring the loss down to under 2…
{‘loss’: 1.9966, ‘learning_rate’: 8.496176720475786e-08, ‘epoch’: 39.93}
The quotes are better than seen by using makemore. I am thinking some of this is part of the fact that it is a transfer learn situation rather than from scratch. I am also thinking it has partly to do with the fact that sub-word tokens are used versus character tokens. It is possible that the standard GPT2 learning gets a boost by ‘knowing’ parts of words already. I plan on exploring this a bit more, maybe by erasing the distil-gpt-2 model by training it on high entropy white noise type of data and training a character level GPT2 with the same parameters as the standard GPT2 via nanoGPT….TBD.
A few more that I cherry picked that were interesting….
The secret of being great is not finding a perfect balance between effort and failure.
The past is a metaphor for the future. We do not need to reconstruct our past.
The future is never a destination it is a destiny it is a vision, a way of living and dreaming.
You do not stop dreaming tomorrow because the past is only dream.
We are in the present moment.
Bad government is good for bad people.
Prompted with Happiness…
Happiness lies in the ways we seek, not in ourselves.
You must love yourself, not in everything.
Happiness lies in our way of knowing and loving ourselves.
Happiness lies in what is easy to find.
Happiness lies in the ways we must seek.
Happiness lies in our way of being.
Happiness lies in giving, not in our ways of being too selfish.
Happiness lies in the ways we find, not in our ways.
Happiness is not how much trouble we get, but in our way.
Happiness lies in giving.
Misato Katsuragi Quotes
Finally, try something not in the quote dataset, which does not contain the word Misato. A homage to Gwern Branwen’s TWDNE, which helped me get motivated again with ML projects, especially the quotes training as a warm-up for something bigger in the future.
Using model path: quotes
Prompt: Misato wrote this on the blackboard,
Misato wrote this on the blackboard, and I’d like to get behind computers and get behind the wheel of this car. I’m very proud.
I’m not afraid of computers, which I think they are the devil. But if they’re not devil, I can’t imagine they would be.
There are too many computers today that do not ask for much.
The internet is not just for those who’ve put up with their own little wheelie machines. They are all for those who’ve been using computers for a bit of creative fun.
The reason that most people don’t take computers off on computers before they set up a movie or internet is that they don’t take them out of their house when they go to bed or have tea in the morning. They don’t take their kids to school or work.
I don’t play Xbox. I can take care of my car and watch my candles and my candles, I’m not worried about computers at all. I’m just happy playing Apple TV.
I would love to come in and do anything on computers with my iPhone.
Internet research and other things. I’m not sure about computers. I’m into the surround myself with artificial intelligence, whether it’s a machine will ever be my main concern. The driving machine will be the computer and whether it is going to be the computer, when I can come in and going.
Time: 11.83
Python 2.7 Code that Generates HTML Quotes Page
Code pulls random line from quote file. Called from CRON output directed to index.html file in quotes directory. Why Python 2.7? This is all that the server supports right now.
#!/usr/bin/python
import random
def extract_line(file_path, line_number):
with open(file_path, 'r') as file:
lines = file.readlines()
if 1 <= line_number <= len(lines):
return lines[line_number - 1]
else:
return None
# Example usage:
file_path = 'cherry-pick-quotes-for-generator.txt'
# Pull a random line for a quote. Must have the right number of lines for file.
line_number = random.randint(1,34) # Replace with the desired line number
extracted_line = extract_line(file_path, line_number)
# Generate output HTML compliant.
print "<html>"
print "<head><title>GPT-2 Generator</title></head>"
print "<body>"
print "<H1>GPT-2 Generated Quote Number:",line_number,"</H1>"
# Sanity checking for the existance for line.
if extracted_line is not None:
print '<blockquote><font size="+2">',extracted_line,"</blockquote></font>"
else:
print("<P>Line",line_number," does not exist in the file.</P>")
print "<br><hr>"
print "<a href='http://erick.heart-centered-living.org/wp-content/uploads/2023/11/verbose-readme.pdf'>Quotes Generator Info PDF: See Appendix 4</a>"
print "</html></body>"