Hacker Cat

Using AWK to pull data from a field within a text file

Intelligence is the power which gives us the ability to distinguish when we are not conscious.

# awk -F” ” ‘{print $11}’ pressure-delta.txt
tail -1 $PLOTTABLE_DIR/$PRESSURE_DELTA_TAIL_FILE > $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt

# Use awk with -F” “, which uses space as field seperator.
# pressure delta in inHg is in the 11th field.
# Multiply by 1000 because bash does not do floating point math natively and we want to compare so use milli-inHg to do the bash math.
# Operate on the file $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt

pressure_delta=`awk -F” ” ‘{print $11*1000}’ $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt`

TMDNE

TMDNE: This Mathematician Does Not Exist

It’s like the math is the seeds or DNA and the code is the fertile ground and the sun shines, or in our case electricity flows to CPUs and GPUs.

Ever pondered the possibilities when algorithms dabble in the divine art of mathematics? Prepare yourselves for TMDNE – where Stable Diffusion meets the blackboard of the mind! 🧮🤯

Inspired by the brilliance of projects like “This Waifu Does Not Exist” (TWDNE) by Gwern Branwen, TMDNE takes a quantum leap into the realm of imaginary mathematicians. These aren’t your conventional number crunchers; they’re the brainchildren of algorithms, immersed in the ethereal dance of imaginary theorems and proofs. 🎨➗

Much like TWDNE introduced us to AI-generated characters that captured our otaku hearts, TMDNE invites you to witness the magic of machine learning as it conjures up mathematical maestros who never attended a single conference. 🤓📚

Get ready for a symphony of computational creativity, where algorithms showcase their virtuosity in crafting mathematicians who not only ponder the mysteries of prime numbers but might just solve the Riemann Hypothesis over a cup of virtual coffee. ☕📏

Embark on this intellectual odyssey where Stable Diffusion meets the abstract canvas of mathematical imagination, proving that in the world of artificial intelligence, even mathematicians can be a product of algorithmic brilliance! 🔢🤖

 

Grid Search in Stable Diffusion

The images in the following video were created by doing a grid search with Stable Diffusion, with slight variations of words in the prompt and scanning the guidance scale. The seed value remained constant. The images were also audited manually for obvious artifacts and flaws.

Finally a slideshow was created by using the following ffmpeg command:

ffmpeg -framerate 1/3 -pattern_type glob -i "image\([0-9]*\).png" -c:v libx264 -r 30 -pix_fmt yuv420p slideshow.mp4

This results in a slide show with a change every three seconds on the images.

The Idea

Math -> AI -> Imaginary Mathematicians

OK, AI is math really, a bunch of chain rule linkage and matrix multiplies for the most part and to make it do something useful the math gets worked into code as the substrate. It’s like the math is the seeds or DNA and the code is the fertile ground and the sun shines, or in our case electricity flows to CPUs and GPUs. In 2024, it’s gotten really good. imagine generation that is. I did some AI image modification years ago but, Stable Diffusion has gone way far, fast. So wouldn’t it be interesting and humorous to generate images of math being worked on, by a mathematician generated by AI. As can be seen from the images, this points out a weakness too. People, it kind of nails it, backgrounds too. Text and numbers, not so much. I’ll be interested in seeing it nail this too, only a matter of time.

Grid Search & Ground Rules

As an AI researcher, I am always interested in pushing the boundaries and grid search is something that is a normal event for me. I’ve played with Stable Diffusion for quite a while and I was interested in seeing what a semi-serious grid search would reveal now that it is seriously capable of generating graphics. I decided to stick with images that are not ‘fantasy’ or too enhanced to keep them real looking so that they are easier to judge.Keeping the ‘look’ in the same zone also helped keep some degree of uniformity but, Stable Diffusion varies the image within a range that seems to work well.

Yet another sidequest

This started as a bit of a sidequest as I am working on a project called TinyMath that is all about identifying how small a GPT model can get and get a passing grade for math. This way it is possible to score objectively and see how model size and loss stack up against the ground truth of taking an ‘exam’. I just happened to create an image using Stable Diffusion of Misato Katsuragi from Neon Genesis Evangelion at a blackboard working out some math when I was pondering this, kind of a nod to TWDNE on a document I wrote on makemore. That combined with an image that I saw in 2020 when my partner Renee was searching for a new hair style, was another inspiration. The image below only shows a partial view of the woman and I often wondered if the other views were out there somewhere, or could AI come close to generating the other views.

The Two Images that got TMDNE going

Misato Katsuragi as a Math Teacher
Misato Katsuragi as a Math Teacher
Layered Stacked Bob for thick hair. Great style and color.
Layered Stacked Bob for thick hair. Great style and color.

 

Notes & References

TWDNE

“This Waifu Does Not Exist” (TWDNE) is a project created by Gwern Branwen. It involves the generation of images of fictional characters using Generative Adversarial Networks (GANs). GANs are a type of artificial intelligence algorithm that can generate realistic-looking images based on patterns learned from a dataset.

In the case of TWDNE, the GANs are trained on a dataset of anime-style characters, and the output is entirely computer-generated images of characters that do not exist in reality. The term “waifu” is a romanization of the Japanese word for “wife” and is often used in fandom to refer to fictional characters, especially those from anime and manga, for whom someone has a strong emotional attachment.

It’s worth noting that the images generated by TWDNE are not real individuals, but rather creations of the algorithm based on the patterns it learned during training. The project is an interesting example of the capabilities of AI in generating realistic-looking content.

Prompts, Seed and Range for Guidance

Styles Tried:

sai-photograpic
None
cinematic-default

Seed: 52794693

Positive Prompts:
young woman with short red hair in graduated bob style teaching math at a blackboard

young woman with short red hair in posh bob style teaching math at a blackboard

young woman with short red hair in victoria beckham posh bob style teaching math at a blackboard

young woman with short red hair stacked  with very short stacked layers in the back   teaching math at a blackboard

young woman with short red hair stacked  with very short stacked layers in the back   teaching math at a blackboard

Negative Prompt:
bangs curls

…. bangs were sometimes screwed up and curls caused artifacts, so these were filtered on.

Guidance range: 5-18.5 ( found to be best)

Additionally, the results were judged by looking at the images for excess artifacts or distortions such as missing fingers, awkward postures and anything else unrealistic beyond the fact that Stable Diffusion is still limited in terms of representing numbers and letters.

 

 

Buddigar

Buddigar: Stable Confusion

Humor is the weapon of self-control.

Budgierigar
Budgierigar

While playing around with Stable Diffusion, I realized that I must have misspelled “Budgerigar” and was surprised to see the monstrous images that it generated. Perhaps I accidentally typed “Buddigar” or some other variation, resulting in the strange images below. Some of them resemble monsters, while others appear to be stores from a distant land outside of the USA.

These images fall into the uncanny valley of creepiness. For instance, I heard on a Lex Fridman podcast that image generators sometimes struggle with creating realistic facial expressions. It’s fascinating to observe what Stable Diffusion produces when prompted to generate an image of a person with a surprised expression. Although the results can be unsettling and bizarre, they’re also quite amusing, like this one….

OMG, a Buddigar!!!!!
OMG,I spot a Buddigar!!!!!

Buddigars

The first one is my favorite in terms of shock and awe.

Buddigar: It's like Golem's Pet.
Buddigar: It’s like Golem’s Pet.

Buddigar tale by ChatGPT (mostly, just with a bit of prompting and edits)

There is a mischievous creature named Buddigar. It’s small and furry, with sharp claws and bright, beady eyes that seemed to sparkle with mischief. Buddigar loves nothing more than causing trouble, and it has a talent for doing so in the most unexpected ways.

If you can’t find your keys, Buddigar is probably to blame. It has a habit of sneaking into people’s pockets and snatching whatever shiny object catches its eye. It wasn’t that Buddigar has any use for these trinkets – it simply enjoys hoarding them, like a dragon with its treasure.

And if your computer is running slow, you could bet that Buddigar had something to do with it. It has a knack for hiding files, moving icons, and generally wreaking havoc on any device it comes across. Even the most tech-savvy of users are no match for Buddigar’s cunning ways.

But Buddigar’s mischief isn’t limited to material possessions. It has a strange talent for causing physical ailments, too. If you have a pain in your hip, for instance, you could be sure that Buddigar has something to do with it. It’s not that Buddigar means to hurt anyone – it simply enjoys the chaos that its actions cause.

The funny thing is, Buddigar doesn’t try to hide its misdeeds. When caught in the act, it simply grunts and declares its mischievous intent proudly. “Buddigar!” it exclaims, with a mischievous twinkle in its eye. “Buddigar take keys! Buddigar slow computer! Buddigar make hip hurt!” And then it scampers away, leaving behind a trail of chaos and confusion.

Despite its mischievous nature, however, Buddigar is impossible to hate. There is just something endearing about its playful antics, and the way it seemed to revel in the chaos it caused. Even Gollum, the famously twisted and corrupted creature from Lord of the Rings, would have adored Buddigar’s mischievous ways. It is so ugly it’s almost gone full circle to cute in an uncanny way.

And so, Buddigar continues to roam the land, causing trouble wherever it goes. Some curse its name, while others simply laugh and shake their heads in amusement. But love it or hate it, there was no denying that Buddigar is one of a kind – a little creature with  a talent for mischief that it’s unmatched by any other.

 

More Buddigars…

Monetary System Escape Hatch

MEH: Money Escape Hatch

 Money is power – your money, the power to spend.

An escape hatch from the Monetary System

Recently I was the victim of several types of fraud through the legacy monetary system, banking counter-party risk basically.

  • Debit card fraud: A transaction that I did not make showed up on my account. Required getting a new card and the bank did take care of the fraudulent charges.
  • A fraud about debit card fraud: A fraudster called me, preceded by a text that told me of a suspicious card transaction, spoofing the banks own phone number no less. They tried to get me to give up my credentials over the phone and then pretended there were several checks written against my checking account. They tried to run me through the process of setting up Zelle and having me send money to a weird email address. I just played along and acted like I didn’t understand how to use the Internet until the call dead ended.
  • Law firm fraud: A law firm, a lawsuit mill, files a fraudulent suit in the wrong venue using a fake address, so I get sued without my knowledge, resulting in a default judgment. I caught this in time before they were able to execute the judgment, or else they could have performed a money grab out of the bank account, without my knowledge. I informed the court of the fraud, the lawfirm, the Consumer Finance Protection Bureau  CFPB and all other parties that had a hand in this. This was a shocker as I didn’t even realize that his type of fraud existed.

Thoughts

After all this I was thinking about what could be done to lock up money, away from the reach of fraudsters as there is counterparty risk to a lot of things money related, especially in the legacy banking world. Some banks don’t even offer good security, like 2FA via a security key only and NOT 2FA texting to a mobile number that can be spoofed or email, equally weak.

Self custody of Bitcoin seems like the most solid way in light of the weaknesses in the monetary system. Why Bitcoin? Other tokens have less utility, why do we need 1000s of them, and some might be counted as securities someday and some have more inflation as  tokens are minted at the whim of the founders or core team and so on.

But, there are caveats:

  • You need to know what you are doing or else you can blow you cover and compromise your keys and therefore coins and also be too ‘public’ with transactions.
  • Corollary: Not your keys, not your coins!
  • Don’t trust, verify
  • Bitcoin is pseudo anonymous. You are sitting behind a public key and if you use a limited set of keys repeatedly it is possible to trace, via history on the blockchain, who you are and how many sats you have stacked. This is particularly true of an address used, lets say as a donation or payment address which is static. Ideally, you want dynamic addresses. Revealing too much information like this could make you a magnet for fraudsters that decide it is worth trying to, let’s say hack your warm wallets by injecting malicious code on your phone or PC, via an email with a cat video. Or spoof your phone number to bypass 2FA and so on. Heck, they might even try to track down where you live via social media and park in front of your house and break into the WiFi by stepping in on the four way handshake used to secure it, then they are on your home network and have access to every device directly!
  • Always visually verify addresses you send to. Just in case some copy/paste virus gets in the middle and changes the address.

 

Suggestions for caveats:

  • Your keys = Your Coins: Store the bulk of your stack on a cold wallet or paper wallet, in a safe place. Seed phrases as well. Keep only a small amount on a hot wallet, like an app or web wallet, exchange,etc.
  • Stay Private: Use methods to conceal the path of transactions by breaking the address linkage, effectively creating dynamic addresses. Porting through privacy coins comes to mind here and for a BTC only solution, Wasabi wallet, linked to a cold wallet such as Coldcard, BTC only wallet that can be air gapped.
  • Equipment SEC: Air gapped wallet allows you to use something like an SD card to move a partially signed transaction to the cold wallet to sign and back to a hot wallet that is watch only, so it can only take in but, not spend BTC. You can only spend by creating a partially signed transaction and moving it by hand using SD card to cold wallet, signing and moving it back via SD card. Sounds complicated but, is secure. No compromise and follows the verify and don’t  trust the hot wallet sitting on the phone or PC connected 24/7 to the Internet, making it only medium secure. Keep phones, PCs and Wifi secure, good passwords/biometrics and keep thinking through the holes lurking in security.
  • Dumb Human Things: Verify addresses when sending, use excellent passwords and PINs, read the instructions on equipment, like wallets, seedphrase security. Don’t get conned, don’t brag, don’t accidentally dox yourself. People are always inventing new ways to screw things up, so even with the best technology and encryption, mistakes happen, look on the Internet for more examples.

 

Idea on using Wasabi Wallet to enhance privacy along with cold storage

You can use Wasabi Wallet to enhance the privacy of your coins before transferring them to a cold wallet. Here’s a step-by-step process:

  1. Transfer Funds to Wasabi Wallet:
    • Transfer your funds from the exchange or other warm wallets to your Wasabi Wallet. This can be done by sending the funds to an address generated by your Wasabi Wallet.
  2. Initiate CoinJoin Transaction in Wasabi:
    • After receiving the funds in your Wasabi Wallet, initiate a CoinJoin transaction within Wasabi. This process will combine your transaction with those of other users, significantly enhancing privacy.
  3. Wait for Confirmation:
    • After initiating the CoinJoin, wait for the transaction to be confirmed on the Bitcoin network. This may take some time, as it depends on network congestion and the number of confirmations required.
  4. Send Funds to Cold Wallet:
    • Once the CoinJoin transaction is confirmed, you can safely send the funds from your Wasabi Wallet to your cold wallet. This step ensures that the funds you send to the cold wallet have undergone the privacy-enhancing CoinJoin process.
  5. Consider Multiple Rounds of CoinJoin:
    • For additional privacy, you may consider repeating the CoinJoin process with the funds in your Wasabi Wallet before sending them to the cold wallet. This can be done by initiating another CoinJoin transaction within Wasabi.

Remember, while this process can significantly enhance privacy, it doesn’t provide absolute anonymity. Also, the privacy features depend on the number of participants in the CoinJoin process, so it’s beneficial if more users are actively participating in CoinJoin transactions.

Always stay informed about the latest features and best practices in using Wasabi Wallet, as the specifics of the wallet’s functionality may evolve over time. Additionally, consider the transaction fees and potential delays associated with the CoinJoin process and Bitcoin network confirmations.

Alternative Idea Using Monero Swap

Just an idea that I was thinking of, not sure if it would be as good as the solution above using Coinjoin. But, the idea is to take some kind of coin BTC, USDC, whatever that you on ramped from USD via an exchange. Use some kind of swap, like SimpleSwap or the swap feature of a wallet such as Exodus and swap the non private coins from an exchange into XMR on a wallet, then swap to something like BTC on the cold wallet. When spending, run backwards, swap to XMR, then to the crypto of your choice and spend

When you convert BTC to XMR, the transaction history of the BTC is effectively broken, as the privacy features of Monero make it difficult to trace the source of funds. However, when you swap back to BTC, the privacy features of Monero may not be as effective, and your transactions could potentially be traced from that point onward.

It’s essential to note that while Monero provides strong privacy features, the overall privacy of any cryptocurrency transaction depends on various factors, including the platforms and services used for the swaps. Additionally, the regulatory environment surrounding cryptocurrency exchanges and transactions may impact the level of privacy you can achieve.

Word-cloud-for-post-on-large-price-moves

Thoughts on the best BTC price moves

As the great and powerful Satoshi Nakamoto once said, “If you don’t believe me or don’t get it, I don’t have time to try to convince you, sorry.”

For a while I have used a piece of code that started as back-testing code and forked it to produce output whenever BTC-USD makes a 2 standard deviation move in price up or down. Basically I manually adjusted the threshold that is normally a fluid value in backtesting to a fixed value, so that 95% of the time a U for Up, or D for Down move is not recorded, It’s counted as O, which means nothing interesting happened and is not printed. CRON runs this hourly and the machine responsible for trading runs 24/7 and collects prices, since August 28,2018. It even plays a sound when one of these outlier moves occurs.

Here are a few lines from the end of the output….

2024-01-03 16:00:02.618000       63874 42197.47 43004.35 U 2 1.02
2024-01-03 19:00:03.434000       63877 42923.71 42269.13 D 2 0.98
O: 43795  95.0%
D: 1165  2.5%
U: 1148  2.5%

The top two lines have a time stamp followed by a tick count, this is where it is in the data set, in order, adding 1 per hour. Next are the two prices, the starting and ending prices, U or D, signalling which way it just went. The next number shows how many hours have elapsed since the last radical move followed by the scale of the move. This is a rounded number that shows what number would be required to multiply the first price to get the current price, how big a move in other words.

2 Standard Deviations for Bitcoin

Daily

I recently adjusted the calibration threshold as BTC is settling down into less radical swings as time goes on. Currently it is….

THRESHOLD = 0.01459 # 0.0156

…which means on an hourly basis a 1.459% move registers as a 2 standard deviation move. It was 1.56% earlier this year, so volatility is trending down over time.

Can that be tradable, not likely as fees and slippage would most likely eat up the 1.459%, granted some higher swings do occur and I have pondered this. What is you bought all the big down moves and sold on the big up moves. Sounds good in theory and would most likely only work in an up market or else, you just wind up buying at bad entries and ‘bag hold’ too much at a loss until the next bull run.

Weekly

I got the idea to look at a daily and weekly time frame. Weekly is such a short output that I will paste it here….

Start of Data: 2018-09-05
Lines of input data: 279
Lines to process: 279
2018-11-21 05:00:11.243000       17809 6268.44 4452.68 D 10 0.71
2019-04-03 04:00:15.519000       17828 3976.88 5050.01 U 18 1.27
2019-05-15 04:00:15.266000       17834 5782.6 8063.78 U 5 1.39
2019-06-26 04:00:16.011000       17840 9139.01 12228.39 U 5 1.34
2019-07-17 04:00:14.972000       17843 12983.74 9411.06 D 2 0.72
2020-03-15 04:00:05.033000       17878 8749.01 5168.75 D 34 0.59
2020-08-02 04:00:06.489000       17898 9686.6 12000.0 U 19 1.24
2020-12-20 05:00:12.005000       17918 18865.0 23460.71 U 19 1.24
2021-01-03 05:00:14.866000       17920 26732.29 33231.68 U 1 1.24
2021-02-14 05:00:19.359000       17926 38365.06 47607.09 U 5 1.24
2021-03-14 05:00:09.349000       17930 49437.95 61191.88 U 3 1.24
2021-08-01 04:00:03.128000       17950 34192.29 42461.2 U 19 1.24
2022-06-19 04:00:02.056000       17996 27515.55 18282.06 D 45 0.66
2023-03-19 04:00:02.137000       18035 20598.23 27282.54 U 38 1.32
D: 4  1.4%
O: 264  95.0%
U: 10  3.6%

A few things to note:

  • There is asymmetry as the market has in general been rising since 2018, so there are only 4 ‘large’ down moves and 10 up. 1.4% weeks are up moves, 3.6% are down. The number after U and D is the weeks between moves.
  • Note the magnitude of the moves. They are truly massive. As the threshold is now 0.235.
  • Note that the 5 sequential up moves in a row 2020-2021 barely made in over the threshold at 1.24
  • Make note of the fact that Bitcoin is calming down over the years by looking at how many times per year it makes these epic moves.
  • Last but not least. I wonder if the what the linear regression of the prices would tell. Perhaps good targets for the future?
Large Weekly BTC Price Moves
Large Weekly BTC Price Moves

From now on, I will monitor this more carefully. As one can see, entry and exit although infrequent would have lead to stellar performance. I’ve been doing crypto TA and writing related code for almost 6 years now and it is still interesting that some new price relationships can be sussed out.

Looking ahead

It’s a guess buy based on the chart above it will be worth seeing if there is a pullback below the price of the blue linear regression termination, 37K-ish down to the termination of the D and E price lines 20598.23 27282.54. It will be interesting to followup on this.

Daily

For daily the THRESHOLD = 0.076, so that means greater that 7.6% moves are counted. Below I have pasted in the 2022 to 2023 results. 2022 being terrible and 2023 awesome. I let the reader ponder the readability of these price moves.

 

2022-02-05 05:00:03.175000       19053 37326.16 41505.25 U 62 1.11
2022-02-24 05:00:04.472000       19072 38042.82 35122.81 D 18 0.92
2022-02-25 05:00:04.643000       19073 35122.81 38740.11 U 0 1.1
2022-03-01 05:00:06.659000       19077 37830.34 43388.77 U 3 1.15
2022-03-09 05:00:02.706000       19085 38554.62 41656.28 U 7 1.08
2022-05-06 04:00:02.780000       19143 39727.55 36446.18 D 57 0.92
2022-05-10 04:00:02.150000       19147 33556.09 30678.51 D 3 0.91
2022-05-12 04:00:02.124000       19149 31191.22 28234.93 D 1 0.91
2022-05-13 04:00:02.622000       19150 28234.93 30523.85 U 0 1.08
2022-06-14 04:00:02.293000       19182 25483.35 22074.01 D 31 0.87
2022-06-17 04:00:02.808000       19185 22182.06 20359.29 D 2 0.92
2022-06-19 04:00:02.056000       19187 20424.2 18282.06 D 1 0.9
2022-06-20 04:00:02.514000       19188 18282.06 19961.12 U 0 1.09
2022-07-08 04:00:02.559000       19206 20465.35 22125.65 U 17 1.08
2022-07-28 04:00:01.952000       19226 21076.25 23137.12 U 19 1.1
2022-09-14 04:00:02.511000       19274 22235.95 20324.97 D 47 0.91
2022-11-10 05:00:02.217000       19331 18242.1 16433.61 D 56 0.9
2023-01-14 05:00:02.777000       19396 18820.73 20901.76 U 64 1.11
2023-01-21 05:00:02.053000       19403 20991.31 22602.95 U 6 1.08
2023-02-16 05:00:02.079000       19429 22092.46 24634.33 U 25 1.12
2023-03-10 05:00:02.124000       19451 21737.7 19877.69 D 21 0.91
2023-03-13 04:00:01.980000       19454 20598.23 22326.45 U 2 1.08
2023-03-14 04:00:02.715000       19455 22326.45 24517.05 U 0 1.1
2023-08-18 04:00:02.811000       19612 28619.6 26395.38 D 156 0.92
2023-10-24 03:00:02.742000       19679 30389.89 34553.97 U 66 1.14
D: 46  2.4%
O: 1856  95.0%
U: 51  2.6%

 

Looking at the plot for daily and remember this does not mean the X axis is time, just moves, even though it does look similar to a price -vs- time chart. In this case both regression lines overlap 100%.

 

Large BTC Daily Price Moves
Large BTC Daily Price Moves

Looking ahead

This is a guess but looking at this I am inclined to think a pullback between the regression line termination at 40K and the termination of the D and E price lines at 30 and 35K would be something to keep and eye out for.

Alternative Daily Move Chart

Taking the geometric mean of the daily and weekly thresholds, the result is 0.133. This produces using a daily tick an output that has as many large price moves as the weekly chart and can be seen as an alternative while slightly different, lines up similarly on the chart. The moves are roughly 3 standard deviation moves in this case.

Start of Data: 2018-08-30
Lines of input data: 1955
Lines to process: 1955
2018-11-20 05:00:17.949000       17880 5495.99 4641.55 D 81 0.84
2018-11-25 05:00:13.877000       17885 4301.01 3632.02 D 4 0.84
2019-04-03 04:00:15.519000       18014 4177.0 5050.01 U 128 1.21
2019-05-14 04:00:15.992000       18055 7016.99 7963.14 U 40 1.13
2019-06-28 04:00:15.313000       18100 12821.64 11110.99 D 44 0.87
2019-10-26 04:00:17.489000       18220 7455.01 9599.17 U 119 1.29
2020-03-13 04:00:04.431000       18359 7659.09 5385.44 D 138 0.7
2020-03-20 04:00:04.240000       18366 5301.62 6163.32 U 6 1.16
2020-04-30 04:00:04.743000       18407 7821.73 9291.9 U 40 1.19
2021-01-06 05:00:08.853000       18658 30880.51 35456.93 U 250 1.15
2021-02-09 05:00:09.512000       18692 38618.01 46888.08 U 33 1.21
2022-03-01 05:00:06.659000       19077 37830.34 43388.77 U 384 1.15
2022-06-14 04:00:02.293000       19182 25483.35 22074.01 D 104 0.87
2023-10-24 03:00:02.742000       19679 30389.89 34553.97 U 496 1.14
U: 9  0.5%
O: 1940  99.3%
D: 5  0.3%

 

Large-BTC-Daily-Moves-w-Linear-Regression-0.133
Large-BTC-Daily-Moves-w-Linear-Regression-0.133

 

Code

Code is on Github, it is quite a hack as it was made from pieces of other backtest code and production code that has not been cleaned or refactored.

https://github.com/erickclasen/cbpro-cli-tools/blob/main/uod.py

Misato Katsuragi

Kelly Criterion

If money is your bread then nothing else matters.

The Kelly Criterion is something anyone who trades, invests or even gambles must know. I have a brief summary below and some additional resources at the end of the summary.

There is also a great book devoted to this topic, Fortune’s Formula by William Poundstone, a must have book for any serious investor.

The Kelly Criterion, also known as the Kelly strategy, Kelly formula, or Kelly bet, is a mathematical formula designed to help people determine the optimal size of a series of bets. It’s named after John L. Kelly, Jr., who introduced the concept in a 1956 paper titled “A New Interpretation of Information Rate.”

The Kelly Criterion is commonly used in gambling, investing, and other areas where decisions involve uncertainty and risk. The goal of the Kelly Criterion is to maximize the long-term growth of capital by finding the optimal fraction of capital to invest in each opportunity.

The basic formula for the Kelly Criterion is:

Kelly-Critierion
Kelly-Critierion

The formula tells you what percentage of your current capital should be invested in a particular opportunity, given your assessment of the probability of success and the odds being offered.

It’s important to note a couple of things about the Kelly Criterion:

1. Risk of Ruin: The Kelly Criterion can be aggressive. Betting too much of your capital, even with a positive expectation, can lead to significant losses. The criterion doesn’t consider the risk of ruin, which is the risk of losing your entire capital.

2. Estimates are Crucial: The accuracy of the criterion depends heavily on the accuracy of your probability estimates. If your estimates are inaccurate, the strategy might not perform well.

3. Logarithmic Utility: The formula assumes logarithmic utility, meaning that the investor’s goal is to maximize the expected logarithm of wealth. This utility function helps to balance between risk and reward.

To use the Kelly Criterion, you would apply the formula to each opportunity, and the resulting fraction represents the proportion of your current capital that you should invest.

Keep in mind that many variations and adaptations of the Kelly Criterion exist to address specific situations and concerns. It’s a tool that requires careful consideration and understanding of the underlying assumptions and risks.

 

To learn more, see these resources

https://www3.cs.stonybrook.edu/~skiena/691/2007/lectures/Kelly.pdf

http://r6.ca/blog/20070816T193609Z.html

https://www.economist.com/media/globalexecutive/fortunes_formula_e.pdf

https://towardsdatascience.com/python-risk-management-kelly-criterion-526e8fb6d6fd

Writing

ChatGPT at it’s finest: writing short helpful code

ChatGPT at it’s finest: writing short helpful code without having to do fancy work.

The following is a dialog with ChatGPT. I had this script that I would normally have to hand edit all of the ticker  symbols by adding them to the end of the lines of code. I am sure there is some clever way using sed or awk but, I figured why not get ChatGPT to help.

ChatGPT then was also able to generate the Tags for this post,by feeding the post into a prompt and I used Stable Diffusion to generate the featured image using the blog title as the prompt. Sometimes it is good to use ChatGPT to proofread as well but, this post was short and simple enough not to do that.


User

I want to show you an output, in the left column are the ticker symbols for a number of cryptocurrencies, I want you to remember all of them as they will be used in the next question that I will pose…..

Currency  Price      Available  Hold      Avail in USD  Held in USD
AAVE 98.16          0.0 0.0              0.0
ACH 0.020353          0.0 0.0              0.0
ADA 0.3859          0.0 0.0              0.0
ALGO 0.1341          0.0 0.0              0.0
AMP 0.00248          0.0 0.0              0.0
ANKR 0.0244          0.0 0.0              0.0
ATOM 9.283          0.0 0.0              0.0
AVAX 20.66          0.0 0.0              0.0
AXS 6.8          0.0 0.0              0.0
BAL 3.68          0.0 0.0              0.0
BAND 1.417          0.0 0.0              0.0
BAT 0.21159          0.0 0.0              0.0
BCH 223.72          0.0 0.0              0.0
BNT 0.75          0.0 0.0              0.0
BOND 2.2          0.0 0.0              0.0
BTC 37964.39          0.0 0.0              0.0
CGLD 0.53          0.0 0.0              0.0
COMP 51.17          0.0 0.0              0.0
CRV 0.5658          0.0 0.0              0.0
CVC 0.1102          0.0 0.0              0.0
DAI 0.9999          0.0 0.0              0.0
DOGE 0.08033          0.0 0.0              0.0
DOT 5.263          0.0 0.0              0.0
ETC 18.86          0.0 0.0              0.0
ETH 2054.84          0.0 0.0              0.0
FARM 32.73          0.0 0.0              0.0
FET 0.5347          0.0 0.0              0.0
FIL 4.545          0.0 0.0              0.0
GLM 0.2418          0.0 0.0              0.0
GRT 0.1483          0.0 0.0              0.0
ICP 4.605          0.0 0.0              0.0
KNC 0.7065          0.0 0.0              0.0
LINK 14.588          0.0 0.0              0.0
LOOM 0.0673          0.0 0.0              0.0
LPT 6.33          0.0 0.0              0.0
LRC 0.216          0.0 0.0              0.0
LTC 69.78          0.0 0.0              0.0
MANA 0.4405          0.0 0.0              0.0
MATIC 0.7528          0.0 0.0              0.0
MIR 0.0941          0.0 0.0              0.0
MKR 1483.6          0.0 0.0              0.0
MLN 15.12          0.0 0.0              0.0
NKN 0.1064          0.0 0.0              0.0
NMR 15.68          0.0 0.0              0.0
OGN 0.12922          0.0 0.0              0.0
OMG 1.7217          0.0 0.0              0.0
OXT 0.0724          0.0 0.0              0.0
PLA 0.204          0.0 0.0              0.0
POLY 0.1963          0.0 0.0              0.0
QNT 99.56          0.0 0.0              0.0
REN 0.0823          0.0 0.0              0.0
REQ 0.083          0.0 0.0              0.0
RLC 1.4794          0.0 0.0              0.0
RLY 0.0126          0.0 0.0              0.0
SKL 0.0436          0.0 0.0              0.0
SNX 3.392          0.0 0.0              0.0
SOL 58.2          0.0 0.0              0.0
STORJ 0.7658          0.0 0.0              0.0
UMA 1.953          0.0 0.0              0.0
UNI 6.161          0.0 0.0              0.0
USD              0.01 0.0
USDC              0.0 0.0
XLM 0.118572          0.0 0.0              0.0
XTZ 0.817          0.0 0.0              0.0
YFI 8519.94          0.0 0.0              0.0
ZEC 29.03          0.0 0.0              0.0
ZRX 0.39338          0.0 0.0              0.0

I don’t have the ability to remember information from one interaction to the next, but I can certainly help you analyze or work with the data you’ve provided in this session. If you have a specific question or task related to this cryptocurrency data, feel free to let me know, and I’ll do my best to assist you!

User

Can you print out all of the symbols in the left column?

Certainly! Here are all the ticker symbols from the left column:

objectivec
AAVE, ACH, ADA, ALGO, AMP, ANKR, ATOM, AVAX, AXS, BAL, BAND, BAT, BCH, BNT, BOND, BTC, CGLD, COMP, CRV, CVC, DAI, DOGE, DOT, ETC, ETH, FARM, FET, FIL, GLM, GRT, ICP, KNC, LINK, LOOM, LPT, LRC, LTC, MANA, MATIC, MIR, MKR, MLN, NKN, NMR, OGN, OMG, OXT, PLA, POLY, QNT, REN, REQ, RLC, RLY, SKL, SNX, SOL, STORJ, UMA, UNI, USD, USDC, XLM, XTZ, YFI, ZEC, ZRX

If you have any specific tasks or questions related to these symbols, feel free to let me know!

User

OK, good, you remembered them. Now for the next step. I am going to show an example using btc and I would like you to create a listing with all of the ticker symbols from the left column. Example: $DIR/rsi-fractals-avgs-bbands-txt-summary.py btc

Sure, I understand. If you could provide an example of the desired output or clarify the specific format or information you’re looking for related to each ticker symbol (similar to the example you provided with btc), I’d be happy to assist you further.

User

I am looking for you to create code that has $DIR/rsi-fractals-avgs-bbands-txt-summary.py on every line but with the ticker symbols from the left column appear on that line right after it.

Certainly! It sounds like you want a list of commands where $DIR/rsi-fractals-avgs-bbands-txt-summary.py is followed by each ticker symbol. Here’s an example Python script that generates those commands:

python
# List of ticker symbols ticker_symbols = [     "AAVE", "ACH", "ADA", "ALGO", "AMP", "ANKR", "ATOM", "AVAX", "AXS", "BAL",     "BAND", "BAT", "BCH", "BNT", "BOND", "BTC", "CGLD", "COMP", "CRV", "CVC",     "DAI", "DOGE", "DOT", "ETC", "ETH", "FARM", "FET", "FIL", "GLM", "GRT", "ICP",     "KNC", "LINK", "LOOM", "LPT", "LRC", "LTC", "MANA", "MATIC", "MIR", "MKR",     "MLN", "NKN", "NMR", "OGN", "OMG", "OXT", "PLA", "POLY", "QNT", "REN", "REQ",     "RLC", "RLY", "SKL", "SNX", "SOL", "STORJ", "UMA", "UNI", "USD", "USDC", "XLM",     "XTZ", "YFI", "ZEC", "ZRX" ] # Directory variable DIR = "$DIR" # Generate commands commands = [f"{DIR}/rsi-fractals-avgs-bbands-txt-summary.py {symbol}" for symbol in ticker_symbols] # Print commands for command in commands:     
       print(command)

Replace $DIR with the actual directory you want to use. This script will generate commands for each ticker symbol, following the format you provided. If you have any specific adjustments or requirements, feel free to let me know!


OK how did it turn out?

This is the output, just as I expected and was looking for. Saved me probably about 15 minutes of tedious hand editing…

$DIR/rsi-fractals-avgs-bbands-txt-summary.py AAVE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ACH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ADA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ALGO
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ANKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ATOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AVAX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AXS
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BCH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BOND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CGLD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py COMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CRV
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CVC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DAI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOGE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FARM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FET
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FIL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GRT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ICP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py KNC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LINK
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LOOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LPT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LRC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MANA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MATIC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MIR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MLN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NKN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NMR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OGN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OMG
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OXT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py PLA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py POLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py QNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REQ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SKL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SNX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SOL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py STORJ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UMA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UNI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USDC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XTZ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py YFI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZEC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZRX

What’s it for?

It is screener code that will go into a script that prints out all the cryptos in the list sorted by their position relative to their 20 day moving average. It uses the same logic as used for the Mayer Multiple, except it is on a 20 day MA basis.

X = Current Price / 20DMA

Code

#!/bin/bash 
DIR="/home/erick/python/ta"


rm $DIR/output-rsi-er-price-sma20-multiple.csv

$DIR/rsi-fractals-avgs-bbands-txt-summary.py AAVE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ACH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ADA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ALGO
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ANKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ATOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AVAX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AXS
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BCH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BOND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CGLD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py COMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CRV
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CVC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DAI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOGE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FARM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FET
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FIL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GRT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ICP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py KNC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LINK
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LOOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LPT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LRC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MANA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MATIC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MIR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MLN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NKN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NMR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OGN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OMG
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OXT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py PLA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py POLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py QNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REQ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SKL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SNX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SOL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py STORJ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UMA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UNI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USDC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XTZ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py YFI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZEC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZRX


echo "Cur  ,Und,Targt,RSI, ER,20Ml,Mayer Mult."


# Sort by desending price/sma20 multiples
sort -nk6 -t, -r $DIR/output-rsi-er-price-sma20-multiple.csv

the script runs the rsi-fractals-avg-bbands-txt-summary code on all the tickers listed and saves output to…

output-rsi-er-price-sma20-multiple.csv

then sorts the output by the 20sma multiple column

 

Results

Best relative performers on top, worst on bottom. This idea comes from the book Fortune’s Formula and appears as a technique used by Princeton-Newport. Basically sell the top and buy the bottom ( deep value…maybe), or alternativly short the top and long the bottom. Or in the case of crypto the bottom ones might be just be duds that are always behind the pack, maybe don’t even bother with them.

Cur  ,Und,Targt,RSI, ER,20Ml,Mayer Mult.
UNI  ,USD,False,73,0.06,1.18,1.31
SKL  ,USD,False,63,0.07,1.1 ,1.59
FET  ,USD,False,67,0.14,1.09,1.97
SNX  ,USD,False,63,0.09,1.07,1.43
GRT  ,USD,False,60,0.05,1.07,1.37
AXS  ,USD,False,56,0.0 ,1.07,1.2
AMP  ,USD,False,63,0.05,1.06,1.17
MKR  ,USD,False,60,0.13,1.05,1.34
AVAX ,USD,False,61,0.07,1.05,1.61
RLC  ,USD,False,61,0.01,1.04,1.19
OGN  ,USD,False,56,0.05,1.03,1.33
UMA  ,USD,False,52,0.07,1.01,1.13
NMR  ,USD,False,55,0.02,1.01,1.14
MANA ,USD,False,48,0.01,1.01,1.18
DOGE ,USD,False,55,0.02,1.01,1.15
BTC  ,USD,False,54,0.14,1.01,1.27
REQ  ,USD,False,52,0.02,1.0 ,1.1
NKN  ,USD,False,59,0.02,1.0 ,1.1
ICP  ,USD,False,51,0.04,1.0 ,1.14
FARM ,USD,False,53,0.04,1.0 ,1.31
ETH  ,USD,False,47,0.04,1.0 ,1.13
DAI  ,USD,False,44,0.01,1.0 ,1.0
ATOM ,USD,False,49,0.05,1.0 ,1.09
ADA  ,USD,False,57,0.01,1.0 ,1.25
AAVE ,USD,False,52,0.08,1.0 ,1.38
STORJ,USD,False,46,0.13,0.99,2.0
SOL  ,USD,False,51,0.19,0.98,2.12
ALGO ,USD,False,47,0.06,0.98,1.13
ACH  ,USD,False,51,0.02,0.98,1.07
ZEC  ,USD,False,43,0.04,0.97,1.0
XLM  ,USD,False,43,0.05,0.97,1.03
LTC  ,USD,False,42,0.03,0.97,0.89
LINK ,USD,False,46,0.17,0.97,1.8
DOT  ,USD,False,45,0.01,0.97,1.08
COMP ,USD,False,49,0.05,0.97,1.11
QNT  ,USD,False,43,0.02,0.96,0.98
PLA  ,USD,False,45,0.02,0.96,1.19
OXT  ,USD,False,47,0.01,0.96,1.13
ETC  ,USD,False,39,0.0 ,0.96,1.07
CRV  ,USD,False,47,0.08,0.96,0.92
BCH  ,USD,False,36,0.1 ,0.96,1.08
LRC  ,USD,False,42,0.08,0.95,0.99
FIL  ,USD,False,41,0.0 ,0.95,1.14
CGLD ,USD,False,44,0.0 ,0.95,1.07
BAL  ,USD,False,38,0.12,0.95,0.88
XTZ  ,USD,False,38,0.03,0.94,1.03
MLN  ,USD,False,36,0.05,0.94,0.9
BAT  ,USD,False,38,0.01,0.94,1.07
BAND ,USD,False,44,0.01,0.94,1.12
ANKR ,USD,False,43,0.01,0.94,1.06
LPT  ,USD,False,41,0.04,0.92,1.13
GLM  ,USD,False,41,0.02,0.92,1.18
CVC  ,USD,False,48,0.05,0.91,1.27
ZRX  ,USD,False,36,0.09,0.9 ,1.68
YFI  ,USD,False,42,0.04,0.9 ,1.32
MATIC,USD,False,28,0.03,0.9 ,1.1
KNC  ,USD,False,27,0.03,0.9 ,1.11
BNT  ,USD,False,32,0.08,0.9 ,1.56
POLY ,USD,False,30,0.01,0.81,1.2

 

 

ChatGPT will create Tags

Finally by posting this post into a prompt you can ask ChatGPT to generate Tags for the post. I used most of them but trimmed a few that were redundant.

SD Cactus

Eastern Prickly Pear: Opuntia humifusa and Opuntia cespitosa

All nature does is to make something happen.

Native Cactus in New York

Yes, there are really native cactus in New York and actually they cover an area from Florida to Canada and west to the Rockies. I obtained some years ago at the former Baker’s Acres near Ithaca, NY. They have done well through the years and the original clump had no problem surviving a transplant while moving. In fact the plant propagates easily from one of the paddles, officially called nopales, which are the stems of the plant, which have adapted thanks to evolutionary pressure into what functions more like a leaf. The leaves in fact have, via evolution become the spines and the tiny hair like spines that you really have to pay attention to.

They have striking flowers that will come out in mid-summer, large yellow flowers that attract pollinators. Once the flowers are done they will be followed by fruit that will be about one inch thick and two to three inches long. The plant requires a lot of sun to flower and it will take a few years from when it is first planted to flower.

Eastern Prickly Pear Flower
Eastern Prickly Pear Flower

Handling the fruit and plant

The trick with the fruit and the cactus in general is safe handling. I find tongs or pliers to work, channel locks are the best. The spines will go though leather gloves too easy . Kevlar might work, or some chain mail gloves, have not tried there.

Eastern Prickly Pear Fruit
Eastern Prickly Pear Fruit

Fruit

For the fruit, I find burning the spines off works, using a gas stove burner or some equivalent type of high heat source like a blow torch. These fruit initially start green but, will ripen to a red or purple color after a few months. They are edible and usually can be picked in November. The fruit does not have much ‘meat’ to it. Hard seeds and a little pulp.

Fruit Drink

I had 12 viable fruits this year and was able to make some delicious extract from them in order to make drinks. I held on to them in the refrigerator until the end o the year as they keep well, but finally just had to do something with them as I was risking spoilage.

After burning off the needles over an open flame. I cut open the fruit and used a spoon pressed hard against the skin from the inside to push the juice, pulp and seeds into a small blender. Then I blended it and used a small screen to filter the thick juice through. This mixed with fresh squeezed line, thinned with some water and with a bit of brown sugar made for some delicious drinks. I could see adding tequila to it as it would make for a great Margarita.

Eastern Prickly Pear in June
Eastern Prickly Pear in June

Growth habit and propagation

The cactus tends to spread easily in a clump formation, when it gets tall the paddles tend to lean over and touch the ground at some point. This allows them to root and spread, similar to other ‘walking’ plants. This tendency lends them to easy propagation from a paddle clipped off or by digging up a part of the clump. I have given out quite a few starters. Visitors are always surprised that a cactus can grow year long, outside in New York. The fact that it is native is just a mind blowing fact for most people.

Late Fall Eastern Prickly Pear
Late Fall Eastern Prickly Pear, Note the fruit
Mid Winter Cactus
Mid Winter Cactus

Wintering

In winter the plant will collapse itself and fall to the ground, the paddles can dry out and look lifeless. This typically happens in New York in November. No worries, it comes back to life in spring again. It will survive in a Zone 4, so it is hardy but not quite as hardy as a Opuntia fragilis (Brittle Prickly Pear) . The brittle prickly pear will grow in the upper Midwest and up into British Columbia. This one it the next on my list to try growing here. https://www.minnesotawildflowers.info/flower/brittle-prickly-pear

Start of Cactus in 2014
Start of Cactus in 2014

Two distinct species

New York is actually home to two distinct species of cacti within the Optunia genus: Opuntia humifusa and Opuntia cespitosa — the latter of which was only recently recognized as a separate species within the state. In fact, it can be difficult to tell the difference between the two species except when they’re in bloom — the flowers of the Optunia cespitosa have a red or orange center.

“They’re fairly separated, although most of the Opuntia cespitosa is mainly south west of Kingston in more limestone areas. The other one, the humifusa, is along the Hudson River from Columbia County all the way down to Long Island,” says Steve Young, a botanist at the New York Natural Heritage Program.

Blooming Now: The Valley’s Own Native Cactus

Opuntia

Eastern prickly pear is in the Cactaceae (Cactus) family. This family contains about 1,800 species all but possibly one or two native to the New World. The prickly pears are considered an old group within the cactus family with about 150 species in Opuntia. It has the largest range of any cactus in the United States and can be found from New Mexico and Montana east to Florida and Massachusetts. It is also found in Ontario. Eastern prickly pear can form large colonies or occur as a few individuals in an area. In older botanical manuals, it is often listed as Opuntia compressa. https://www.fs.usda.gov/wildflowers/plant-of-the-week/opuntia_humifusa.shtml

https://plants.usda.gov/home/plantProfile?symbol=OPHU

Habitat

This plant is very intolerant of shade and instead thrives in sunny, hot and dry environments with well-draining, sandy soil.[10] O. humifusa will grow in open areas in sandy, rocky and coastal scrub habits. It is capable of surviving cool winters unlike many cacti, although harsh winter storms are known to cause habitat loss.[10] https://en.wikipedia.org/wiki/Opuntia_humifusa

The eastern prickly pear favors rocky terrain on ridges and summits and dry, sandy soils. It does well in desolate maritime areas and on cliff faces or outcroppings on hillsides. Like all cactuses, it has a shallow, expansive root system called a mat that allows it to quickly absorb water out of the surface of the soil it’s rooted in. Like many New Yorkers, its success depends on real estate. 

Its success is the result of a bold leap northward—a journey made by some pioneering cactus fragment or seed long in the past, probably soon after the Wisconsin Ice Sheet began to recede from New York City some 25,000 years ago. It washed up, or was carried by an animal, and spread its shallow roots in our little corner of the country. https://hellgatenyc.com/new-york-nature-cactus

As a houseplant

It even can be used as a houseplant as can be seen in the following short video.

One more video to round it all out.

makemore

Makemore

The best way to predict a future is to look for it. We live in the moment where the world is a vast machine for predicting the future.

This summer, I spent a significant amount of time contemplating large language models and delving deeper into their research. My first encounter with GPT-2 was back in 2019, where I explored its code and experimented with it. During this period, I became curious about transfer learning and its applications. Additionally, I had some prior knowledge about transformers, but it wasn’t as comprehensive as my understanding of LSTMs and RNNs. I couldn’t confidently explain what they did, for example.

While researching transfer learning with smaller models like GPT-2, I stumbled upon Gwern Branwen’s website (https://gwern.net/) and, in particular, his TWDNE Project (https://gwern.net/twdne). I found it clever because it combined a generative model for both images and text. I decided to focus on the text side of the project, as the image aspect was already well-addressed by applications like Stable Diffusion….

Misato Katsuragi as a Math Teacher
Misato Katsuragi as a Math Teacher

I might revisit the image style transfer aspect in the future, as I had previously explored it to some extent. You can find more about this in my “How to Generate Art Demo Followup.”

Before this, I had predominantly explored machine learning with code from the ground up using Python (PMLC). I have used ML practically in the form of genetic algorithms for tuning parameters on investing models for years, non-differentiable, so no chain rule! An offshoot was a project called gen-gen-algo, a generic genetic algorithm. Now, finally after all these side quests, I was ready to tackle something more complex and cutting-edge using GPT.

I found excellent resources on GitHub and in video format from Andrej Karpathy (https://github.com/karpathy). The following repositories were particularly helpful in my learning journey. The first one, “nn-zero-to-hero,” features a series of videos that provided a solid foundation in understanding transformers.

The second repository, “makemore,” served as my warm-up exercise to get back into working with transformers and Large Language Models (LLMs) after a period of dormancy in the field. You can access these repositories here:

1. “nn-zero-to-hero”: https://github.com/karpathy/nn-zero-to-hero
2. “makemore”: https://github.com/karpathy/makemore

Fork of makemore

My experience with “makemore” went beyond the basic examples provided in the original repository, which generated new names based on a dataset of names. Initially, my goal was to apply “makemore” to various datasets other than “names.txt.” I experimented with larger and smaller datasets, including those with extensive collections of English words, numbers for addition, square roots, and a substantial dataset of quotes containing nearly 10 million entries, some of which had lines as long as 505 characters. By using scripts and modifications to “makemore.py,” I conducted a grid search to optimize hyperparameters, including constraints on model size. Output from “makemore.py” was saved to a CSV file, along with hexadecimal hash values for easy tracking and analysis during the tuning process.

To further enhance the code, I introduced a grid search optimization method using a Bash script. This allowed for exploring the hyperparameter space while maintaining a ceiling on the model size. Without such constraints, optimization typically led to increasingly larger models that resulted in the lowest loss.

I also introduced the concept of assigning a random hexadecimal tag to the output of “makemore.py.” This tagging system facilitated the easy identification of the best loss and the associated set of hyperparameters that produced it. Additionally, I incorporated an early stopping mechanism into the “makemore.py” code.

If you’re interested in exploring my fork of Andrej Karpathy’s “makemore” code, you can find it here:

https://github.com/erickclasen/makemore

For a more detailed understanding, I’ve created a comprehensive “verbose-readme.pdf” that provides additional information:

Version on this site, opens in browser:

verbose-readme

GitHub Version requires downloading:

https://github.com/erickclasen/makemore/blob/master/verbose-readme.pdf