Raspberry Pi

Sparkleshare on Raspberry Pi

This was my process of setting up Sparkleshare, server side on the Raspberry Pi. This guide is basically a dump of the terminal that I noted important points on.

This guide is color coded

My Input on Command Line

My  Notes

Run the Dazzle script via Curl

…from https://www.sparkleshare.org/ under Setting Up a Host.

erick@raspberrypi ~ $ sudo curl https://raw.githubusercontent.com/hbons/Dazzle/master/dazzle.sh \
>   –output /usr/bin/dazzle && chmod +x /usr/bin/dazzle
[sudo] password for erick:
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100  8639  100  8639    0     0   2954      0  0:00:02  0:00:02 –:–:–  4674
chmod: changing permissions of `/usr/bin/dazzle’: Operation not permitted

 Using sudo causes this, so just execute the last line again with sudo…

erick@raspberrypi ~ $ sudo chmod +x /usr/bin/dazzle

Next Run Dazzle Setup

Interesting stuff going on when the project is created, look at the dump to get an idea of what it is doing.
erick@raspberrypi ~ $ sudo dazzle setup
1/4 | Installing the Git package…
-> The Git package has already been installed (version 1.7.10.4).
2/4 | Creating account “storage”…
-> useradd storage –create-home –home /home/storage –system –shell /usr/bin/git-shell –password “*” –user-group
3/4 | Configuring account “storage”…
-> mkdir –parents /home/storage/.ssh
-> touch /home/storage/.ssh/authorized_keys
-> chmod 700 /home/storage/.ssh
-> chmod 600 /home/storage/.ssh/authorized_keys
4/4 | Reloading the SSH config…
-> /etc/init.d/ssh reload

Setup complete!
To create a new project, run “dazzle create PROJECT_NAME”.

Creating a Project called Rasp_Pi_Main_Share

Interesting stuff going on when the project is created, look at the dump to get an idea of what it is doing.

erick@raspberrypi ~ $ sudo dazzle create Rasp_Pi_Main_Share
Creating project “Rasp_Pi_Main_Share”…
-> /usr/bin/git init –bare /home/storage/Rasp_Pi_Main_Share
-> /usr/bin/git config –file /home/storage/Rasp_Pi_Main_Share/config receive.denyNonFastForwards true
-> echo “*.jpg -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.JPG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.jpeg -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.JPEG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.png -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.PNG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.tiff -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.TIFF -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.gif -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.GIF -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.psd -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.PSD -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.xcf -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.XCF -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.flac -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.FLAC -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.mp3 -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.MP3 -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ogg -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.OGG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.oga -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.OGA -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.avi -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.AVI -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.mov -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.MOV -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.mpg -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.MPG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.mpeg -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.MPEG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.mkv -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.MKV -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ogv -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.OGV -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ogx -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.OGX -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.webm -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.WEBM -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes     -> echo “*.zip -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ZIP -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.gz -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.GZ -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.bz -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.BZ -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.xz -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.XZ -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.bz2 -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.BZ2 -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.rpm -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.RPM -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.deb -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.DEB -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.tgz -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.TGZ -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.rar -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.RAR -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ace -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ACE -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.7z -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.7Z -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes       -> echo “*.pak -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.PAK -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.msi -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.MSI -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.iso -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.ISO -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.dmg -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes      -> echo “*.DMG -delta” >> /home/storage/Rasp_Pi_Main_Share/info/attributes
-> chown –recursive storage:storage /home/storage
-> chmod –recursive o-rwx /home/storage/Rasp_Pi_Main_Share

Project “Rasp_Pi_Main_Share” was successfully created.
To link up a SparkleShare client, enter the following
details into the “Add Hosted Project…” dialog:

Address: ssh://storage@192.168.1.17:22
Remote Path: /home/storage/Rasp_Pi_Main_Share

To link up (more) computers, use the “dazzle link” command.

Save this info somewhere good!

Address: ssh://storage@192.168.1.17:22
Remote Path: /home/storage/Rasp_Pi_Main_Share

You will need it to hook up clients to the server.

Dazzle Link, Linking clients to server

All that the dazzle link command does is add the key to the authorized_keys in the .ssh folder of the /home/storage directory.

erick@raspberrypi /home/storage $ sudo dazzle link
Paste your Client ID (found in the status icon menu) below and press <ENTER>.

Client ID: ssh-rsa AAAAB3Nza………………………….Plex-790

The client with this ID can now access projects.
Repeat this step to give access to more clients.

 

Optional:Moving this repository to the USB Stick

To get the storage folder off of the SD card and onto the external media, i.e. USB stick in my case…

Move the files…

sudo rsync -rPz /home/storage/Rasp_Pi_Main_Share/ /media/sda/Rasp_Pi_Main_Share/

Mount with bind. Using a symlink might not work here as it might not get followed. I haven’t tried it.

sudo mount --bind /media/sda/Rasp_Pi_Main_Share/ /home/storage/Rasp_Pi_Main_Share/

..this is optional, but with the limited SD card storage on the Raspberry Pi, it makes it much more useful for a shared storage device.

To Install on Client: Linux

To install Sparkleshare on the client Linux computer, Ubuntu/Mint based..

sudo apt-get install sparkleshare

 

 

Are Transformers Here to Stay for a While?

A Transformer Theory

Part of my pet theory on transformers is that they might be a thing that will stick around for a while. I know ML changes fast but, I have an example from history that serves to illustrate a design that has staying power. Radio started out with many schemes to detect signals progressing along the following lines, coherers, galena crystals, diodes, tuned radio frequency, regeneration, super regeneration, all forms of direct conversion ( signal gets demodulated right from the incoming frequency) and finally superheterodyne which takes the incoming frequency and mixes it down or up to an intermediate frequency before detecting/de modulating the signal. The first methods each lasted but a few years and all under performed and had flaws. Once the superheterodyne was invested there were just a few different flavors of the same idea, so called single, double and triple conversions, really just more layers to reject out of band signals more efficiently. The superheterodyne, like the transformer has staying power. After 100 years it is the ‘way’ to handle a radio signal. So that tech is in your Wifi, Phone, TV, Stereo, Modem, 2 way radios, GPS and so on and is unlikely to be replaced by anything but a tech that just maps it onto digital tech. So my theory is that the transformer is in the same ballpark, it is the superheterodyne of ML or at least close, a step more or two away, that’s all. The types of radio reception using earlier methods each had their day in the sun, just as RNNs, LSTMs, GRUs were each the cutting edge ML go to architecture for a while.

 

sellbuy-o-meter

The Golden Ratio Multiplier Trading Algorithm

This post covers a trading algorithm developed in Python code that uses Fibonacci numbers to set levels to trade off of. It is not a short term strategy, however it is a long-term strategy more of an investment strategy. It is based on a post by Philip Swift on https://www.lookintobitcoin.com/charts/golden-ratio-multiplier/

And he also has an original article on it on medium.

https://medium.com/@positivecrypto/the-golden-ratio-multiplier-c2567401e12a?sk=e38c4eff4357a1131bed57a3f4b3eeae

I have code for this here:

cbpro-cli-tools

Theory

The golden ratio multiplier is a great indicator, visually easy to confirm and makes a lot of sense on how it works. Best of all no fussy back-testing with parameter tweaks, you just need your eyes. I like how Josh Olszewicz  added the centerline, that is a nice addition. His video was a pleasant and positive surprise that I used to review the working of the golden ratio multiplier. I started using this indicator in early 2020.

On top of the basics that are covered by Phillip Swift , I have made some additions to the it that will work well in algorithmic form to allow it to run with constraints as code. Constraints are needed to allow it to pick the correct points to enter and exit along the curve. The method of (Dollar Cost Averaging) DCAing in and out is to use 20SMA BBands (Bollenger Bands) as a point to trigger DCA, using this as a filter to trigger on price exceeding lower/upper BBand. This tweak is based on the fact that I wrote code around this and was looking for a good way for when it should reasonably DCA, versus just triggering on a crossing the Fib # * YearlySMA (SMA, Simple Moving Average), or DCAing daily when under/above this line. As far as amounts to DCA, generally with accumulate/distribute investing, I tend to compound the trades in a way that accumulates heavier as distance increases below the YearlySMA line and distributes heavier as it goes further above the target Fib # * YearlySMA. The trading scaler used is a logistic curve , with n being the ratio of the current price to the yearly avg for buys (For sales ‘n’ is inverted by n being subtracted from the target Fib level multiplier + 1, which positions the curve out to where it scales up until it reaches the next higher Fib # and back down. Using the logistic curve formula (1+k)/(k + n^e) as a trade multiplier, for example with k=0.2 and e=4, it DCA’s in around 2x at the 0.8 * yearlySMA point,, 4x at 0.5…never more than 6x as it approaches that as a limit as n –> 0. I am planning publishing this code and documentation around it and will cite your video in it when I do.

Golden Ratio Multiplier Tutorial

Also Josh Olszewicz has a video tutorial on YouTube about this particular indicator using it with TradingView code it is a worth taking me 10 minutes or so to review this as background material along with looking at https://www.lookintobitcoin.com/charts/golden-ratio-multiplier/ and the original medium article . I am going to try to stick to the essence of the code and not so much the background material which could be read in viewed in those locations.

Code

This code is light weight and can be run on a Raspberry Pi as well as a Linux PC. It requires Python 3 as this is needed for the API.

The code is designed to use the Coin Base Pro API, however there is a middle layer that interfaces with the API via cbpro_buy_sell.py  If someone wants to interface with another API this just needs to be modified to make the proper calls into another exchange as an API on the back end of it. Think of it as a thin interface layer that goes between the code and the API itself.

cbpro-cli-tools

Dependancy : One import

this would be the import cbpro code that constitutes the module for the CV pro interface API.

Other Internal Dependancies

There are a few dependencies in the code thankfully they are local dependencies. The intent was to make this code as freestanding as possible without having to import various library’s other than what is barely necessary to make it operate. The first is a file called coretamodule.py. It holds a lot of functions that are used in multiple algorithm some of which will be used in the code to implement the golden ratio multiplier which is known as the arc just Python. The other necessary dependency is a piece of code that calls for polls periodically the CB pro ticker to add prices and volumes to aC SV file. This is run from some thing like Cron on a regular basis hourly daily five minutes what have you it is run at the same rate as a fib-ad high is run calling this code 1st to fill in the CSV file which will be read in by fib active high. I will try to make this publicly available on a site where it could  be pulled down with that lease this up to the current days data. Current days data

Core concepts

The idea of the code is that a 350 day moving average is calculated.This forms the basis of the ratio that is the golden ratio between the average and the price itself. When the price is below a threshold which is coded in to a dictionary of two pools as one. When below this level it will accumulate in the particular currency pair. There is another number in this threshold when it is above this ratio of price to 350 day moving average price. this forms the basis of the ratio that is the golden ratio between the average and the price itself. When the price is below a threshold which is coded in to a dictionary of tuples as one. When below this level it will accumulate in the particular currency pair. There is another number in this threshold when it is above this ratio of price to 350 day moving average price.above this range it will distribute the currency pair. Some fine tuning is made when it is above or below the threshold. The fine-tuning is based on Bollinger bands when it is touching the bottom bands in the Bollinger band 20 day bands and the minimum price for the 20 day period is also the same as this the price they buy is initiated.C onversely for the sale when the price is at the upper edge of the Bollinger band and the maximum price for the 20 day. Has been hit as a target a sale is initiated. This makes it dollar cost average in and out on a periodic basis. The idea is to have extra filtering to kind of find a peek in the valley of price movement without DCA (Dollar Cost Average) in too much or out too much.

defines.py

there is also a dependency file called defines.py this file has the portfolio amount in it. Now it is being dynamically allocated by using the cbpro_read_accts.py.  It will scale the amount of USD traded however BTC an ETH have to be adjusted within the main function to pick values that are comfortable and proper for the trading circumstances. This allows for configuration between all of the pairs.Also in the defines file you will have to enter the key passphrase and be 64 secret, this will allow the trades to occur through the API as this is passed on by the main code.

 

Also in the defines file you will have to enter the key passphrase and be 64 secret, this will allow the trades to occur through the API as this is passed on by the main code.

 

 

Outputs and Logs

The output from the code is interpret able humanly to understand what is going on as well. There are rose of prices after the initial currency pairs listed these prices represent the various thresholds at the end it states whether it is holding or not and also what the threshold and target are the threshold being the Bollinger bands edge and the target being the lowest to highest price for the 28th time.Additionally there are log files created when the code runs there are several that are verbose and specific to a underlying currency and there is one summary file. The verbose files contain the returned output from the API function call which is a dictionary of values return from CB pro. There will also appear within the non-verbose output as a message usually relates to some thing that needs to be corrected possibly a bug or some thing like insufficient funds. This is driven by the message key from the CB Pro API return dictionary.

 

The code makes every  attempt to avoid this because it uses limit orders and it also checks the balances in the supply. The underlying currency and the to be bought or sold currency itself.

Compounding Positions

There is also a logistic scaler in this code. The logistics scaler works by increasing the amount of currency bought when the price falls below the threshold of the golden ratio multiplier which is by default one. This allows increasing the amount purchased automatically but also reaches a limit as the ratio between the price and the 350 day moving average approaches zero. It will go to a limiting constant. The opposite is true for cells there is the threshold that is coded into the dictionary of tuples the logistic scaler takes in that value plus one informs a curve that goes from the upper threshold to all 1+ this value and increases the amount sold after this plus one position it will decrease once again to zero. The idea is that it scales out hard as the price rises but has a limiting factor if it rises above the threshold too far. The idea would be to manually address these thresholds to what is expected of the currency. So periodically maintenance might be required on the threshold or they might just be a set it and forget it for some people. This all depends on how you want to invest.

 

As the price rises but has a limiting factor if it rises above the threshold too far. The idea would be to manually address these thresholds to what is expected of the currency. So periodically maintenance might be required on the threshold or they might just be a set it and forget it for some people. This all depends on how you want to invest.The default or one and two in a two pole for every currency pair. It is possible to adjust days as needed. It is also possible to adjust the logistics scaler Constants which are KNEE is the exponential constant which controls the rate of rise of the curve and key controls the multiplier affect in the amount that would be traded in the limit.

Outer Loop

The code works by looping through a list actually a dictionary of underlying currencies US D/ETH/BTC as underlyings. Then there is a function call and the inner currencies are called in a loop these other currencies that are actually traded against the underlying currencies. This allows money currency pairs to be traded and others to be added in the future.

CSV Data Collection using API

The caveat here is that there has to be enough data in the CSV file to go back for the time. And if a currency one is to be added it has to be attitude there to harvest the ticker data. I suppose in theory it would be possible to fit all data into the file itself to add a prior prior currency that’s been running in the ticker for a while if someone is motivated not to do that. Having this code in a folder and having it called by Cron preferably using a script calling the script first then harvesting the price volume data then calling for code it will work seamlessly as a plug and play algorithm. Because it is using the golden ratio multiplier there is no need for back testing as this has been proven out to work by Philip Smith Swift in his presentation and write up online.

Obviously this code could be modified in anyway the 350 day. Could be changed to something else along with the 20 day Bollinger band. And the threshold as I said earlier our configurable.

 

 

 

 

Hacker Cat

Using AWK to pull data from a field within a text file

Intelligence is the power which gives us the ability to distinguish when we are not conscious.

# awk -F” ” ‘{print $11}’ pressure-delta.txt
tail -1 $PLOTTABLE_DIR/$PRESSURE_DELTA_TAIL_FILE > $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt

# Use awk with -F” “, which uses space as field seperator.
# pressure delta in inHg is in the 11th field.
# Multiply by 1000 because bash does not do floating point math natively and we want to compare so use milli-inHg to do the bash math.
# Operate on the file $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt

pressure_delta=`awk -F” ” ‘{print $11*1000}’ $PLOTTABLE_DIR/pressure-delta-tail-one-line.txt`

TMDNE

TMDNE: This Mathematician Does Not Exist

It’s like the math is the seeds or DNA and the code is the fertile ground and the sun shines, or in our case electricity flows to CPUs and GPUs.

Ever pondered the possibilities when algorithms dabble in the divine art of mathematics? Prepare yourselves for TMDNE – where Stable Diffusion meets the blackboard of the mind! 🧮🤯

Inspired by the brilliance of projects like “This Waifu Does Not Exist” (TWDNE) by Gwern Branwen, TMDNE takes a quantum leap into the realm of imaginary mathematicians. These aren’t your conventional number crunchers; they’re the brainchildren of algorithms, immersed in the ethereal dance of imaginary theorems and proofs. 🎨➗

Much like TWDNE introduced us to AI-generated characters that captured our otaku hearts, TMDNE invites you to witness the magic of machine learning as it conjures up mathematical maestros who never attended a single conference. 🤓📚

Get ready for a symphony of computational creativity, where algorithms showcase their virtuosity in crafting mathematicians who not only ponder the mysteries of prime numbers but might just solve the Riemann Hypothesis over a cup of virtual coffee. ☕📏

Embark on this intellectual odyssey where Stable Diffusion meets the abstract canvas of mathematical imagination, proving that in the world of artificial intelligence, even mathematicians can be a product of algorithmic brilliance! 🔢🤖

 

Grid Search in Stable Diffusion

The images in the following video were created by doing a grid search with Stable Diffusion, with slight variations of words in the prompt and scanning the guidance scale. The seed value remained constant. The images were also audited manually for obvious artifacts and flaws.

Finally a slideshow was created by using the following ffmpeg command:

ffmpeg -framerate 1/3 -pattern_type glob -i "image\([0-9]*\).png" -c:v libx264 -r 30 -pix_fmt yuv420p slideshow.mp4

This results in a slide show with a change every three seconds on the images.

The Idea

Math -> AI -> Imaginary Mathematicians

OK, AI is math really, a bunch of chain rule linkage and matrix multiplies for the most part and to make it do something useful the math gets worked into code as the substrate. It’s like the math is the seeds or DNA and the code is the fertile ground and the sun shines, or in our case electricity flows to CPUs and GPUs. In 2024, it’s gotten really good. imagine generation that is. I did some AI image modification years ago but, Stable Diffusion has gone way far, fast. So wouldn’t it be interesting and humorous to generate images of math being worked on, by a mathematician generated by AI. As can be seen from the images, this points out a weakness too. People, it kind of nails it, backgrounds too. Text and numbers, not so much. I’ll be interested in seeing it nail this too, only a matter of time.

Grid Search & Ground Rules

As an AI researcher, I am always interested in pushing the boundaries and grid search is something that is a normal event for me. I’ve played with Stable Diffusion for quite a while and I was interested in seeing what a semi-serious grid search would reveal now that it is seriously capable of generating graphics. I decided to stick with images that are not ‘fantasy’ or too enhanced to keep them real looking so that they are easier to judge.Keeping the ‘look’ in the same zone also helped keep some degree of uniformity but, Stable Diffusion varies the image within a range that seems to work well.

Yet another sidequest

This started as a bit of a sidequest as I am working on a project called TinyMath that is all about identifying how small a GPT model can get and get a passing grade for math. This way it is possible to score objectively and see how model size and loss stack up against the ground truth of taking an ‘exam’. I just happened to create an image using Stable Diffusion of Misato Katsuragi from Neon Genesis Evangelion at a blackboard working out some math when I was pondering this, kind of a nod to TWDNE on a document I wrote on makemore. That combined with an image that I saw in 2020 when my partner Renee was searching for a new hair style, was another inspiration. The image below only shows a partial view of the woman and I often wondered if the other views were out there somewhere, or could AI come close to generating the other views.

The Two Images that got TMDNE going

Misato Katsuragi as a Math Teacher
Misato Katsuragi as a Math Teacher
Layered Stacked Bob for thick hair. Great style and color.
Layered Stacked Bob for thick hair. Great style and color.

 

Notes & References

TWDNE

“This Waifu Does Not Exist” (TWDNE) is a project created by Gwern Branwen. It involves the generation of images of fictional characters using Generative Adversarial Networks (GANs). GANs are a type of artificial intelligence algorithm that can generate realistic-looking images based on patterns learned from a dataset.

In the case of TWDNE, the GANs are trained on a dataset of anime-style characters, and the output is entirely computer-generated images of characters that do not exist in reality. The term “waifu” is a romanization of the Japanese word for “wife” and is often used in fandom to refer to fictional characters, especially those from anime and manga, for whom someone has a strong emotional attachment.

It’s worth noting that the images generated by TWDNE are not real individuals, but rather creations of the algorithm based on the patterns it learned during training. The project is an interesting example of the capabilities of AI in generating realistic-looking content.

Prompts, Seed and Range for Guidance

Styles Tried:

sai-photograpic
None
cinematic-default

Seed: 52794693

Positive Prompts:
young woman with short red hair in graduated bob style teaching math at a blackboard

young woman with short red hair in posh bob style teaching math at a blackboard

young woman with short red hair in victoria beckham posh bob style teaching math at a blackboard

young woman with short red hair stacked  with very short stacked layers in the back   teaching math at a blackboard

young woman with short red hair stacked  with very short stacked layers in the back   teaching math at a blackboard

Negative Prompt:
bangs curls

…. bangs were sometimes screwed up and curls caused artifacts, so these were filtered on.

Guidance range: 5-18.5 ( found to be best)

Additionally, the results were judged by looking at the images for excess artifacts or distortions such as missing fingers, awkward postures and anything else unrealistic beyond the fact that Stable Diffusion is still limited in terms of representing numbers and letters.

 

 

Buddigar

Buddigar: Stable Confusion

Humor is the weapon of self-control.

Budgierigar
Budgierigar

While playing around with Stable Diffusion, I realized that I must have misspelled “Budgerigar” and was surprised to see the monstrous images that it generated. Perhaps I accidentally typed “Buddigar” or some other variation, resulting in the strange images below. Some of them resemble monsters, while others appear to be stores from a distant land outside of the USA.

These images fall into the uncanny valley of creepiness. For instance, I heard on a Lex Fridman podcast that image generators sometimes struggle with creating realistic facial expressions. It’s fascinating to observe what Stable Diffusion produces when prompted to generate an image of a person with a surprised expression. Although the results can be unsettling and bizarre, they’re also quite amusing, like this one….

OMG, a Buddigar!!!!!
OMG,I spot a Buddigar!!!!!

Buddigars

The first one is my favorite in terms of shock and awe.

Buddigar: It's like Golem's Pet.
Buddigar: It’s like Golem’s Pet.

Buddigar tale by ChatGPT (mostly, just with a bit of prompting and edits)

There is a mischievous creature named Buddigar. It’s small and furry, with sharp claws and bright, beady eyes that seemed to sparkle with mischief. Buddigar loves nothing more than causing trouble, and it has a talent for doing so in the most unexpected ways.

If you can’t find your keys, Buddigar is probably to blame. It has a habit of sneaking into people’s pockets and snatching whatever shiny object catches its eye. It wasn’t that Buddigar has any use for these trinkets – it simply enjoys hoarding them, like a dragon with its treasure.

And if your computer is running slow, you could bet that Buddigar had something to do with it. It has a knack for hiding files, moving icons, and generally wreaking havoc on any device it comes across. Even the most tech-savvy of users are no match for Buddigar’s cunning ways.

But Buddigar’s mischief isn’t limited to material possessions. It has a strange talent for causing physical ailments, too. If you have a pain in your hip, for instance, you could be sure that Buddigar has something to do with it. It’s not that Buddigar means to hurt anyone – it simply enjoys the chaos that its actions cause.

The funny thing is, Buddigar doesn’t try to hide its misdeeds. When caught in the act, it simply grunts and declares its mischievous intent proudly. “Buddigar!” it exclaims, with a mischievous twinkle in its eye. “Buddigar take keys! Buddigar slow computer! Buddigar make hip hurt!” And then it scampers away, leaving behind a trail of chaos and confusion.

Despite its mischievous nature, however, Buddigar is impossible to hate. There is just something endearing about its playful antics, and the way it seemed to revel in the chaos it caused. Even Gollum, the famously twisted and corrupted creature from Lord of the Rings, would have adored Buddigar’s mischievous ways. It is so ugly it’s almost gone full circle to cute in an uncanny way.

And so, Buddigar continues to roam the land, causing trouble wherever it goes. Some curse its name, while others simply laugh and shake their heads in amusement. But love it or hate it, there was no denying that Buddigar is one of a kind – a little creature with  a talent for mischief that it’s unmatched by any other.

 

More Buddigars…

Monetary System Escape Hatch

MEH: Money Escape Hatch

 Money is power – your money, the power to spend.

An escape hatch from the Monetary System

Recently I was the victim of several types of fraud through the legacy monetary system, banking counter-party risk basically.

  • Debit card fraud: A transaction that I did not make showed up on my account. Required getting a new card and the bank did take care of the fraudulent charges.
  • A fraud about debit card fraud: A fraudster called me, preceded by a text that told me of a suspicious card transaction, spoofing the banks own phone number no less. They tried to get me to give up my credentials over the phone and then pretended there were several checks written against my checking account. They tried to run me through the process of setting up Zelle and having me send money to a weird email address. I just played along and acted like I didn’t understand how to use the Internet until the call dead ended.
  • Law firm fraud: A law firm, a lawsuit mill, files a fraudulent suit in the wrong venue using a fake address, so I get sued without my knowledge, resulting in a default judgment. I caught this in time before they were able to execute the judgment, or else they could have performed a money grab out of the bank account, without my knowledge. I informed the court of the fraud, the lawfirm, the Consumer Finance Protection Bureau  CFPB and all other parties that had a hand in this. This was a shocker as I didn’t even realize that his type of fraud existed.

Thoughts

After all this I was thinking about what could be done to lock up money, away from the reach of fraudsters as there is counterparty risk to a lot of things money related, especially in the legacy banking world. Some banks don’t even offer good security, like 2FA via a security key only and NOT 2FA texting to a mobile number that can be spoofed or email, equally weak.

Self custody of Bitcoin seems like the most solid way in light of the weaknesses in the monetary system. Why Bitcoin? Other tokens have less utility, why do we need 1000s of them, and some might be counted as securities someday and some have more inflation as  tokens are minted at the whim of the founders or core team and so on.

But, there are caveats:

  • You need to know what you are doing or else you can blow you cover and compromise your keys and therefore coins and also be too ‘public’ with transactions.
  • Corollary: Not your keys, not your coins!
  • Don’t trust, verify
  • Bitcoin is pseudo anonymous. You are sitting behind a public key and if you use a limited set of keys repeatedly it is possible to trace, via history on the blockchain, who you are and how many sats you have stacked. This is particularly true of an address used, lets say as a donation or payment address which is static. Ideally, you want dynamic addresses. Revealing too much information like this could make you a magnet for fraudsters that decide it is worth trying to, let’s say hack your warm wallets by injecting malicious code on your phone or PC, via an email with a cat video. Or spoof your phone number to bypass 2FA and so on. Heck, they might even try to track down where you live via social media and park in front of your house and break into the WiFi by stepping in on the four way handshake used to secure it, then they are on your home network and have access to every device directly!
  • Always visually verify addresses you send to. Just in case some copy/paste virus gets in the middle and changes the address.

 

Suggestions for caveats:

  • Your keys = Your Coins: Store the bulk of your stack on a cold wallet or paper wallet, in a safe place. Seed phrases as well. Keep only a small amount on a hot wallet, like an app or web wallet, exchange,etc.
  • Stay Private: Use methods to conceal the path of transactions by breaking the address linkage, effectively creating dynamic addresses. Porting through privacy coins comes to mind here and for a BTC only solution, Wasabi wallet, linked to a cold wallet such as Coldcard, BTC only wallet that can be air gapped.
  • Equipment SEC: Air gapped wallet allows you to use something like an SD card to move a partially signed transaction to the cold wallet to sign and back to a hot wallet that is watch only, so it can only take in but, not spend BTC. You can only spend by creating a partially signed transaction and moving it by hand using SD card to cold wallet, signing and moving it back via SD card. Sounds complicated but, is secure. No compromise and follows the verify and don’t  trust the hot wallet sitting on the phone or PC connected 24/7 to the Internet, making it only medium secure. Keep phones, PCs and Wifi secure, good passwords/biometrics and keep thinking through the holes lurking in security.
  • Dumb Human Things: Verify addresses when sending, use excellent passwords and PINs, read the instructions on equipment, like wallets, seedphrase security. Don’t get conned, don’t brag, don’t accidentally dox yourself. People are always inventing new ways to screw things up, so even with the best technology and encryption, mistakes happen, look on the Internet for more examples.

 

Idea on using Wasabi Wallet to enhance privacy along with cold storage

You can use Wasabi Wallet to enhance the privacy of your coins before transferring them to a cold wallet. Here’s a step-by-step process:

  1. Transfer Funds to Wasabi Wallet:
    • Transfer your funds from the exchange or other warm wallets to your Wasabi Wallet. This can be done by sending the funds to an address generated by your Wasabi Wallet.
  2. Initiate CoinJoin Transaction in Wasabi:
    • After receiving the funds in your Wasabi Wallet, initiate a CoinJoin transaction within Wasabi. This process will combine your transaction with those of other users, significantly enhancing privacy.
  3. Wait for Confirmation:
    • After initiating the CoinJoin, wait for the transaction to be confirmed on the Bitcoin network. This may take some time, as it depends on network congestion and the number of confirmations required.
  4. Send Funds to Cold Wallet:
    • Once the CoinJoin transaction is confirmed, you can safely send the funds from your Wasabi Wallet to your cold wallet. This step ensures that the funds you send to the cold wallet have undergone the privacy-enhancing CoinJoin process.
  5. Consider Multiple Rounds of CoinJoin:
    • For additional privacy, you may consider repeating the CoinJoin process with the funds in your Wasabi Wallet before sending them to the cold wallet. This can be done by initiating another CoinJoin transaction within Wasabi.

Remember, while this process can significantly enhance privacy, it doesn’t provide absolute anonymity. Also, the privacy features depend on the number of participants in the CoinJoin process, so it’s beneficial if more users are actively participating in CoinJoin transactions.

Always stay informed about the latest features and best practices in using Wasabi Wallet, as the specifics of the wallet’s functionality may evolve over time. Additionally, consider the transaction fees and potential delays associated with the CoinJoin process and Bitcoin network confirmations.

Alternative Idea Using Monero Swap

Just an idea that I was thinking of, not sure if it would be as good as the solution above using Coinjoin. But, the idea is to take some kind of coin BTC, USDC, whatever that you on ramped from USD via an exchange. Use some kind of swap, like SimpleSwap or the swap feature of a wallet such as Exodus and swap the non private coins from an exchange into XMR on a wallet, then swap to something like BTC on the cold wallet. When spending, run backwards, swap to XMR, then to the crypto of your choice and spend

When you convert BTC to XMR, the transaction history of the BTC is effectively broken, as the privacy features of Monero make it difficult to trace the source of funds. However, when you swap back to BTC, the privacy features of Monero may not be as effective, and your transactions could potentially be traced from that point onward.

It’s essential to note that while Monero provides strong privacy features, the overall privacy of any cryptocurrency transaction depends on various factors, including the platforms and services used for the swaps. Additionally, the regulatory environment surrounding cryptocurrency exchanges and transactions may impact the level of privacy you can achieve.

Word-cloud-for-post-on-large-price-moves

Thoughts on the best BTC price moves

As the great and powerful Satoshi Nakamoto once said, “If you don’t believe me or don’t get it, I don’t have time to try to convince you, sorry.”

For a while I have used a piece of code that started as back-testing code and forked it to produce output whenever BTC-USD makes a 2 standard deviation move in price up or down. Basically I manually adjusted the threshold that is normally a fluid value in backtesting to a fixed value, so that 95% of the time a U for Up, or D for Down move is not recorded, It’s counted as O, which means nothing interesting happened and is not printed. CRON runs this hourly and the machine responsible for trading runs 24/7 and collects prices, since August 28,2018. It even plays a sound when one of these outlier moves occurs.

Here are a few lines from the end of the output….

2024-01-03 16:00:02.618000       63874 42197.47 43004.35 U 2 1.02
2024-01-03 19:00:03.434000       63877 42923.71 42269.13 D 2 0.98
O: 43795  95.0%
D: 1165  2.5%
U: 1148  2.5%

The top two lines have a time stamp followed by a tick count, this is where it is in the data set, in order, adding 1 per hour. Next are the two prices, the starting and ending prices, U or D, signalling which way it just went. The next number shows how many hours have elapsed since the last radical move followed by the scale of the move. This is a rounded number that shows what number would be required to multiply the first price to get the current price, how big a move in other words.

2 Standard Deviations for Bitcoin

Daily

I recently adjusted the calibration threshold as BTC is settling down into less radical swings as time goes on. Currently it is….

THRESHOLD = 0.01459 # 0.0156

…which means on an hourly basis a 1.459% move registers as a 2 standard deviation move. It was 1.56% earlier this year, so volatility is trending down over time.

Can that be tradable, not likely as fees and slippage would most likely eat up the 1.459%, granted some higher swings do occur and I have pondered this. What is you bought all the big down moves and sold on the big up moves. Sounds good in theory and would most likely only work in an up market or else, you just wind up buying at bad entries and ‘bag hold’ too much at a loss until the next bull run.

Weekly

I got the idea to look at a daily and weekly time frame. Weekly is such a short output that I will paste it here….

Start of Data: 2018-09-05
Lines of input data: 279
Lines to process: 279
2018-11-21 05:00:11.243000       17809 6268.44 4452.68 D 10 0.71
2019-04-03 04:00:15.519000       17828 3976.88 5050.01 U 18 1.27
2019-05-15 04:00:15.266000       17834 5782.6 8063.78 U 5 1.39
2019-06-26 04:00:16.011000       17840 9139.01 12228.39 U 5 1.34
2019-07-17 04:00:14.972000       17843 12983.74 9411.06 D 2 0.72
2020-03-15 04:00:05.033000       17878 8749.01 5168.75 D 34 0.59
2020-08-02 04:00:06.489000       17898 9686.6 12000.0 U 19 1.24
2020-12-20 05:00:12.005000       17918 18865.0 23460.71 U 19 1.24
2021-01-03 05:00:14.866000       17920 26732.29 33231.68 U 1 1.24
2021-02-14 05:00:19.359000       17926 38365.06 47607.09 U 5 1.24
2021-03-14 05:00:09.349000       17930 49437.95 61191.88 U 3 1.24
2021-08-01 04:00:03.128000       17950 34192.29 42461.2 U 19 1.24
2022-06-19 04:00:02.056000       17996 27515.55 18282.06 D 45 0.66
2023-03-19 04:00:02.137000       18035 20598.23 27282.54 U 38 1.32
D: 4  1.4%
O: 264  95.0%
U: 10  3.6%

A few things to note:

  • There is asymmetry as the market has in general been rising since 2018, so there are only 4 ‘large’ down moves and 10 up. 1.4% weeks are up moves, 3.6% are down. The number after U and D is the weeks between moves.
  • Note the magnitude of the moves. They are truly massive. As the threshold is now 0.235.
  • Note that the 5 sequential up moves in a row 2020-2021 barely made in over the threshold at 1.24
  • Make note of the fact that Bitcoin is calming down over the years by looking at how many times per year it makes these epic moves.
  • Last but not least. I wonder if the what the linear regression of the prices would tell. Perhaps good targets for the future?
Large Weekly BTC Price Moves
Large Weekly BTC Price Moves

From now on, I will monitor this more carefully. As one can see, entry and exit although infrequent would have lead to stellar performance. I’ve been doing crypto TA and writing related code for almost 6 years now and it is still interesting that some new price relationships can be sussed out.

Looking ahead

It’s a guess buy based on the chart above it will be worth seeing if there is a pullback below the price of the blue linear regression termination, 37K-ish down to the termination of the D and E price lines 20598.23 27282.54. It will be interesting to followup on this.

Daily

For daily the THRESHOLD = 0.076, so that means greater that 7.6% moves are counted. Below I have pasted in the 2022 to 2023 results. 2022 being terrible and 2023 awesome. I let the reader ponder the readability of these price moves.

 

2022-02-05 05:00:03.175000       19053 37326.16 41505.25 U 62 1.11
2022-02-24 05:00:04.472000       19072 38042.82 35122.81 D 18 0.92
2022-02-25 05:00:04.643000       19073 35122.81 38740.11 U 0 1.1
2022-03-01 05:00:06.659000       19077 37830.34 43388.77 U 3 1.15
2022-03-09 05:00:02.706000       19085 38554.62 41656.28 U 7 1.08
2022-05-06 04:00:02.780000       19143 39727.55 36446.18 D 57 0.92
2022-05-10 04:00:02.150000       19147 33556.09 30678.51 D 3 0.91
2022-05-12 04:00:02.124000       19149 31191.22 28234.93 D 1 0.91
2022-05-13 04:00:02.622000       19150 28234.93 30523.85 U 0 1.08
2022-06-14 04:00:02.293000       19182 25483.35 22074.01 D 31 0.87
2022-06-17 04:00:02.808000       19185 22182.06 20359.29 D 2 0.92
2022-06-19 04:00:02.056000       19187 20424.2 18282.06 D 1 0.9
2022-06-20 04:00:02.514000       19188 18282.06 19961.12 U 0 1.09
2022-07-08 04:00:02.559000       19206 20465.35 22125.65 U 17 1.08
2022-07-28 04:00:01.952000       19226 21076.25 23137.12 U 19 1.1
2022-09-14 04:00:02.511000       19274 22235.95 20324.97 D 47 0.91
2022-11-10 05:00:02.217000       19331 18242.1 16433.61 D 56 0.9
2023-01-14 05:00:02.777000       19396 18820.73 20901.76 U 64 1.11
2023-01-21 05:00:02.053000       19403 20991.31 22602.95 U 6 1.08
2023-02-16 05:00:02.079000       19429 22092.46 24634.33 U 25 1.12
2023-03-10 05:00:02.124000       19451 21737.7 19877.69 D 21 0.91
2023-03-13 04:00:01.980000       19454 20598.23 22326.45 U 2 1.08
2023-03-14 04:00:02.715000       19455 22326.45 24517.05 U 0 1.1
2023-08-18 04:00:02.811000       19612 28619.6 26395.38 D 156 0.92
2023-10-24 03:00:02.742000       19679 30389.89 34553.97 U 66 1.14
D: 46  2.4%
O: 1856  95.0%
U: 51  2.6%

 

Looking at the plot for daily and remember this does not mean the X axis is time, just moves, even though it does look similar to a price -vs- time chart. In this case both regression lines overlap 100%.

 

Large BTC Daily Price Moves
Large BTC Daily Price Moves

Looking ahead

This is a guess but looking at this I am inclined to think a pullback between the regression line termination at 40K and the termination of the D and E price lines at 30 and 35K would be something to keep and eye out for.

Alternative Daily Move Chart

Taking the geometric mean of the daily and weekly thresholds, the result is 0.133. This produces using a daily tick an output that has as many large price moves as the weekly chart and can be seen as an alternative while slightly different, lines up similarly on the chart. The moves are roughly 3 standard deviation moves in this case.

Start of Data: 2018-08-30
Lines of input data: 1955
Lines to process: 1955
2018-11-20 05:00:17.949000       17880 5495.99 4641.55 D 81 0.84
2018-11-25 05:00:13.877000       17885 4301.01 3632.02 D 4 0.84
2019-04-03 04:00:15.519000       18014 4177.0 5050.01 U 128 1.21
2019-05-14 04:00:15.992000       18055 7016.99 7963.14 U 40 1.13
2019-06-28 04:00:15.313000       18100 12821.64 11110.99 D 44 0.87
2019-10-26 04:00:17.489000       18220 7455.01 9599.17 U 119 1.29
2020-03-13 04:00:04.431000       18359 7659.09 5385.44 D 138 0.7
2020-03-20 04:00:04.240000       18366 5301.62 6163.32 U 6 1.16
2020-04-30 04:00:04.743000       18407 7821.73 9291.9 U 40 1.19
2021-01-06 05:00:08.853000       18658 30880.51 35456.93 U 250 1.15
2021-02-09 05:00:09.512000       18692 38618.01 46888.08 U 33 1.21
2022-03-01 05:00:06.659000       19077 37830.34 43388.77 U 384 1.15
2022-06-14 04:00:02.293000       19182 25483.35 22074.01 D 104 0.87
2023-10-24 03:00:02.742000       19679 30389.89 34553.97 U 496 1.14
U: 9  0.5%
O: 1940  99.3%
D: 5  0.3%

 

Large-BTC-Daily-Moves-w-Linear-Regression-0.133
Large-BTC-Daily-Moves-w-Linear-Regression-0.133

 

Code

Code is on Github, it is quite a hack as it was made from pieces of other backtest code and production code that has not been cleaned or refactored.

https://github.com/erickclasen/cbpro-cli-tools/blob/main/uod.py

Misato Katsuragi

Kelly Criterion

If money is your bread then nothing else matters.

The Kelly Criterion is something anyone who trades, invests or even gambles must know. I have a brief summary below and some additional resources at the end of the summary.

There is also a great book devoted to this topic, Fortune’s Formula by William Poundstone, a must have book for any serious investor.

The Kelly Criterion, also known as the Kelly strategy, Kelly formula, or Kelly bet, is a mathematical formula designed to help people determine the optimal size of a series of bets. It’s named after John L. Kelly, Jr., who introduced the concept in a 1956 paper titled “A New Interpretation of Information Rate.”

The Kelly Criterion is commonly used in gambling, investing, and other areas where decisions involve uncertainty and risk. The goal of the Kelly Criterion is to maximize the long-term growth of capital by finding the optimal fraction of capital to invest in each opportunity.

The basic formula for the Kelly Criterion is:

Kelly-Critierion
Kelly-Critierion

The formula tells you what percentage of your current capital should be invested in a particular opportunity, given your assessment of the probability of success and the odds being offered.

It’s important to note a couple of things about the Kelly Criterion:

1. Risk of Ruin: The Kelly Criterion can be aggressive. Betting too much of your capital, even with a positive expectation, can lead to significant losses. The criterion doesn’t consider the risk of ruin, which is the risk of losing your entire capital.

2. Estimates are Crucial: The accuracy of the criterion depends heavily on the accuracy of your probability estimates. If your estimates are inaccurate, the strategy might not perform well.

3. Logarithmic Utility: The formula assumes logarithmic utility, meaning that the investor’s goal is to maximize the expected logarithm of wealth. This utility function helps to balance between risk and reward.

To use the Kelly Criterion, you would apply the formula to each opportunity, and the resulting fraction represents the proportion of your current capital that you should invest.

Keep in mind that many variations and adaptations of the Kelly Criterion exist to address specific situations and concerns. It’s a tool that requires careful consideration and understanding of the underlying assumptions and risks.

 

To learn more, see these resources

https://www3.cs.stonybrook.edu/~skiena/691/2007/lectures/Kelly.pdf

http://r6.ca/blog/20070816T193609Z.html

https://www.economist.com/media/globalexecutive/fortunes_formula_e.pdf

https://towardsdatascience.com/python-risk-management-kelly-criterion-526e8fb6d6fd