All posts by erickclasen

About erickclasen

This is my Blog for writing about technical items and my other thoughts. I also use it to experiment with WordPress. View all posts by erickclasen →

Thoughts on the best BTC price moves

January 25, 2024MoneyBitcoin, DCA, Linear Regression, trading, trading strategyerickclasen

As the great and powerful Satoshi Nakamoto once said, “If you don’t believe me or don’t get it, I don’t have time to try to convince you, sorry.”

For a while I have used a piece of code that started as back-testing code and forked it to produce output whenever BTC-USD makes a 2 standard deviation move in price up or down. Basically I manually adjusted the threshold that is normally a fluid value in backtesting to a fixed value, so that 95% of the time a U for Up, or D for Down move is not recorded, It’s counted as O, which means nothing interesting happened and is not printed. CRON runs this hourly and the machine responsible for trading runs 24/7 and collects prices, since August 28,2018. It even plays a sound when one of these outlier moves occurs.

Here are a few lines from the end of the output….

2024-01-03 16:00:02.618000 63874 42197.47 43004.35 U 2 1.02
2024-01-03 19:00:03.434000 63877 42923.71 42269.13 D 2 0.98
O: 43795 95.0%
D: 1165 2.5%
U: 1148 2.5%

The top two lines have a time stamp followed by a tick count, this is where it is in the data set, in order, adding 1 per hour. Next are the two prices, the starting and ending prices, U or D, signalling which way it just went. The next number shows how many hours have elapsed since the last radical move followed by the scale of the move. This is a rounded number that shows what number would be required to multiply the first price to get the current price, how big a move in other words.

2 Standard Deviations for Bitcoin

Daily

I recently adjusted the calibration threshold as BTC is settling down into less radical swings as time goes on. Currently it is….

THRESHOLD = 0.01459 # 0.0156

…which means on an hourly basis a 1.459% move registers as a 2 standard deviation move. It was 1.56% earlier this year, so volatility is trending down over time.

Can that be tradable, not likely as fees and slippage would most likely eat up the 1.459%, granted some higher swings do occur and I have pondered this. What is you bought all the big down moves and sold on the big up moves. Sounds good in theory and would most likely only work in an up market or else, you just wind up buying at bad entries and ‘bag hold’ too much at a loss until the next bull run.

Weekly

I got the idea to look at a daily and weekly time frame. Weekly is such a short output that I will paste it here….

Start of Data: 2018-09-05
Lines of input data: 279
Lines to process: 279
2018-11-21 05:00:11.243000       17809 6268.44 4452.68 D 10 0.71
2019-04-03 04:00:15.519000       17828 3976.88 5050.01 U 18 1.27
2019-05-15 04:00:15.266000       17834 5782.6 8063.78 U 5 1.39
2019-06-26 04:00:16.011000       17840 9139.01 12228.39 U 5 1.34
2019-07-17 04:00:14.972000       17843 12983.74 9411.06 D 2 0.72
2020-03-15 04:00:05.033000       17878 8749.01 5168.75 D 34 0.59
2020-08-02 04:00:06.489000       17898 9686.6 12000.0 U 19 1.24
2020-12-20 05:00:12.005000       17918 18865.0 23460.71 U 19 1.24
2021-01-03 05:00:14.866000       17920 26732.29 33231.68 U 1 1.24
2021-02-14 05:00:19.359000       17926 38365.06 47607.09 U 5 1.24
2021-03-14 05:00:09.349000       17930 49437.95 61191.88 U 3 1.24
2021-08-01 04:00:03.128000       17950 34192.29 42461.2 U 19 1.24
2022-06-19 04:00:02.056000       17996 27515.55 18282.06 D 45 0.66
2023-03-19 04:00:02.137000       18035 20598.23 27282.54 U 38 1.32
D: 4 1.4%
O: 264 95.0%
U: 10 3.6%

A few things to note:

There is asymmetry as the market has in general been rising since 2018, so there are only 4 ‘large’ down moves and 10 up. 1.4% weeks are up moves, 3.6% are down. The number after U and D is the weeks between moves.
Note the magnitude of the moves. They are truly massive. As the threshold is now 0.235.
Note that the 5 sequential up moves in a row 2020-2021 barely made in over the threshold at 1.24
Make note of the fact that Bitcoin is calming down over the years by looking at how many times per year it makes these epic moves.
Last but not least. I wonder if the what the linear regression of the prices would tell. Perhaps good targets for the future?

From now on, I will monitor this more carefully. As one can see, entry and exit although infrequent would have lead to stellar performance. I’ve been doing crypto TA and writing related code for almost 6 years now and it is still interesting that some new price relationships can be sussed out.

Looking ahead

It’s a guess buy based on the chart above it will be worth seeing if there is a pullback below the price of the blue linear regression termination, 37K-ish down to the termination of the D and E price lines 20598.23 27282.54. It will be interesting to followup on this.

Daily

For daily the THRESHOLD = 0.076, so that means greater that 7.6% moves are counted. Below I have pasted in the 2022 to 2023 results. 2022 being terrible and 2023 awesome. I let the reader ponder the readability of these price moves.

2022-02-05 05:00:03.175000       19053 37326.16 41505.25 U 62 1.11
2022-02-24 05:00:04.472000       19072 38042.82 35122.81 D 18 0.92
2022-02-25 05:00:04.643000       19073 35122.81 38740.11 U 0 1.1
2022-03-01 05:00:06.659000       19077 37830.34 43388.77 U 3 1.15
2022-03-09 05:00:02.706000       19085 38554.62 41656.28 U 7 1.08
2022-05-06 04:00:02.780000       19143 39727.55 36446.18 D 57 0.92
2022-05-10 04:00:02.150000       19147 33556.09 30678.51 D 3 0.91
2022-05-12 04:00:02.124000       19149 31191.22 28234.93 D 1 0.91
2022-05-13 04:00:02.622000       19150 28234.93 30523.85 U 0 1.08
2022-06-14 04:00:02.293000       19182 25483.35 22074.01 D 31 0.87
2022-06-17 04:00:02.808000       19185 22182.06 20359.29 D 2 0.92
2022-06-19 04:00:02.056000       19187 20424.2 18282.06 D 1 0.9
2022-06-20 04:00:02.514000       19188 18282.06 19961.12 U 0 1.09
2022-07-08 04:00:02.559000       19206 20465.35 22125.65 U 17 1.08
2022-07-28 04:00:01.952000       19226 21076.25 23137.12 U 19 1.1
2022-09-14 04:00:02.511000       19274 22235.95 20324.97 D 47 0.91
2022-11-10 05:00:02.217000       19331 18242.1 16433.61 D 56 0.9
2023-01-14 05:00:02.777000       19396 18820.73 20901.76 U 64 1.11
2023-01-21 05:00:02.053000       19403 20991.31 22602.95 U 6 1.08
2023-02-16 05:00:02.079000       19429 22092.46 24634.33 U 25 1.12
2023-03-10 05:00:02.124000       19451 21737.7 19877.69 D 21 0.91
2023-03-13 04:00:01.980000       19454 20598.23 22326.45 U 2 1.08
2023-03-14 04:00:02.715000       19455 22326.45 24517.05 U 0 1.1
2023-08-18 04:00:02.811000       19612 28619.6 26395.38 D 156 0.92
2023-10-24 03:00:02.742000       19679 30389.89 34553.97 U 66 1.14
D: 46 2.4%
O: 1856 95.0%
U: 51 2.6%

Looking at the plot for daily and remember this does not mean the X axis is time, just moves, even though it does look similar to a price -vs- time chart. In this case both regression lines overlap 100%.

Looking ahead

This is a guess but looking at this I am inclined to think a pullback between the regression line termination at 40K and the termination of the D and E price lines at 30 and 35K would be something to keep and eye out for.

Alternative Daily Move Chart

Taking the geometric mean of the daily and weekly thresholds, the result is 0.133. This produces using a daily tick an output that has as many large price moves as the weekly chart and can be seen as an alternative while slightly different, lines up similarly on the chart. The moves are roughly 3 standard deviation moves in this case.

Start of Data: 2018-08-30
Lines of input data: 1955
Lines to process: 1955
2018-11-20 05:00:17.949000       17880 5495.99 4641.55 D 81 0.84
2018-11-25 05:00:13.877000       17885 4301.01 3632.02 D 4 0.84
2019-04-03 04:00:15.519000       18014 4177.0 5050.01 U 128 1.21
2019-05-14 04:00:15.992000       18055 7016.99 7963.14 U 40 1.13
2019-06-28 04:00:15.313000       18100 12821.64 11110.99 D 44 0.87
2019-10-26 04:00:17.489000       18220 7455.01 9599.17 U 119 1.29
2020-03-13 04:00:04.431000       18359 7659.09 5385.44 D 138 0.7
2020-03-20 04:00:04.240000       18366 5301.62 6163.32 U 6 1.16
2020-04-30 04:00:04.743000       18407 7821.73 9291.9 U 40 1.19
2021-01-06 05:00:08.853000       18658 30880.51 35456.93 U 250 1.15
2021-02-09 05:00:09.512000       18692 38618.01 46888.08 U 33 1.21
2022-03-01 05:00:06.659000       19077 37830.34 43388.77 U 384 1.15
2022-06-14 04:00:02.293000       19182 25483.35 22074.01 D 104 0.87
2023-10-24 03:00:02.742000       19679 30389.89 34553.97 U 496 1.14
U: 9 0.5%
O: 1940 99.3%
D: 5 0.3%

Large-BTC-Daily-Moves-w-Linear-Regression-0.133

Code

Code is on Github, it is quite a hack as it was made from pieces of other backtest code and production code that has not been cleaned or refactored.

https://github.com/erickclasen/cbpro-cli-tools/blob/main/uod.py

Kelly Criterion

January 7, 2024Money, TradingGambling, Investing, Kelly criterion, tradingerickclasen

If money is your bread then nothing else matters.

The Kelly Criterion is something anyone who trades, invests or even gambles must know. I have a brief summary below and some additional resources at the end of the summary.

There is also a great book devoted to this topic, Fortune’s Formula by William Poundstone, a must have book for any serious investor.

The Kelly Criterion, also known as the Kelly strategy, Kelly formula, or Kelly bet, is a mathematical formula designed to help people determine the optimal size of a series of bets. It’s named after John L. Kelly, Jr., who introduced the concept in a 1956 paper titled “A New Interpretation of Information Rate.”

The Kelly Criterion is commonly used in gambling, investing, and other areas where decisions involve uncertainty and risk. The goal of the Kelly Criterion is to maximize the long-term growth of capital by finding the optimal fraction of capital to invest in each opportunity.

The basic formula for the Kelly Criterion is:

The formula tells you what percentage of your current capital should be invested in a particular opportunity, given your assessment of the probability of success and the odds being offered.

It’s important to note a couple of things about the Kelly Criterion:

1. Risk of Ruin: The Kelly Criterion can be aggressive. Betting too much of your capital, even with a positive expectation, can lead to significant losses. The criterion doesn’t consider the risk of ruin, which is the risk of losing your entire capital.

2. Estimates are Crucial: The accuracy of the criterion depends heavily on the accuracy of your probability estimates. If your estimates are inaccurate, the strategy might not perform well.

3. Logarithmic Utility: The formula assumes logarithmic utility, meaning that the investor’s goal is to maximize the expected logarithm of wealth. This utility function helps to balance between risk and reward.

To use the Kelly Criterion, you would apply the formula to each opportunity, and the resulting fraction represents the proportion of your current capital that you should invest.

Keep in mind that many variations and adaptations of the Kelly Criterion exist to address specific situations and concerns. It’s a tool that requires careful consideration and understanding of the underlying assumptions and risks.

To learn more, see these resources

https://www3.cs.stonybrook.edu/~skiena/691/2007/lectures/Kelly.pdf

http://r6.ca/blog/20070816T193609Z.html

https://www.economist.com/media/globalexecutive/fortunes_formula_e.pdf

https://towardsdatascience.com/python-risk-management-kelly-criterion-526e8fb6d6fd

ChatGPT at it’s finest: writing short helpful code

December 28, 2023ChatGPT, Scripts, Trading20-day moving average, automation, Cryptocurrencies, cryptocurrency analysis, cryptocurrency screener, deep value, Mayer Multiple, Princeton-Newport, Python script, relative performance, screener code, sorting, ticker symbols, trading strategy, WordPresserickclasen

ChatGPT at it’s finest: writing short helpful code without having to do fancy work.

The following is a dialog with ChatGPT. I had this script that I would normally have to hand edit all of the ticker symbols by adding them to the end of the lines of code. I am sure there is some clever way using sed or awk but, I figured why not get ChatGPT to help.

ChatGPT then was also able to generate the Tags for this post,by feeding the post into a prompt and I used Stable Diffusion to generate the featured image using the blog title as the prompt. Sometimes it is good to use ChatGPT to proofread as well but, this post was short and simple enough not to do that.

I want to show you an output, in the left column are the ticker symbols for a number of cryptocurrencies, I want you to remember all of them as they will be used in the next question that I will pose…..

Currency Price    Available Hold    Avail in USD Held in USD
AAVE 98.16        0.0 0.0            0.0
ACH 0.020353        0.0 0.0            0.0
ADA 0.3859        0.0 0.0            0.0
ALGO 0.1341        0.0 0.0            0.0
AMP 0.00248        0.0 0.0            0.0
ANKR 0.0244        0.0 0.0            0.0
ATOM 9.283        0.0 0.0            0.0
AVAX 20.66        0.0 0.0            0.0
AXS 6.8        0.0 0.0            0.0
BAL 3.68        0.0 0.0            0.0
BAND 1.417        0.0 0.0            0.0
BAT 0.21159        0.0 0.0            0.0
BCH 223.72        0.0 0.0            0.0
BNT 0.75        0.0 0.0            0.0
BOND 2.2        0.0 0.0            0.0
BTC 37964.39        0.0 0.0            0.0
CGLD 0.53        0.0 0.0            0.0
COMP 51.17        0.0 0.0            0.0
CRV 0.5658        0.0 0.0            0.0
CVC 0.1102        0.0 0.0            0.0
DAI 0.9999        0.0 0.0            0.0
DOGE 0.08033        0.0 0.0            0.0
DOT 5.263        0.0 0.0            0.0
ETC 18.86        0.0 0.0            0.0
ETH 2054.84        0.0 0.0            0.0
FARM 32.73        0.0 0.0            0.0
FET 0.5347        0.0 0.0            0.0
FIL 4.545        0.0 0.0            0.0
GLM 0.2418        0.0 0.0            0.0
GRT 0.1483        0.0 0.0            0.0
ICP 4.605        0.0 0.0            0.0
KNC 0.7065        0.0 0.0            0.0
LINK 14.588        0.0 0.0            0.0
LOOM 0.0673        0.0 0.0            0.0
LPT 6.33        0.0 0.0            0.0
LRC 0.216        0.0 0.0            0.0
LTC 69.78        0.0 0.0            0.0
MANA 0.4405        0.0 0.0            0.0
MATIC 0.7528        0.0 0.0            0.0
MIR 0.0941        0.0 0.0            0.0
MKR 1483.6        0.0 0.0            0.0
MLN 15.12        0.0 0.0            0.0
NKN 0.1064        0.0 0.0            0.0
NMR 15.68        0.0 0.0            0.0
OGN 0.12922        0.0 0.0            0.0
OMG 1.7217        0.0 0.0            0.0
OXT 0.0724        0.0 0.0            0.0
PLA 0.204        0.0 0.0            0.0
POLY 0.1963        0.0 0.0            0.0
QNT 99.56        0.0 0.0            0.0
REN 0.0823        0.0 0.0            0.0
REQ 0.083        0.0 0.0            0.0
RLC 1.4794        0.0 0.0            0.0
RLY 0.0126        0.0 0.0            0.0
SKL 0.0436        0.0 0.0            0.0
SNX 3.392        0.0 0.0            0.0
SOL 58.2        0.0 0.0            0.0
STORJ 0.7658        0.0 0.0            0.0
UMA 1.953        0.0 0.0            0.0
UNI 6.161        0.0 0.0            0.0
USD            0.01 0.0
USDC            0.0 0.0
XLM 0.118572        0.0 0.0            0.0
XTZ 0.817        0.0 0.0            0.0
YFI 8519.94        0.0 0.0            0.0
ZEC 29.03        0.0 0.0            0.0
ZRX 0.39338        0.0 0.0            0.0

I don’t have the ability to remember information from one interaction to the next, but I can certainly help you analyze or work with the data you’ve provided in this session. If you have a specific question or task related to this cryptocurrency data, feel free to let me know, and I’ll do my best to assist you!

Can you print out all of the symbols in the left column?

Certainly! Here are all the ticker symbols from the left column:

objectivec
AAVE, ACH, ADA, ALGO, AMP, ANKR, ATOM, AVAX, AXS, BAL, BAND, BAT, BCH, BNT, BOND, BTC, CGLD, COMP, CRV, CVC, DAI, DOGE, DOT, ETC, ETH, FARM, FET, FIL, GLM, GRT, ICP, KNC, LINK, LOOM, LPT, LRC, LTC, MANA, MATIC, MIR, MKR, MLN, NKN, NMR, OGN, OMG, OXT, PLA, POLY, QNT, REN, REQ, RLC, RLY, SKL, SNX, SOL, STORJ, UMA, UNI, USD, USDC, XLM, XTZ, YFI, ZEC, ZRX

If you have any specific tasks or questions related to these symbols, feel free to let me know!

OK, good, you remembered them. Now for the next step. I am going to show an example using btc and I would like you to create a listing with all of the ticker symbols from the left column. Example: $DIR/rsi-fractals-avgs-bbands-txt-summary.py btc

Sure, I understand. If you could provide an example of the desired output or clarify the specific format or information you’re looking for related to each ticker symbol (similar to the example you provided with btc), I’d be happy to assist you further.

I am looking for you to create code that has $DIR/rsi-fractals-avgs-bbands-txt-summary.py on every line but with the ticker symbols from the left column appear on that line right after it.

Certainly! It sounds like you want a list of commands where $DIR/rsi-fractals-avgs-bbands-txt-summary.py is followed by each ticker symbol. Here’s an example Python script that generates those commands:

python
# List of ticker symbols ticker_symbols = [     "AAVE", "ACH", "ADA", "ALGO", "AMP", "ANKR", "ATOM", "AVAX", "AXS", "BAL",     "BAND", "BAT", "BCH", "BNT", "BOND", "BTC", "CGLD", "COMP", "CRV", "CVC",     "DAI", "DOGE", "DOT", "ETC", "ETH", "FARM", "FET", "FIL", "GLM", "GRT", "ICP",     "KNC", "LINK", "LOOM", "LPT", "LRC", "LTC", "MANA", "MATIC", "MIR", "MKR",     "MLN", "NKN", "NMR", "OGN", "OMG", "OXT", "PLA", "POLY", "QNT", "REN", "REQ",     "RLC", "RLY", "SKL", "SNX", "SOL", "STORJ", "UMA", "UNI", "USD", "USDC", "XLM",     "XTZ", "YFI", "ZEC", "ZRX" ] # Directory variable DIR = "$DIR" # Generate commands commands = [f"{DIR}/rsi-fractals-avgs-bbands-txt-summary.py {symbol}" for symbol in ticker_symbols] # Print commands for command in commands:     
       print(command)

Replace $DIR with the actual directory you want to use. This script will generate commands for each ticker symbol, following the format you provided. If you have any specific adjustments or requirements, feel free to let me know!

OK how did it turn out?

This is the output, just as I expected and was looking for. Saved me probably about 15 minutes of tedious hand editing…

$DIR/rsi-fractals-avgs-bbands-txt-summary.py AAVE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ACH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ADA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ALGO
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ANKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ATOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AVAX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AXS
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BCH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BOND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CGLD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py COMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CRV
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CVC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DAI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOGE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FARM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FET
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FIL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GRT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ICP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py KNC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LINK
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LOOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LPT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LRC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MANA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MATIC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MIR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MLN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NKN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NMR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OGN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OMG
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OXT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py PLA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py POLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py QNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REQ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SKL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SNX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SOL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py STORJ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UMA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UNI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USDC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XTZ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py YFI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZEC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZRX

What’s it for?

It is screener code that will go into a script that prints out all the cryptos in the list sorted by their position relative to their 20 day moving average. It uses the same logic as used for the Mayer Multiple, except it is on a 20 day MA basis.

X = Current Price / 20DMA

Code

#!/bin/bash 
DIR="/home/erick/python/ta"


rm $DIR/output-rsi-er-price-sma20-multiple.csv

$DIR/rsi-fractals-avgs-bbands-txt-summary.py AAVE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ACH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ADA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ALGO
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ANKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ATOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AVAX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py AXS
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BAT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BCH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BOND
$DIR/rsi-fractals-avgs-bbands-txt-summary.py BTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CGLD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py COMP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CRV
$DIR/rsi-fractals-avgs-bbands-txt-summary.py CVC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DAI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOGE
$DIR/rsi-fractals-avgs-bbands-txt-summary.py DOT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ETH
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FARM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FET
$DIR/rsi-fractals-avgs-bbands-txt-summary.py FIL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py GRT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ICP
$DIR/rsi-fractals-avgs-bbands-txt-summary.py KNC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LINK
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LOOM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LPT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LRC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py LTC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MANA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MATIC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MIR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MKR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py MLN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NKN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py NMR
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OGN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OMG
$DIR/rsi-fractals-avgs-bbands-txt-summary.py OXT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py PLA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py POLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py QNT
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REN
$DIR/rsi-fractals-avgs-bbands-txt-summary.py REQ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py RLY
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SKL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SNX
$DIR/rsi-fractals-avgs-bbands-txt-summary.py SOL
$DIR/rsi-fractals-avgs-bbands-txt-summary.py STORJ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UMA
$DIR/rsi-fractals-avgs-bbands-txt-summary.py UNI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USD
$DIR/rsi-fractals-avgs-bbands-txt-summary.py USDC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XLM
$DIR/rsi-fractals-avgs-bbands-txt-summary.py XTZ
$DIR/rsi-fractals-avgs-bbands-txt-summary.py YFI
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZEC
$DIR/rsi-fractals-avgs-bbands-txt-summary.py ZRX


echo "Cur  ,Und,Targt,RSI, ER,20Ml,Mayer Mult."


# Sort by desending price/sma20 multiples
sort -nk6 -t, -r $DIR/output-rsi-er-price-sma20-multiple.csv

the script runs the rsi-fractals-avg-bbands-txt-summary code on all the tickers listed and saves output to…

output-rsi-er-price-sma20-multiple.csv

then sorts the output by the 20sma multiple column

Results

Best relative performers on top, worst on bottom. This idea comes from the book Fortune’s Formula and appears as a technique used by Princeton-Newport. Basically sell the top and buy the bottom ( deep value…maybe), or alternativly short the top and long the bottom. Or in the case of crypto the bottom ones might be just be duds that are always behind the pack, maybe don’t even bother with them.

Cur ,Und,Targt,RSI, ER,20Ml,Mayer Mult.
UNI ,USD,False,73,0.06,1.18,1.31
SKL ,USD,False,63,0.07,1.1 ,1.59
FET ,USD,False,67,0.14,1.09,1.97
SNX ,USD,False,63,0.09,1.07,1.43
GRT ,USD,False,60,0.05,1.07,1.37
AXS ,USD,False,56,0.0 ,1.07,1.2
AMP ,USD,False,63,0.05,1.06,1.17
MKR ,USD,False,60,0.13,1.05,1.34
AVAX ,USD,False,61,0.07,1.05,1.61
RLC ,USD,False,61,0.01,1.04,1.19
OGN ,USD,False,56,0.05,1.03,1.33
UMA ,USD,False,52,0.07,1.01,1.13
NMR ,USD,False,55,0.02,1.01,1.14
MANA ,USD,False,48,0.01,1.01,1.18
DOGE ,USD,False,55,0.02,1.01,1.15
BTC ,USD,False,54,0.14,1.01,1.27
REQ ,USD,False,52,0.02,1.0 ,1.1
NKN ,USD,False,59,0.02,1.0 ,1.1
ICP ,USD,False,51,0.04,1.0 ,1.14
FARM ,USD,False,53,0.04,1.0 ,1.31
ETH ,USD,False,47,0.04,1.0 ,1.13
DAI ,USD,False,44,0.01,1.0 ,1.0
ATOM ,USD,False,49,0.05,1.0 ,1.09
ADA ,USD,False,57,0.01,1.0 ,1.25
AAVE ,USD,False,52,0.08,1.0 ,1.38
STORJ,USD,False,46,0.13,0.99,2.0
SOL ,USD,False,51,0.19,0.98,2.12
ALGO ,USD,False,47,0.06,0.98,1.13
ACH ,USD,False,51,0.02,0.98,1.07
ZEC ,USD,False,43,0.04,0.97,1.0
XLM ,USD,False,43,0.05,0.97,1.03
LTC ,USD,False,42,0.03,0.97,0.89
LINK ,USD,False,46,0.17,0.97,1.8
DOT ,USD,False,45,0.01,0.97,1.08
COMP ,USD,False,49,0.05,0.97,1.11
QNT ,USD,False,43,0.02,0.96,0.98
PLA ,USD,False,45,0.02,0.96,1.19
OXT ,USD,False,47,0.01,0.96,1.13
ETC ,USD,False,39,0.0 ,0.96,1.07
CRV ,USD,False,47,0.08,0.96,0.92
BCH ,USD,False,36,0.1 ,0.96,1.08
LRC ,USD,False,42,0.08,0.95,0.99
FIL ,USD,False,41,0.0 ,0.95,1.14
CGLD ,USD,False,44,0.0 ,0.95,1.07
BAL ,USD,False,38,0.12,0.95,0.88
XTZ ,USD,False,38,0.03,0.94,1.03
MLN ,USD,False,36,0.05,0.94,0.9
BAT ,USD,False,38,0.01,0.94,1.07
BAND ,USD,False,44,0.01,0.94,1.12
ANKR ,USD,False,43,0.01,0.94,1.06
LPT ,USD,False,41,0.04,0.92,1.13
GLM ,USD,False,41,0.02,0.92,1.18
CVC ,USD,False,48,0.05,0.91,1.27
ZRX ,USD,False,36,0.09,0.9 ,1.68
YFI ,USD,False,42,0.04,0.9 ,1.32
MATIC,USD,False,28,0.03,0.9 ,1.1
KNC ,USD,False,27,0.03,0.9 ,1.11
BNT ,USD,False,32,0.08,0.9 ,1.56
POLY ,USD,False,30,0.01,0.81,1.2

ChatGPT will create Tags

Finally by posting this post into a prompt you can ask ChatGPT to generate Tags for the post. I used most of them but trimmed a few that were redundant.

Eastern Prickly Pear: Opuntia humifusa and Opuntia cespitosa

November 24, 2023PlantsBotany, Cactus care, Cactus flowers, Cactus in New York, Cactus species, Edible cactus, Garden tips, Gardening in New York, Hardy plants, Horticulture, Native Cactus, Native plants, New York flora, Opuntia cespitosa, Opuntia humifusa, Outdoor gardening, Plant propagation, Planting tips, Prickly pear, Succulents, Sustainable gardening, Wintering plants, Zone 4 gardeningerickclasen

All nature does is to make something happen.

Native Cactus in New York

Yes, there are really native cactus in New York and actually they cover an area from Florida to Canada and west to the Rockies. I obtained some years ago at the former Baker’s Acres near Ithaca, NY. They have done well through the years and the original clump had no problem surviving a transplant while moving. In fact the plant propagates easily from one of the paddles, officially called nopales, which are the stems of the plant, which have adapted thanks to evolutionary pressure into what functions more like a leaf. The leaves in fact have, via evolution become the spines and the tiny hair like spines that you really have to pay attention to.

They have striking flowers that will come out in mid-summer, large yellow flowers that attract pollinators. Once the flowers are done they will be followed by fruit that will be about one inch thick and two to three inches long. The plant requires a lot of sun to flower and it will take a few years from when it is first planted to flower.

Handling the fruit and plant

The trick with the fruit and the cactus in general is safe handling. I find tongs or pliers to work, channel locks are the best. The spines will go though leather gloves too easy . Kevlar might work, or some chain mail gloves, have not tried there.

Fruit

For the fruit, I find burning the spines off works, using a gas stove burner or some equivalent type of high heat source like a blow torch. These fruit initially start green but, will ripen to a red or purple color after a few months. They are edible and usually can be picked in November. The fruit does not have much ‘meat’ to it. Hard seeds and a little pulp.

Fruit Drink

I had 12 viable fruits this year and was able to make some delicious extract from them in order to make drinks. I held on to them in the refrigerator until the end o the year as they keep well, but finally just had to do something with them as I was risking spoilage.

After burning off the needles over an open flame. I cut open the fruit and used a spoon pressed hard against the skin from the inside to push the juice, pulp and seeds into a small blender. Then I blended it and used a small screen to filter the thick juice through. This mixed with fresh squeezed line, thinned with some water and with a bit of brown sugar made for some delicious drinks. I could see adding tequila to it as it would make for a great Margarita.

Growth habit and propagation

The cactus tends to spread easily in a clump formation, when it gets tall the paddles tend to lean over and touch the ground at some point. This allows them to root and spread, similar to other ‘walking’ plants. This tendency lends them to easy propagation from a paddle clipped off or by digging up a part of the clump. I have given out quite a few starters. Visitors are always surprised that a cactus can grow year long, outside in New York. The fact that it is native is just a mind blowing fact for most people.

Late Fall Eastern Prickly Pear, Note the fruit

Wintering

In winter the plant will collapse itself and fall to the ground, the paddles can dry out and look lifeless. This typically happens in New York in November. No worries, it comes back to life in spring again. It will survive in a Zone 4, so it is hardy but not quite as hardy as a Opuntia fragilis (Brittle Prickly Pear) . The brittle prickly pear will grow in the upper Midwest and up into British Columbia. This one it the next on my list to try growing here. https://www.minnesotawildflowers.info/flower/brittle-prickly-pear

Two distinct species

New York is actually home to two distinct species of cacti within the Optunia genus: Opuntia humifusa and Opuntia cespitosa — the latter of which was only recently recognized as a separate species within the state. In fact, it can be difficult to tell the difference between the two species except when they’re in bloom — the flowers of the Optunia cespitosa have a red or orange center.

“They’re fairly separated, although most of the Opuntia cespitosa is mainly south west of Kingston in more limestone areas. The other one, the humifusa, is along the Hudson River from Columbia County all the way down to Long Island,” says Steve Young, a botanist at the New York Natural Heritage Program.

Blooming Now: The Valley’s Own Native Cactus

Opuntia

Eastern prickly pear is in the Cactaceae (Cactus) family. This family contains about 1,800 species all but possibly one or two native to the New World. The prickly pears are considered an old group within the cactus family with about 150 species in Opuntia. It has the largest range of any cactus in the United States and can be found from New Mexico and Montana east to Florida and Massachusetts. It is also found in Ontario. Eastern prickly pear can form large colonies or occur as a few individuals in an area. In older botanical manuals, it is often listed as Opuntia compressa. https://www.fs.usda.gov/wildflowers/plant-of-the-week/opuntia_humifusa.shtml

https://plants.usda.gov/home/plantProfile?symbol=OPHU

Habitat

This plant is very intolerant of shade and instead thrives in sunny, hot and dry environments with well-draining, sandy soil.^[10] O. humifusa will grow in open areas in sandy, rocky and coastal scrub habits. It is capable of surviving cool winters unlike many cacti, although harsh winter storms are known to cause habitat loss.^[10]https://en.wikipedia.org/wiki/Opuntia_humifusa

The eastern prickly pear favors rocky terrain on ridges and summits and dry, sandy soils. It does well in desolate maritime areas and on cliff faces or outcroppings on hillsides. Like all cactuses, it has a shallow, expansive root system called a mat that allows it to quickly absorb water out of the surface of the soil it’s rooted in. Like many New Yorkers, its success depends on real estate.

Its success is the result of a bold leap northward—a journey made by some pioneering cactus fragment or seed long in the past, probably soon after the Wisconsin Ice Sheet began to recede from New York City some 25,000 years ago. It washed up, or was carried by an animal, and spread its shallow roots in our little corner of the country. https://hellgatenyc.com/new-york-nature-cactus

As a houseplant

It even can be used as a houseplant as can be seen in the following short video.

One more video to round it all out.

Makemore

November 2, 2023Machine Learning, Text GenerationAndrej Karpathy, art generation, artificial intelligence, Coding, Data Science, deep learning, Github, GitHub Repositories, GPT2, Grid Search, Language Models, Machine Learning, Model Optimization, NLP (Natural Language Processing), Python, Research, Tagging System, Text Generation, Transfer Learning, Transformerserickclasen

The best way to predict a future is to look for it. We live in the moment where the world is a vast machine for predicting the future.

This summer, I spent a significant amount of time contemplating large language models and delving deeper into their research. My first encounter with GPT-2 was back in 2019, where I explored its code and experimented with it. During this period, I became curious about transfer learning and its applications. Additionally, I had some prior knowledge about transformers, but it wasn’t as comprehensive as my understanding of LSTMs and RNNs. I couldn’t confidently explain what they did, for example.

While researching transfer learning with smaller models like GPT-2, I stumbled upon Gwern Branwen’s website (https://gwern.net/) and, in particular, his TWDNE Project (https://gwern.net/twdne). I found it clever because it combined a generative model for both images and text. I decided to focus on the text side of the project, as the image aspect was already well-addressed by applications like Stable Diffusion….

I might revisit the image style transfer aspect in the future, as I had previously explored it to some extent. You can find more about this in my “How to Generate Art Demo Followup.”

Before this, I had predominantly explored machine learning with code from the ground up using Python (PMLC). I have used ML practically in the form of genetic algorithms for tuning parameters on investing models for years, non-differentiable, so no chain rule! An offshoot was a project called gen-gen-algo, a generic genetic algorithm. Now, finally after all these side quests, I was ready to tackle something more complex and cutting-edge using GPT.

I found excellent resources on GitHub and in video format from Andrej Karpathy (https://github.com/karpathy). The following repositories were particularly helpful in my learning journey. The first one, “nn-zero-to-hero,” features a series of videos that provided a solid foundation in understanding transformers.

The second repository, “makemore,” served as my warm-up exercise to get back into working with transformers and Large Language Models (LLMs) after a period of dormancy in the field. You can access these repositories here:

1. “nn-zero-to-hero”: https://github.com/karpathy/nn-zero-to-hero
2. “makemore”: https://github.com/karpathy/makemore

Fork of makemore

My experience with “makemore” went beyond the basic examples provided in the original repository, which generated new names based on a dataset of names. Initially, my goal was to apply “makemore” to various datasets other than “names.txt.” I experimented with larger and smaller datasets, including those with extensive collections of English words, numbers for addition, square roots, and a substantial dataset of quotes containing nearly 10 million entries, some of which had lines as long as 505 characters. By using scripts and modifications to “makemore.py,” I conducted a grid search to optimize hyperparameters, including constraints on model size. Output from “makemore.py” was saved to a CSV file, along with hexadecimal hash values for easy tracking and analysis during the tuning process.

To further enhance the code, I introduced a grid search optimization method using a Bash script. This allowed for exploring the hyperparameter space while maintaining a ceiling on the model size. Without such constraints, optimization typically led to increasingly larger models that resulted in the lowest loss.

I also introduced the concept of assigning a random hexadecimal tag to the output of “makemore.py.” This tagging system facilitated the easy identification of the best loss and the associated set of hyperparameters that produced it. Additionally, I incorporated an early stopping mechanism into the “makemore.py” code.

If you’re interested in exploring my fork of Andrej Karpathy’s “makemore” code, you can find it here:

https://github.com/erickclasen/makemore

For a more detailed understanding, I’ve created a comprehensive “verbose-readme.pdf” that provides additional information:

Version on this site, opens in browser:

verbose-readme

GitHub Version requires downloading:

https://github.com/erickclasen/makemore/blob/master/verbose-readme.pdf

Calendar Tricks

November 2, 2023My ThoughtsCalendar, dates, days, Pythonerickclasen

Time is like dancing.

There are a few patterns in the modern calendar that can help you figure out days of the week through the year. I just happened to notice this first pattern once when looking at the calendar for the year closely.

9 Months of the year “line up” in a pattern with them starting on the same day of the week. This means that there days line up together.

So if you know for example that September 25th is on a Monday, Christmas will be on Monday as well. It is a quick and easy calendar cheat.

In non-leap years:
– January and October start on the same day of the week.
– February, March, and November start on the same day of the week.
– April and July start on the same day of the week.
– September and December start on the same day of the week.

In leap years:
– January, April, and July start on the same day of the week.
– February and August start on the same day of the week.
– March and November start on the same day of the week.
– September and December start on the same day of the week.

This type of pattern can be useful for various purposes, such as planning and organizing events that need to occur on specific days of the week or identifying months with similar day-of-the-week characteristics.

Mentally Calculating the Day of the Week

Also the calendar can have a so called anchor day of the week that appears in a pattern and allows quick mental calculation of a day of the week. This anchor day will ‘move’ through the week from year to year, one day forward for non-leap and two days for leap years. For example the anchor day is Tuesday in 2023, so all the following days of the year will fall on a Tuesday. For 2024, being a leap year the lower dates all fall on a Thursday.

All of the following dates have the same day of the week. From these days that anchor the day of the week it is easy to mentally find the day for any date in the month. It is possible to remember that weeks go back 7 and work to the week and then just add or subtract to get to the date that you want to now the day of week for.

Non Leap 1/3, 2/28, 4/4, 5/9, 6/6, 7/11, 8/8, 9/5, 10/10, 11/7, 12/12

Leap 1/4, 2/29, 4/4, 5/9, 6/6, 7/11, 8/8, 9/5, 10/10, 11/7, 12/12

This trick works to help nail down a few holidays too.

July 4th being 1 week before 7/11, falls on that anchor day as well. Halloween too. Christmas and New Years, the day before.

Need to know what date is Labor Day, remember that 9/5 is an anchor day, so just work forward or backward from there.

https://www.timeanddate.com/date/doomsday-rule.html

Calendars also repeat

A helpful thing to know is that calendars repeat with a pattern. So it is possible to look back or forward in time to figure out what dates and days line up. This trick also would allow you to reuse calendars, even antique ones. I once saw this in a government office I visited, the person had an old calendar up on the wall that lined up right for the current year.

https://www.timeanddate.com/calendar/repeating.html

I worked a bit with ChatGPT to help me figure out the pattern. It took making it write some code examples, getting it wrong a bit, me explaining what is wrong and then then it was finally right. For me it was a good exercise in prompting it and getting it to nail the problem. I find it useful to have ChatGPT provide code examples as this makes a good ‘language’ within which I can communicate back and forth with it. It can be as simple as telling ti that the output is wrong and where it is wrong, up to a type of collaboration on the code.

I think it came up with a fair explanation which I have pasted in after the code snippet.

def find_years_with_same_calendar(base_year, num_years=10):
    years_with_same_calendar = []

    while len(years_with_same_calendar) < num_years:
        if is_same_calendar(base_year, 2023):
            years_with_same_calendar.append(base_year)
        base_year -= 1

    return years_with_same_calendar

def is_same_calendar(year1, year2):
    # Check if the days of the week for January 1st match in both years
    return day_of_week(year1, 1, 1) == day_of_week(year2, 1, 1)

def day_of_week(year, month, day):
    anchor_day = datetime.date(2023, 1, 1)
    target_day = datetime.date(year, month, day)
    delta = target_day - anchor_day
    days = delta.days
    return (days + 1) % 7

import datetime

# Example: Find 10 years with the same calendar as the year 2023
years_to_check = find_years_with_same_calendar(2023, num_years=10)
print(f"Years with the same calendar as 2023: {years_to_check}")

ChatGPT on Repeating Calendars

Certainly! The pattern you’ve described and that the corrected code now follows is based on finding years with the same calendar as a reference year (2023). This pattern is based on matching the day of the week for January 1st in various years to determine if they share the same calendar.

Here’s how the pattern works:

1. **Reference Year (2023)**: This is the year you’ve selected as your reference point.

2. **Matching the Day of the Week**: The pattern involves matching the day of the week for January 1st in other years with that of the reference year (2023).

3. **6-Year and 11-Year Cycles**: The pattern alternates between a 6-year cycle and an 11-year cycle. In the 6-year cycle, you go back in time by 6 years, and in the 11-year cycle, you go back by 11 years.

4. **Leap Year Influence**: Leap years, which occur every 4 years, can influence the pattern. When you encounter a leap year in the cycle, it might disrupt the alignment by adding an extra day (February 29). Therefore, you need to account for whether the year is a leap year to correctly match the pattern.

The pattern essentially finds years that, despite differences in their actual numerical values, share the same calendar in terms of the day of the week for January 1st. This pattern repeats as you move back in time, allowing you to identify years with the same calendar as your reference year.

For example, you can see that 2017 is 6 years before 2023, 1989 is 11 years before 2023, and 1961 is 11 years before 1989. The pattern repeats with adjustments for leap years and results in a sequence of years with the same calendar.

guestbook.cgi

September 16, 2023Linux, Server, Unpolished GemsCGI, Check Guestbook, Guestbook, Perlerickclasen

The best way to understand a thing is to try to comprehend it.

I ran John Callendar’s guestbook.cgi on one of my servers. It wasn’t a very popular location on the web and the guestbook did not get abused fortunately! It would be very easy for comment spammers, trollers and the like to have a field day with it.
One thing I did to keep an eye on it and see if it got posted to was a script in /etc/cron.weekly to check for updates of the guestbook. It was located on a Raspberry Pi running Raspberian.
The script requires that ssmtp or some mail program is installed.

guestbook-check.sh

#!/bin/bash

# Checks to see if the Guestbook has been written to in the past week.
 # Sends out notification if it has been written to.
 find /usr/lib/cgi-bin/guestbookrev.txt -mtime -7 -exec mail -s "Guestbook Updated" myemail@myisp.net \;

The trick is in the find command where the option -mtime -7 means check to see if the file have been has a modification time of less than 7 days, if so, then execute via the -exec option, whatever comes next on the command line until the \;

Subject line

The mail command mails only a subject line of “Guestbook Updated”. It would be possible to have something in the body and even “cat” the guestbook in to send it in the email.

Auto de Spam

If the guestbook got abused by a spammer or something nasty it might be possible to run a script that would periodically do a cleanup on the file via a search and replace. Using a list of blacklisted words to search on and then replace them with a null character or space.

Reverse Order in Guestbook

It is possible to flip the posts on the guestbook as well if you want them ordered in the opposite order, Last In On Top versus the default of First In On Top. This is done by using…

 pop @all_entries instead of shift @all_entries

…in the code.

More on CGI and Perl

If you are new to CGI and/or Perl scripts be sure to check out John Callender’s tutorial that covers the workings of guestbook.cgi.
guestbook.cgi

Taking Control of Your Online Experience: The Case for Client-Side Content Filtering

August 12, 2023Machine Learning, My ThoughtsAdaptive Filtering, AI in User Experience, Client-Side Solutions, Content Customization, Content Filtering, Digital Wellbeing, Information Control, Media Literacy, Misinformation Mitigation, Online Experience, Privacy and Security, Social Media Ethics, User Empowerment, User-Centric Technology, Web Browsing Enhancementserickclasen

I love technology. We love movies, we love television, we love technology.
We are a society where technology is used in an almost inhuman manner.

It appears that social media companies are implementing filters to regulate content, with the intention of creating a more suitable experience for consumers and curbing the dissemination of misinformation. However, the COVID-19 pandemic revealed a potential for bias, suppression, and censorship in this process. Such manipulation could also extend to political motives, where information might be skewed to suit a particular agenda.

Control should be on the users side

This raises the question: Why employ content filtering on the server side when it could be done on the client side? Could users have settings that allow them to customize content filtering based on their preferences? There are valid reasons for such an approach. Users may wish to filter out redundant, repetitive, or annoying content, similar to how ad blockers or spam filters function, but on a more sophisticated scale.

This concept could involve a machine learning component, perhaps utilizing a Bayesian filter. Users could prime it with specific keywords, examples of unwanted posts, and other content to be avoided. This filter would reside within the user’s computer, phone, or even as a browser plug-in. By processing the content at the browser level, the filter could operate effectively, unlike at the router level where encrypted content would be inaccessible. This client-side filter would work similarly to a firewall, analyzing and potentially blocking or redirecting content based on predefined criteria.

AI in the users toolbox

Considering concerns about AI alignment and safety, it’s important to note that the genie is already out of the bottle. Machine learning resources and knowledgeable individuals are widespread. Thus, rather than trying to rein in AI development, the focus should be on leveraging it to the user’s advantage. While organizations like OpenAI strive for responsible AI, and governments aim to establish their standards, consumers require a means to navigate through the information landscape. Users would also want to choose their own alignment rather than have a government or corporation choose it for them as a form of soft censorship.

Fake out the Fake News

This approach could even encompass traditional media, which sometimes appears to convey biased narratives. To address this, a client-side filter could be developed. The name for such a tool is open to discussion—whether it’s dubbed a firewall, content blocker, or a “BS Defender.” Regardless of its name, the need for a user-configurable, trainable, and adaptive filter is evident. This filter could operate through reinforcement learning via human preference selection and incorporate Bayesian adaptive learning.

Next Steps

For someone with coding experience spanning four decades, the framework for such a tool is already something that I can imagine on a high level and it could be done in a few different ways. Right now, I am still thinking on it. The potential exists for this idea to evolve into a comprehensive white paper. On the coding side, a proof of concept could be created, perhaps using Python, to showcase the core filtering concept in action. Sharing this concept for consideration and exploration is crucial in a time when content filtering, covering everything from traditional media to social media and ads, is becoming increasingly important. An adaptable and user-driven filter, bridging the gap between the browser and user experience, holds immense value.

Linux-vs-Windows

August 1, 2023My Thoughts, Unpolished GemsLinux, Malaga Bay, Windowserickclasen

The greatest gift of all to mankind is the friendship and understanding that which we have cultivated with each other and in cooperation.

Nice site Tim. A little backstory on how I found myself here. I found your site while looking up Phillip S. Callahan after reading about him in Dan Barber’s Book, The Third Plate. You have some interesting info on him as well as what I have seen so far on calendar discrepancies.Clocks, calendars, precision timekeeping are other interests of mine and I enjoyed those posts. After that I checked out your categories and that led me here to this post.

I will be speaking from personal experience with what I have experienced on my machines and others that I have worked on. There is a bit of a chronology to this as well.
Back when Windows started, I was a late adopter. I stayed in the command line, the DOS world, until Windows 95. It was out when I was in college and I briefly had Win3.1 until I could install 95 on the machine I had at that point. At the same time I was using the universities computers, a bank of Win95 PCs was located in a convenient computer lab. The Internet was really coming on hard and fast, so the inevitable occurred, the room was packed to the gills with students and there was a waiting line most of the time. But, there was another computer lab mostly for computer science majors, full of Sun Sparcs running UNIX, barely used at all. The room was cooler and quieter too, a bonus. This was when I got a feel for what a non Microsoft OS could be like. I would up learning it enough to use it with fair competency, a struggle at time to remember how to do something at times, but worth the effort to stick with it as it ran so smooth. I wondered if there was anything like this that I could load on a PC. A few years went by and I started to do this with Linux.
The first few machines I used Linux on were set up with dual boot. Red Hat/W98 and later Ubuntu/XP combos on two separate machines, one after the other in time. Setting up Red hat was a pain at the time and not for anyone that is not “good” with computers. Ubuntu was easy to set up, almost as easy as setting up Windows. But, it was much easy to work with than the earlier Red Hat 9.0 and that was the key. It was easy enough for my non-technical minded spouse to use, she was not lost in it in other words and could actually could use it without a lot of questions or frustration. On top of that the performance of both machines was hands down better with Linux. Things like time from a cold boot to the time you could click and open a program were faster. More programs could be run simultaneously without bogging the machine down. Moving around on the screen and opening files went faster as well. On Linux there was minimal weird behavior and very infrequent total lockups, requiring a reboot. There was no degradation either. What I mean is that it seems after having a Windows install running on a machine for years and then loading programs on it one after the other over time, it seems to get more unstable and flaky over time to the point that a fresh install is needed. This has gotten better at least with Win 8, I have noticed. On a machine that I had after the XP/Ubuntu, one was to be the last Windows machine. A Xeon machine (XP/Lubuntu) that had 1GB RAM, it was expensive RDRAM and I chose to ride it out a while as is, Linux seemed to run a bit better with less memory. In other words it would take longer to hit the out of RAM wall and start to swap to the drive and when it did it was less aggressive and didn’t do a lockup for a long time like it did while running XP. A lockup meaning the time you have to just wait for the machine to start responding again as the disk just grinds. As I said, this was the final Windows machine for me, with expensive memory, it paid to toss the PC and get a newer used machine for the same amount of money as an upgrade. This is the machine that I am on now, 6 years old and running Mint XFCE. Right now I am actually composing this while on it running Slackware in a Virtual Box, to test it out a bit. She, my spouse, has basically the same machine, same age, same CPU, with Windows 7 ( after a brief try at Windows 10, which was short as the performance was sub-par, plus the fact that when it did updates it “inhaled” 100% of the bandwidth on my connection for long time periods was frustrating), the speed difference is quite noticeable between the two machines, Win7 vs Mint XFCE. On a cold start with Mint, I can click and open something like Firefox or Word Processor, as soon as the network card is recognized, about 9 seconds after boot. The Win 7 machine takes at least 3-4 times longer. It also performs much more sluggishly overall when it finally “arrives” after a few minutes. My estimate of the speed at which I can maneuver on the Win 7 machine is along the lines of equivalence to when I tried Ubuntu on a Pentium 4 machine, single core, circa 2004, so 14 years old. One final comparison. I had a neighbor with a new machine, a budget one, but new, with Windows 10 and it still moved a lot slower than the 6 year old machine that I have with Mint.

The difference in performance is just what I have experienced and motivated me to move to Linux 100%. Not to mention the stability as well, less odd behavior and virus and malware issues are bonuses. Linux has come of age, it once was a tool that was too technical for the common user but, at this point most people could get up to speed with it fairly quickly. A little learning upfront is an investment that will save time in the long run with all of the spare seconds saved over waiting for Windows to respond to human inputs.
Microsoft has had a few hits, XP and 7 come to mind, but the product seems to go off the rails badly almost every other release, Vista and 8 come to mind. I wonder why 9 was skipped, maybe it was going in the wrong direction early on and that was realized in house before launch, I don’t know the history with that.

To all the readers, happy computing to all, with whatever OS you run,
Erick

Windows Death Cross Malaga Bay

August 1, 2023Unpolished GemsLinux, Philip S. Callahan, Windowserickclasen

Imagination is the power to make a difference in yourself.

My comments on Windows -vs- Linux from Malaga Bay. https://malagabay.wordpress.com/

I thought this was worth a repost here. I came across the Malaga Bay site.
while doing some research on Philip S. Callahan. A very interesting fellow who studied among other things “why is it that crops which are grown on healthy soils never attract diseases and insects.”
https://malagabay.wordpress.com/2016/03/07/philip-callahan-paramagnetism/

https://malagabay.wordpress.com/?s=Philip+S.+Callahan

Microsoft has had a few OK releases in my opinion, Windows XP and 7 come to mind, the rest seem like they have been wrong turns or at least not fully baked in the oven of development and testing….
https://malagabay.wordpress.com/2016/12/30/windows-death-cross/comment-page-1/#comment-14768

November 2024
M	T	W	T	F	S	S
« Oct
				1	2	3
4	5	6	7	8	9	10
11	12	13	14	15	16	17
18	19	20	21	22	23	24
25	26	27	28	29	30