1. For each analysis I make a local directory, ~/work/YYYY-MM-DD--topic/. These contain large files I'm copying locally to work with, temporary files, and outputs. When these get too big I delete them; they're not backed up, and I can rebuild them from things that are backed up.

2. Code goes in a git repo, in files named like YYYY-MM-DD--topic.py. Most of my work lately has been going into an internal repo, but if there's nothing sensitive I'll use a public one. I don't bother with meaningful commit messages; the goal is just to get the deltas backed up. If I later want to run an analysis similar to an old one I duplicate the code and make a new work directory.

3. Code is run from the command line in the work directory, which means that in my permanent shell history every command I ran related to topic will be tagged with ~/work/YYYY-MM-DD--topic/.

For example, the code for the figures in my recent NAO blog post on flu is in 2024-09-05--flu-chart.py and 2024-09-12--rai1pct-violins.py.

This approach optimized for writing over reading, but maintaining enough context that I can figure out what I was doing if I need to. I'll usually link the code from documents that depend on it, but even if I forget to it's pretty fast to figure out which code it would have been from names and dates. Running git grep and histgrep get me a lot of what other people seem to get from LLM-autocomplete, and someday I'd like to try priming an LLM with my personal history.

Often something I'm doing moves from "playing around trying to understand" to "something real that my team will continue to rely on". I try to pay attention to whether I'm getting to that point and then start taking care of the code properly, in an appropriate repo with meaningful commit messages etc.