|August 19th, 2013|
|boston, housing, map, tech [html]|
There are two scripts involved:
$ crontab -l ... # fetch the data at 2:02am on the 18th of the month; update the maps # at 2:02 on the 19th. 02 02 18 * * python /home/jefftk/query_padmapper.py 02 02 19 * * python /home/jefftk/draw_heatmaps.shThe first, query_padmapper.py, pulls apartment data from PadMapper and saves it in a timestamped file like apts-1376823721.txt. Unless Eric changes something, this just does its job. If something goes wrong, cron sends me an email with the error message.
(I really like this method of doing background tasks. Unless they're critical, don't try to recover from errors. Crash, print something informative, and have it show up in my email.)
The second script is a wrapper around draw_heatmap.py. It's safe to run any time because it looks for apartment data dumps that haven't been processed yet, but I intentionally only run it when I know it needs ot do something. It is:
The update_index function changes a small piece of index.html which has a list of which dates have data available:
So now the page stays up to date without me doing anything. Or else I wake up to an error in my email and figure out why.