::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact

Rent Map Data Sources

October 29th, 2016
housing, map  [html]
I just finished updating my rent map to handle the recent Padmapper UI refresh, and someone asked how Padmapper not including Craigslist listings affected the map. This confused me: I had thought Padmapper got its data by buying it from 3Taps who scraped Google's cache who crawled Craigslist. But it turns out that Padmapper and 3Taps settled the lawsuit, and Padmapper has only gotten its listings from other sources since then.

One issue, though, is it could be that the cheapest apartments are listed only on Craigslist [1] and not on the other services that Padmapper pulls from. To get a rough check this, I took ten random listings from the Boston Craigslist page, and tried to figure out which Padmapper listing it went with.

Predictions summary:
Listing Estimate Error In Padmapper
$1700 $2605 +53% no
$1700 $2065 +21% no
$1495 $1775 +19% no
$3000 $3155 +5% yes
$3000 $2995 -0% no
$1575 $1590 -1% yes
$2900 $2800 -3% yes
$6000 $3650 -64% no (dubious)
While this is a small sample, it looks like the predictions are pretty good for the ones in Padmapper (which is what you would expect) and consistently too high (0%, 19%, 21%, 53%, avg=23%) for the ones not in Padmapper.

Fixing this is pretty tricky. I could do a larger sample to try to get a better sense of what the error is, and then adust my map down by the combination of how much lower the non-Padmapper apartments are and what fraction aren't in padmapper. In this case, ignoring the dubious listing, 4 of 9 weren't on padmapper, with an average error of 23%, that would mean adjusting all my estimates down by 10%. On the other hand, as people's listing behavior changes this could get obsolete pretty quickly, and it's a pain to calculate the first time let alone on an ongoing basis. Ideas?


[1] Or, worse for my map, listed only with signs in windows or something else not available online.

Comment via: google plus, facebook

Recent posts on blogs I like:

I’m Giving a Talk About Construction Costs Tomorrow

By popular demand, I’m giving the talk I gave 2 weeks ago at NYU, again. The database will be revised slightly to include more examples (like Ukraine, which I added between when I gave the talk and when I blogged about it), and I may switch around a few t…

via Pedestrian Observations December 2, 2019

Your room can be as bright as the outdoors

The effect was huge: I became dramatically more productive between 3:30pm and whenever I turned off the light. I estimate the lamp bought me between half an hour and two hours a day, depending on how overcast it was.

via benkuhn.net November 26, 2019

git-subtrac: all your git submodules in one place

Long ago, I wrote git-subtree to work around some of my annoyances with git submodules. I've learned a lot since then, and the development ecosystem has improved a lot (shell scripts are no longer the best way to manipulate git repos? Whoa!). Thus, I …

via apenwarr November 24, 2019

more     (via openring)

More Posts:


  ::  Posts  ::  RSS  ::  ◂◂RSS  ::  Contact