Can Twitter Predict the Future? (of the Real Estate Market, and Otherwise)

Twitter is a great tool for finding out what’s hot, in real time. You can find trending topics right on Twitter’s new home page, or on cool tools such as Twitscoop. So the next question is: can you use it to find longer term trends – beyond the buzz of the moment?

What we have in mind is something like Google Insights for Search, where you use people’s search queries to figure out what’s interesting. Chart that over time and you can sometimes predict the future (for more information on that, check out Google’s blog posts on tracking flu trends and the predictability of search trends).
The real estate market heatmap that we launched today does something along the same lines, using tweets. We recommend that you check it out first, then come back to read how it works.

Try the interactive version here.

Twitter seems to be just as good (if not a better) indicator of what people think, need, want and care about. With two caveats: (1) unlike search queries, where people use keywords, on Twitter people use language, so it takes some nifty software to figure out their intent and (2) Twitter Search only shows relatively fresh information, so some information needs to be stored somewhere else to enable longer-term analysis.
Luckily, in DemandSpot we already take care of both issues: (1) our software uses semantic analysis to find people who say, in oh-so-many ways, that they’re looking to buy some real estate and (2) we store meta-data on our servers, so it’s still accessible even when you can’t find it on Twitter Search.

So here’s how it works:

1. As you may already know, in DemandSpot we find people who say on Twitter that they’re in the market for real-estate (see our home page). We also figure out where they’re looking (for instance, here’s a list of leads from around Boston).
2. Today we summarized the demand for each location for the entire month of July, then for August.
3. Then we compared the two to figure out which markets are hot – have the highest change in number of leads from month to month
4. Finally we overlayed the results on a map.

A few words on the data, and on necessary caution:

1. The results are based on over 10,000 Tweets per month. While a large sample, it’s still nowhere near the sample size that Google probably uses for its Insights for Search. Twitter is still a small phenomenon compared with Google (or Facebook for that matter).
2. Since Twitter itself is growing month-to-month, it’s possible that the growth in number of Tweets in a local market is the result of new people in that locality joining Twitter over the past month.
3. We only “read” English tweets. We have some data for countries where English is not the primary spoken language, but I wouldn’t make much of it. The Countries that contributed most tweets to our sample over July and August are the US (10′s of thousands), Great Britain (thousands), Canada (thousands), Australia (several hundred).
4. While the US as a whole has lots of tweets, small local markets may have just a few. In building the heat map, we ignored localities where we found fewer than 10 tweets per month (but you can still find them on DemandSpot).
5. We hope that you find this map useful and interesting, but please understand that we give no guarantees as to the accuracy of the results, and won’t be held liable for decisions you make based on them.

So what’s next? That’s entirely up to you!
Don’t be shy, use the comments to let us know what you think, and what’s missing – we’d love to add more features that the community finds useful.

Advertisement