The Daily Parker

Politics, Weather, Photography, and the Dog

Weather Now gazetteer now really, really fast

I mentioned over a month ago that, given some free time, I would fix the search feature on Weather Now. Well, I just deployed the fix, and it's kind of cool.

I used Lucene.NET as the search engine, incorporating it into the Inner Drive Gazetteer that underlies the geographic information for Weather Now. I won't go into too many details about it right now, except that I was surprised at how much the index writer was able to crunch and store (in Azure blobs). The entire index takes up 815 MB of blob space. That's so small a fraction of a cent per month I can't even calculate it right now.

The indexing process took about 6 minutes per 500,000 rows. (The entire database has 7.25 million rows.) It helped that I ran the indexing process on an Azure virtual machine, because at one point during index optimization I clocked the data throughput at 200 Mbps. Yes, two hundred megabits per second. The entire index ran in a little less than two hours on a VM while I was doing other things. And once the index initializes in the Weather Now app, searches only take a second or so.

Go ahead. Try a search. Put in your ZIP code or the name of a prominent building near you.

I still have a lot I want to do with the application, including updating it to a responsive theme and MVC, but this is a pretty big leap.

Comments are closed