Topix Weblog

January 20, 2004

Feedster is cool

by skrenta at 6:52 AM

I didn't quite get Feedster before launching topix.net and this blog. Why would I want to search blogs? But it's become an indispensable tool to me since our launch to keep track of public feedback about our system. I've begun using it to research what the online community is saying about other topics as well.

Searching the blogspace has replaced the role Usenet search used to fill. When most of the chatty online audience was on Usenet, a search tool such as DejaNews was needed to find out what people were saying about a subject. But online users who post have mostly left Usenet and moved onto blogs.

Regular search engines aren't very well suited to scanning for blog stuff, with their monthly crawls, spotty daily index coverage, and spam web results mixed in with the blogs. Instead, you need a crawl that's as up-to-date as possible (daily at least, if not hourly or better), and one that pays attention to scrubbing the crud out (akin to running the essential cleanfeed on a Usenet feed).

Plus I like the name Feedster. :-)

Should we add blog search to Topix.net? We include some blog results in our crawl, but nowhere near the number the dedicated blog search engines do. I'm undecided where inclusion of blogs in the mix of material we search and categorize should land on our to-do list. Getting all of the online police blotters we can reach onto our local pages is a higher priority for me at the moment...