September 30, 2004

Cool site awards

by skrenta at 9:16 AM

In the pre-dawn of 1998 there was only Search Engine Watch. Danny Sullivan's pioneering daily newsletter had the best deep coverage of the search space, and there wasn't much else.

Post Google, we're now awash in coverage. The good stuff is still in out-of-the-way places though. Regular blog.topix readers know that I have a short list of key industry-watching sources that I regularly scan. (Plus this, of course. :-)

But some new entrants have recently punched through into the top-tier. Steve Rubel's Micro Persuasion just recently launched, but rapidly flashed to first-class status. A PR guy providing insightful analysis of the online industry? You bet. Maybe it's his unique perspective, but the guy digs up cool stuff, and provides sharp analysis and commentary.

Another blog that I've been obsessively checking is Greg Linden's Geeking with Greg. Greg's got technical depth, is working on news aggregation too, and has been posting up a storm lately.

September 27, 2004

Rich at Gnomedex

by skrenta at 10:32 AM

I'll be up at Chris Pirillo's Gnomedex the latter part of this week. Any fellow geeks in attendance who want to chat about rss, aggregation, doing AI in perl, or cheap linux terabytes and load balancers give me a ping. :-)

September 22, 2004 RSS Amazon product watch

by skrenta at 7:29 AM

Carlo Zottman has put together a nifty-cool new RSS service, WatchCow will monitor wishlists and alert you when a watched item's price changes, or more used inventory for the items become available. Worth a look.

September 21, 2004 Powering Local News on Ask Jeeves

by skrenta at 7:27 AM

We're excited to announce that is now powering local news on Ask Jeeves. ASKJ is the first major search engine to feature's news technology. Our full press release appears in the extended entry.

Ask Jeeves Taps to Deliver Local News

Major Search Engine Selects's Local News Technology

PALO ALTO, Calif. - September 21, 2004 -, the Internet's Largest News Site, today announced that it has been selected by Ask Jeeves, Inc. to provide news to Ask Jeeves ( for its Local News channel. Through this partnership, Ask Jeeves becomes the first major search engine to feature's news technology. provides visitors to Ask Jeeves access to local news gathered from over 7,000 sources. Unlike any other news aggregator, provides localized news for over 32,500 US and international localities. Ask Jeeves' utilization of includes the use of NewsRank(tm),'s algorithmic story editing technology, which improves the user experience by leveraging a set of semantic category filters to improve the relevancy of the news selection in favor of major and interesting stories.

"Connecting our users to local news with is something we're very excited about here at Ask Jeeves," said Jim Lanzone, Senior Vice President of Search Properties at Ask Jeeves, "Getting relevant and timely local news to our users is an important part of our local strategy."

For partners interested in subject specific or locally relevant news, licenses its news aggregation technology, giving any web site the ability to integrate contextually relevant content onto their site, increasing page views and repeat visitors. distributes news organized by topic and location for over 150,000 categories, enabling web sites to be able to feature news that matters to their users.

"We're excited to provide Ask Jeeves. users with neighborhood-level local news," said Chief Executive Officer Rich Skrenta. "By featuring the news we aggregate from every newspaper, radio and TV station, and thousands of other sources, Ask Jeeves users will be able to quickly find the news that matters to them."

Powering news for over 1,500 web sites, has the richest and most comprehensive syndicated news solution available today.

About is the Internet's largest news site, providing news about over 150,000 subjects, including news for every locality in the US, every major sports team, every public company, and subject based news pages on industries, health, entertainment and thousands of other categories. aggregates from over 7,000 news sources, and distributes its 150,000 RSS news feed, as well powering the site itself. was founded in 2002 by an experienced Silicon Valley technology and management team that built the Open Directory Project (acquired by Netscape) as well as executives from Sun, AOL, Netscape, and cofounders of Terraspring, Spoke Software and the Internet Commerce Group at Sun Microsystems. is headquartered in Palo Alto, California. Contact: Mike Markson, [email protected], 650-493-9100 x115

September 20, 2004

Why online newspapers require registration

by skrenta at 9:24 AM

John Dvorak, in a PC Magazine article (found via Greg), wonders why newspapers put up registration barriers to reading their articles:
I have to conclude that the typical newspaper in this country does not want you going on its Web site, and deliberately creates a barrier in order to prove to the shareholders that the Web is losing them money. It's a feeble attempt to emphasize the printed version of the paper at the Web site's expense. The Web seems to be just something that newspaper people feel they have to do because everyone else is doing it.
This is yet another example of net head-scratching about why online newspaper sites don't seem to be pursuing "normal" success goals for websites -- increasing traffic, return users, and so on. Putting up a registration barrier clearly devastates traffic -- check out this excerpted Alexa graph of one online newspaper that went registration in April. So why are they doing this? To collect better marketing info? But banner ad networks can't make use of any of that data. The media kit has reader demographics, but it's easy to get those with a survey.

Two factors are pushing newspapers into putting up registration gates on their content: an innovator's dilemma between print and online ad sales, and the high value of obtaining email addresses for email list marketing (i.e., spam).

First, there is an innovator's dilemma at work between the print and online ad sale at newspapers. The print ad has been around forever, and is expensive, since paper and ink need to be consumed, and trucks have to carry the result all over town. The paper's new website can't justify the same cost of sale; it may not have as many users yet, the ad impressions it does have are tracked far more effectively than the print newspaper's (does anyone really see your ad on page 22 of section E?), and since moving physical paper and ink around isn't involved, the actual impression is naturally a lot cheaper.

So newspapers are often throwing the online ad in for free with a print ad sale. What this does is value the online ad at zero. And if online ads are worthless, then incremental traffic to the website is worthless too.

In fact, website traffic is worrisome, since it indicates users are shifting from the print newspaper, which is where all the value of the ads being sold is. At this point, the success goals for the newspaper's website have been completely undermined. Not only does additional traffic not yield additional revenue for the website, but success for the website is a barometer for erosion in the existing print business (as is apparently the case at the UK's Sun Online). This sets up formidable challenges for the online staff to work against to achieve success.

The second cause of registration site is that there are a number of aggressive email list marketers pushing registration gate solutions to newspapers. (They even approached us during our last tradeshow.) In short, newspapers are putting up reg gates in order to collect your email address and demographics to rent for spam. The more intrusive the ad product, the more it pays ... and email lists are worth $300 CPM per rental.

A typical notice on the registration page will say something like:

"From time to time, we will send you e-mail announcements on new features, products and services from [us] and selected advertisers and affiliates...

Please check this box if you do not wish to be contacted directly by a non-affiliated advertiser.... HOWEVER, we will continue to include you in our e-mail announcements, as explained above. Information is used as described in our Privacy Policy."

Once you sign up there, you can be emailed for marketing purposes by anyone who rents the email list from the newspaper, forever. You have agreed to receive third-party marketing messages from affiliates of the newspaper. In other words, anyone the newspaper's email list manager rents their list to. Email lists are worth a lot of money. More than they can make from pop-ups, or AdSense, or having their salespeople try to sell ads on the website to local businesses.

The combination of these two factors means that newspapers don't receive value from incremental users on their web sites -- or keeping existing users there. Since online ad impressions are valueless to them, and since, once registered, you've permanently transferred the economic productivity of your email address to them, it doesn't matter whether you ever come back to the site.

This isn't meant to be a condemnation of sites that have put up registration gates. In many ways, it's a completely rational decision when seen in the context of the newspaper business, and current online advertising market. The question of whether they make good long term strategic sense remains to be seen however.

September 19, 2004

Some updates from the robots vs. humans front

by skrenta at 11:28 AM

More to come...

September 17, 2004

Content Value Bundles

by skrenta at 11:03 AM

Tim Oren goes deep on the economics and value proposition for content bundles and ponders how aggregators and RSS are reorienting the landscape for users and advertisers.

September 11, 2004

by skrenta at 12:10 AM

Greg Linden of Findory expands his staff and adds syndicated results for Findory and Blogory.

September 9, 2004

"RSS is Broken"

by skrenta at 8:24 AM

Scoble: "RSS is broken". MSDN had to shut off full-text RSS feeds because of skyrocketing bandwidth costs. (via Dave)

Perfect Search

by skrenta at 12:07 AM

John Battelle asked me to speculate about Perfect Search.
Imagine the ability to ask any question and get not just an accurate answer, but your perfect answer -- an answer that suits the context and intent of your question, an answer that is informed by who you are and why you might be asking. The engine providing this answer is capable of incorporating all the world's knowledge to the task at hand...
Immediately a bunch of sci-fi imagery flooded my mind. I was going to contrast the moral lessons between Danny Dunn and the Homework Machine and Colossus: The Forbin Project. I pictured Beverly, alone on the bridge, talking to the Enterprise Star Trek computer, trying to reason out her predicament with her friendly savant assistant.

But hold on a sec. Everyone on the planet is walking around with a Star Trek communicator, but it doesn't seem like a big deal anymore. Is the Oracle of all human knowledge going to be any different?

We've got the entire world's libraries wired into a little device on everyone's desk. All anyone needs is the right library reference string to access any document. But that's not good enough! It's sometimes hard to track down documents in the world library. And then after you find them, you have have to read & study them, even reason based on them. Laziness spurs productivity, so we ask: Is this automatable work?

Part of the tech bargain seems to be that intelligent helper-humans get too expensive, but we get a big boost in personal productivity to make up for it. You can't delegate to your personal assistant (most can't afford one) but you have a collection of whiz-bang tools instead.

But the confidant, the sounding board, the Doctor Watson to Sherlock Holmes... we need a tool for that, since all of us modern Sherlocks can't afford Watson's anymore, and are sitting at home alone in front of our library terminals. The tool that we can think out loud to, and will echo back the right spur to the key connection.

Nah, that's all junk. Premature. No Hal yet.

The next rev is an interpretive semantic overlay on the web, with the ability to manipulate results during the search/processing event.

Direct Mail folks computerized way back in the 70's, to data-mine postal address databases for insights to who would buy Alaskan cruises, or steaks in the mail, or insurance. Silicon Valley marketing types might look down their nose at these old-school east coast database marketers, but those guys know SQL. They enjoy browsing relational databases looking for insights.

Say you run every sentence on the web through a part-of-speech tagger, isolate noun phrases, classify them (person? place? public company? drug?) and have a collection of intention-classifying patterns.

Blabble is doing part of that, and is doing part of that. Whizbang Labs was going to do it on a big scale. (This is all old-hat to AI academics, but commercial applications are always more modest and incremental than what can be imagined.)

Now throw some kind of SQL like database language on top, so you can issue queries to find out the most popular baby names this year, from indexing birth announcements on the web and in blogs, or to be alerted about new developments at a competitor -- reading every job board everywhere, as well as monitoring every public communication from any employee, or individual closely associated with an employee, at the target firm. Blogs, org charts, resume databases, business card fishbowl scans, social networking/'s all there, waiting to be parsed & mined.

Before we can have our intelligent-assistant DWIM mind-reading turing-complete search engine, we'll have something ghastly like SQL on top of a semantically-tagged representation of the web. 20 minutes of manual searching and reading will be turned into a few seconds of work. Hal can be built on top of those APIs, but skilled humans will use them first. It will be way cool to search with.