Topix Blog

January 8, 2008

Anonymous Comments – By The Numbers

by tolles at 4:01 PM

There has been a recent spate of hand wringing from public editors looking at the commentary on their own websites, despairing the quality of discourse, and generally criticizing the public of bad behavior. (Original articles from Miami Herald, Hartford Courant are no longer up, so I'm pointing to some commentary by Howard Owens. UPDATE: The Hartford Courant Editorial is up here still)

Anonymous commentary takes a beating within many of these editorials, with a lot of smoke but not much light in the way of data around the issues. So I figured it would be interesting to actually run some data on the quality of anonymous posting vs. registered users – as we have optional registration, and enough of both kinds of users to run an interesting sample on the posts that we killed on site.

Methodology and Assumptions

We have an AI based heat language detection based system to auto-moderate the posts prior to their being run, as well as a staff of human moderators which look over user flagged issues and handle complaints. Posts can be killed by either component to the system, and both are counted here.

This looks at one days worth of moderation on our posts. We could probably look to a larger sample, but this is probably a good representative sample

Topix registration requires an email confirmation, like most registration system, but most of the registered users started commenting anonymously, so the percentage of registered users is not comparable to a registration only system. Also, registered users aren’t authenticated to be rela people using their own names (which is representative of most registration systems).

The Data

Total posts: 83108 (visible: 78021)
Total by registered users: 22336
Total by non-registered: 60772

Posts killed: 5087
Posts by registered users that got killed: 992
Posts by unregistered users that got killed: 4095

% posts killed (overall): 5.1%
% posts killed (registered users): 4.4%
% posts killed (unregistered): 6.7%

Conclusions

While anonymous posts have a roughly 50% higher kill rate, they also account for 3X the comment and commenter volume. If one asks, “where are we getting the most acceptable comments from?”, the answer is clearly the non-registered user base. As pointed out above, that there are as many registered users on Topix is partially due to offering anonymous comments

Also, its important to note that the ability to manage “anonymous” commenters and “registered” commenters is equivalent from a moderation standpoint. It’s just as easy to identify someone by their IP address for the most part as it is through a registration system. While a 50% difference is certainly something to look at, it’s not an order of magnitude, and we’re also looking at a grand total of way under 10% of total commentary.

So, Where’s the Love?

You get much more in discourse, than you lose in “extra” banned commentary – so why the gnashing of editorial teeth over this issue? The “anonymous” issue is just a red herring. Really, what these journalists are threatened by is the nature of truly public discourse on the web. These people are not barbarians that appeared one day the net went up.

They’re your audience

Certainly, the editorial policy for the brand should be the decision of the folks that own that brand. That being said, systems that require registration get an order of magnitude less commentary than systems that don’t. If you have a human pre-screening every comment, you are going to spend a lot of money if you get any volume (which means you probably will constrain the volume), and if you do this by proxy by eliminating anonymous comments, what you’re really doing is just limiting the scalability of your system and keeping it small -- and you open yourself open to becoming displaced by someone with a more open editorial policy.

So, you can opt for not wanting to listen to what your audience wants to say, and keep most of them out of your brand. But looking at the numbers, eliminating anonymous comments is only going to limit the number of comments at the end of the day, not dramatically improve the quality of discourse on the web.

UPDATE: Howard Owens posts a response
UPDATE: Rich Skrenta and Scott Anderson from Tribune Interactive both have a couple of points to make