January 24th, 2008

Digg Demonstrates The Failure Of Completely Open Collaborative Networks

by

Digg is a great experiment in web “democracy” — a site where ANYONE can submit links to content and vote on links to their favorite content. The positive outcome of the Digg experiment has been demonstrating the power of “networked human intelligence” to filter the vast sea of content on the web and allocate attention to content on a scale only rivaled by search.

But Digg has also demonstrated that a completely open network will be subject to so much gaming and manipulating that it’s not possible to maintain that openness.

The choice Digg has faced is either:

  1. Allow the site to be taken over by an “elite” group power users and by bad actors seeking to game it, which ceases to be an open system
  2. Implement so many algorithmic and manual controls that the it ceases to be an open system

Yesterday, Digg announced a change to its algorithm that makes it clear they’ve gone with the second option:

As we point out in our FAQ, occasionally you will see stories in the upcoming section with 100+ Diggs – this is evidence of our promotion algorithm hard at work. One of the keys to getting a story promoted is diversity in Digging activity. When the algorithm gets the diversity it needs, it will promote a story from the Upcoming section to the home page. This way, the system knows a large variety of people will be into the story.

The algorithm change effectively holds back from the homepage any story that is Dugg by the same groups of friends, i.e. a group that is not “diverse,” which has been a principal dynamic for stories getting promoted to the Digg homepage. This is astonishing on the face of it — Digg’s struggle with gaming is so extreme that they had no choice but to band certain forms of collaboration in a system that is defined by its collaborative nature.

It’s also striking how Digg has had to spin this change, calling it an effort to promote “diversity,” as if it were some kind of affirmative action — which effectively it is.

This and other mechanisms that Digg has been forced to put in place to combat rampant gaming of its system have alienated many of Digg’s top users, who used Digg not because of its openness but because of the power they have to control a disproportionate percentage of homepage content — which gives them the power to control large volumes of traffic.

A new blog was set up to organize an open revolt among Digg’s top users — called “revoltnation,” a swipe at Digg founder Kevin Rose’s video show diggnation. In a post yesterday, the Digg revolutionaries outlined their grievances against Digg:

    1. Lack of communication and disregard for the Digg community
    2. Unexplained and unacknowledged banning of top users
    3. Lack of transparency – Digg only shows you the stories that people have dugg, but not the ones that are buried.
    4. The auto-bury list – For months, dozens of sites have been on an auto-bury list, often with no explanation whatsoever.
    5. Repeated and flagrant disrespect of its top users

I wrote last spring about the problem of Digg’s anonymous users, which is one of the defining elements of Digg’s open system and which has opened the door to gamers. The lesson of Digg appears to be that a completely open collaborative network, without a defined user base, cannot function as such. Digg’s “diversity” algorithm is attempting not to keep the system open but to limit and define it in a way that it can actually function.

The counterexample of a successful open collaborative network on the web is of course Wikipedia, but its well documented that Wikipedia isn’t a completely open system, and that a group of editors exerts a great deal of control.

Well, there’s always the web itself — that’s completely open, right? It is, but Google demonstrates that web is most valuable when defined and delimited — that’s the essence of PageRank, which gives the links on some sites more authority than others. And Google also wages a constant battle against gaming the open system of the web in order to game Google’s search rankings.

The future of the web and media lies in the middle ground between completely open networks and linear, command-and-control editorial systems.

  • The shocking reality: What REALLY goes on behind the curtains at digg.com >> http://supaswag.blogspot.com/2...

  • If people are gaming your system, then you don't have an open system anymore. Your initial stated options for Digg cancel each other out. "The will of the people" naturally includes "the will of all people," not just "the will of those people with enough time on their hands," and naturally adjustments will need to be made to make sure the latter group's vote is weighed appropriately. I get the impression you're making the assumption that one man = one vote = democracy. With Digg, one man can equal many, many votes, obviously, and if their data skews too far from public opinion, it's useless. Hence the countermeasures.

  • @comment 16

    If a website has as much traffic as Digg's - then there is a story there.

    Anybody who thinks this was just about the algorithm change didn't actually read any of the complaints.

    Banned accounts for no reason.
    Secret editors, auto-burries, etc. It's about transparency. Can you imagine if Flickr or Del.icio.us had rumors of banned accounts with no explanation? People would be up in arms.

    I'm not saying these rumors are true: But they are very prominent rumors - and that was the cause of things.

  • I agree with comments 13 and 15 -- there is nothing "non-open" about Digg's reform. In fact, this new algorithm can probably be justified by information theory:

    1) A person who Diggs everything is not providing any information about the quality of the content: it's just like grade inflation.
    2) A large group with perfectly coordinated behavior does not provide any more information that a single individual from that group.

  • You call that a revolt?

    Dave,

    The reason there were no reporters is because there was no story. The vast majority of the community did not care about your so-called "revolt". The digg about you guys whining getting close to 2000 diggs while your "digg is a game, let's play" post getting buried is proof of that. Reporters look for news, not manufactured publicity stunts propagated by crybabies.

    Here's something we all know that you guys might have not figured out yet: you need digg more than digg needs you.

blog comments powered by Disqus

Subscribe

Receive new posts by email

Recent Posts