January 24th, 2008

Digg Demonstrates The Failure Of Completely Open Collaborative Networks

by

Digg is a great experiment in web “democracy” — a site where ANYONE can submit links to content and vote on links to their favorite content. The positive outcome of the Digg experiment has been demonstrating the power of “networked human intelligence” to filter the vast sea of content on the web and allocate attention to content on a scale only rivaled by search.

But Digg has also demonstrated that a completely open network will be subject to so much gaming and manipulating that it’s not possible to maintain that openness.

The choice Digg has faced is either:

  1. Allow the site to be taken over by an “elite” group power users and by bad actors seeking to game it, which ceases to be an open system
  2. Implement so many algorithmic and manual controls that the it ceases to be an open system

Yesterday, Digg announced a change to its algorithm that makes it clear they’ve gone with the second option:

As we point out in our FAQ, occasionally you will see stories in the upcoming section with 100+ Diggs – this is evidence of our promotion algorithm hard at work. One of the keys to getting a story promoted is diversity in Digging activity. When the algorithm gets the diversity it needs, it will promote a story from the Upcoming section to the home page. This way, the system knows a large variety of people will be into the story.

The algorithm change effectively holds back from the homepage any story that is Dugg by the same groups of friends, i.e. a group that is not “diverse,” which has been a principal dynamic for stories getting promoted to the Digg homepage. This is astonishing on the face of it — Digg’s struggle with gaming is so extreme that they had no choice but to band certain forms of collaboration in a system that is defined by its collaborative nature.

It’s also striking how Digg has had to spin this change, calling it an effort to promote “diversity,” as if it were some kind of affirmative action — which effectively it is.

This and other mechanisms that Digg has been forced to put in place to combat rampant gaming of its system have alienated many of Digg’s top users, who used Digg not because of its openness but because of the power they have to control a disproportionate percentage of homepage content — which gives them the power to control large volumes of traffic.

A new blog was set up to organize an open revolt among Digg’s top users — called “revoltnation,” a swipe at Digg founder Kevin Rose’s video show diggnation. In a post yesterday, the Digg revolutionaries outlined their grievances against Digg:

    1. Lack of communication and disregard for the Digg community
    2. Unexplained and unacknowledged banning of top users
    3. Lack of transparency – Digg only shows you the stories that people have dugg, but not the ones that are buried.
    4. The auto-bury list – For months, dozens of sites have been on an auto-bury list, often with no explanation whatsoever.
    5. Repeated and flagrant disrespect of its top users

I wrote last spring about the problem of Digg’s anonymous users, which is one of the defining elements of Digg’s open system and which has opened the door to gamers. The lesson of Digg appears to be that a completely open collaborative network, without a defined user base, cannot function as such. Digg’s “diversity” algorithm is attempting not to keep the system open but to limit and define it in a way that it can actually function.

The counterexample of a successful open collaborative network on the web is of course Wikipedia, but its well documented that Wikipedia isn’t a completely open system, and that a group of editors exerts a great deal of control.

Well, there’s always the web itself — that’s completely open, right? It is, but Google demonstrates that web is most valuable when defined and delimited — that’s the essence of PageRank, which gives the links on some sites more authority than others. And Google also wages a constant battle against gaming the open system of the web in order to game Google’s search rankings.

The future of the web and media lies in the middle ground between completely open networks and linear, command-and-control editorial systems.

Comments (27 Responses so far)

  1. Disrespect of the top users? I would argue that they haven’t done it enough. It’s laughable to think that users who’ve submitted 5,000 stories or voted on even more care about quality anymore. The inflation of diggs it takes to get to the top is a direct result of that. And ultimately that alienates the other 2 million users–who in the aggregate mean much much more to the site than those 5 or 6 elite people whining in a podcast.

  2. [...] you’re out at lunch. And on bad days… This seems to be the basis of a piece over at Publishing 2.0, which argues that Digg – the original open social voting system – has all but forgotten its open [...]

  3. What this proves is something that has been known (and resolutely ignored by pundits) for quite some time: that the network effect is not cumulative.

    People keep portraying ‘the wisdom of crowds’ as though it were some sort of democracy – people vote, and whomever has the most votes wins. That’s how Digg operated.

    But the failure of Digg is analagous to the failure of democracy. The ‘wisdom of crowds’ is not obtained by mere voting. What is required – as the new Digg algorithm explicitly recognizes – is diversity.

  4. Your piece starts with a great survey & critical look at Digg’s recent moves, but the close is rather disappointing, no?

    The ol’ middle ground conclusion…yawn.

  5. I played a small part in this little fiasco last night – I participated as a community member, not a journalist.

    But I do think there was a lesson in there for reporters
    “”My question is – where were the tech reporters from the larger news organizations? They found out about this from ValleyWag, VentureBeat and Mashable the next morning. Those were the beat bloggers – those are the reporters who got the scoop.”

  6. [...] Carp of Publishing 2.0 has written a better explanation of the digg’s new algorithm, and makes some good points as [...]

  7. Mat,

    Middle ground may not be sexy, but does have the virtue, I think based on much empirical evidence, of being right.

    Dave,

    Indeed, there’s a big lesson for reporters — as sources move to online network, slamming phones isn’t going to get the scoop anymore.

  8. [...] of the site, of which the last tweak is only the most recent example — shows that a completely open social-media network is bound to fail, and I would agree. The only point I would make is that there has rarely ever been [...]

  9. [...] of the site, of which the last tweak is only the most recent example — shows that a completely open social-media network is bound to fail, and I would agree. The only point I would make is that there has rarely ever been [...]

  10. [...] go loco Digg Demonstrates The Failure Of Open Collaborative Networks The Long Term Benefit of Digg Digg Demonstrates The Failure Of Completely Open Collaborative Networks Digg Causes ED Among Social Marketers; Will it Now Go Limp Itself? What’s With the Digg [...]

  11. [...] a case study, Digg seems to be proving that a social news system is a tenuous proposition: “Digg has also demonstrated that a completely open network will be subject to so much [...]

  12. Eventually we start talking about somehow a regulated environment and we all know that could be more fatal than open network – i.e. let’s work on how to make open network a success rather than falling in right wingers’ hands?

  13. I think it may be worth considering what constitutes ‘completely open’ and/or ‘regulated’ in a network.

    All networks – including Digg – are constructed. All networks are therefore regulated, that is, the manner of their construction impacts their conduct.

    Perhaps we should say that to ‘regulate’ is to manage transactions in a network on a case by case basis, as opposed to ‘design’, which is the creation (or one-time adjustment) of network parameters.

    As for what constitutes ‘completely open’ in a network that has been designed, I am at a bit of a loss.

    Strictly speaking, ‘completely open’ would entail no design whatsoever, but that would also entail no network at all.

    We could say that ‘completely open’ means that any person may participate as fuly as anyone else. But if so, then the recent change by Digg does not change its status as ‘completely open’.

    I don’t have any faith in the press to actually comprehend any of these subtleties. But I think it would be nice were the press to move beyond empty slogans.

  14. [...] Scott Karp argues that completely open social networks fail. He takes two examples: Digg and [...]

  15. Its true that the internet is a vast sea of information difficult to harness without tools that aggregate the content in a meaningful way. However that which constitutes a meaningful representation of the body of information varies dramatically from one person to the next. It is almost as unique as our fingerprints.

    What is odd to me is that we are still seeing the predominance of the MEGA-AGGREGATORS like Digg and Google.

    I worry that Google has a strangle hold on content because it does serve as the primary aggregation of information for so many people around the world. It’s sort of given us Googley eyes through which we see the world according to Google’s logarithms…..which makes me uncomfortable.

    What Digg has done makes sense to me and I hope they stick with it. I also hope that people that are disaffected are able to find another aggregation that evolves to fill their niche. I what I really want is to see a greater diversity of aggregations. Where is the long-tail of aggregation channels?

  16. You call that a revolt?

    Dave,

    The reason there were no reporters is because there was no story. The vast majority of the community did not care about your so-called “revolt”. The digg about you guys whining getting close to 2000 diggs while your “digg is a game, let’s play” post getting buried is proof of that. Reporters look for news, not manufactured publicity stunts propagated by crybabies.

    Here’s something we all know that you guys might have not figured out yet: you need digg more than digg needs you.

  17. [...] I don’t Digg Britney Spears (in 50 words or less) Go read this article instead of wasting your time here. Suffice to say, I will not use any system that tells me what [...]

  18. [...] Digg Demonstrates The Failure Of Completely Open Collaborative Networks. Good article by Scott Karp with some good comments by Stephen Downes. Media [...]

  19. [...] Digg Demonstrates The Failure Of Completely Open Collaborative Networks – Publishing 2.0 (tags: digg transparency community) This entry was written by delicious and posted on at 3:33 am and filed under delicious. Bookmark the permalink. Follow any comments here with the RSS feed for this post. Post a comment or leave a trackback: Trackback URL. These icons link to social bookmarking sites where readers can share and discover new web pages. [...]

  20. I agree with comments 13 and 15 — there is nothing “non-open” about Digg’s reform. In fact, this new algorithm can probably be justified by information theory:

    1) A person who Diggs everything is not providing any information about the quality of the content: it’s just like grade inflation.
    2) A large group with perfectly coordinated behavior does not provide any more information that a single individual from that group.

  21. @comment 16

    If a website has as much traffic as Digg’s – then there is a story there.

    Anybody who thinks this was just about the algorithm change didn’t actually read any of the complaints.

    Banned accounts for no reason.
    Secret editors, auto-burries, etc. It’s about transparency. Can you imagine if Flickr or Del.icio.us had rumors of banned accounts with no explanation? People would be up in arms.

    I’m not saying these rumors are true: But they are very prominent rumors – and that was the cause of things.

  22. [...] lleva a pensar en que si no habrá un pequeño “fracaso” de este modelo y lo digo por este articulo que también cuestiona el sistema democrático alegando que no es tan democrático como parece ya [...]

  23. [...] Digg Demonstrates The Failure Of Completely Open Collaborative Networks [...]

  24. [...] Digg Demonstrates The Failure Of Completely Open Collaborative Networks Conceivably Related Posts“Are Editors Moribund?”BBC NEWS | The Editors | That’s the [...]

  25. If people are gaming your system, then you don’t have an open system anymore. Your initial stated options for Digg cancel each other out. “The will of the people” naturally includes “the will of all people,” not just “the will of those people with enough time on their hands,” and naturally adjustments will need to be made to make sure the latter group’s vote is weighed appropriately. I get the impression you’re making the assumption that one man = one vote = democracy. With Digg, one man can equal many, many votes, obviously, and if their data skews too far from public opinion, it’s useless. Hence the countermeasures.

  26. [...] Digg Demonstrates The Failure Of Completely Open Collaborative Networks [...]

  27. The shocking reality: What REALLY goes on behind the curtains at digg.com >> http://supaswag.blogspot.com/2008/02/can-you-digg-it.html

Add Your Comment

Subscribe

Receive new posts by email