Harnessing Collective Intelligence

Overview

digg logo

The Digg logo

Henry Jenkins describes collective intelligence as “the ability of virtual communities to leverage the combined expertise of their members”, which accurately describes the collective spirit demonstrated by the users of the many online communities in existence today. In fact, “crowdsourcing”, “collective intelligence” and “the wisdom of the crowds” (a term originally coined by James Surowiecki in his influential book titled the same) are probably the buzzwords that most accurately describe the nature of Web 2.0 as it stands today. The theory behind all these terms is deceptively simple: large groups of “unwashed” people are in general smarter, wiser and better at solving problems than an “elite club” of experts.

Officially touted as a user-driven, collective content discovery tool, the website Digg aims at harnessing the collective intelligence of it’s user base to gather, filter and analyse content so the very best of the best can rise the top. The premise is that by bringing together literally millions of people to do the massive work of finding, submitting, categorising, reviewing, discussing and featuring news items, blog entries, articles, images and just about every bit of conceivable information that is to be discovered on the vast perpetual data flow of the World Wide Web, Digg would eventually surface the most interesting, most wanted and most relevant content—”the best stuff”, as voted by their online community.

Comparisons

Originally the brainchild of Kevin Rose, an American Internet entrepreneur and former TechTv co-host, Digg was first launched in December 2004 after an initial investment of $1000. From it’s humble beginnings it rapidly rose to an enormous success, becoming in a flash one of the most prominent and influential social bookmarking sites of the Internet, it’s user base growing exponentially, hitting the 2.7 million individual user account mark as soon as in 2008, according to JCG.org’s estimates.

The basic function of Digg is quite easy to grasp: after having logged in, the user is presented with the moment’s most popular stories on the front page. It is possible to browse stories, filter content, create customised categories, add comments to a particularly interesting story, “follow” each others activity (similar to Twitter) but most importantly, to “Bury” (down-vote) or “Digg” content (up-vote, very similar to Facebook‘s “Like” concept). In the beginning, this novel concept of voting content up or down was what set Digg apart from existing online social bookmarking offerings, a concept that prompted the creating of countless similar social networking sites with content submission and voting systems.

Implications

From the perspective of the user, Digg is an excellent tool to find content worth spending time reading, especially when taking user definable categories into account which enable the user to effectively create customised feeds that closely match their interests. In the heyday of the service being featured on the front page used to be every blogger’s dream and effectively the best way to increase traffic in an explosive way. It didn’t take long that the term “The Digg Effect” was coined (also known as the phrase “dugg to death”), which refers to the situation when the traffic generated by a particularly popular front page story overloads the website’s server, causing it to collapse under the large number of simultaneous users and thus becoming unavailable for period of time.

Potential legal and ethical issues

People often mistakenly believe that the content that rises to the top on Digg is indeed representative of what the majority of their user base thinks is important, but as it has been recently pointed out, in most cases this couldn’t be further from the truth. According to some recent statistical analysis, more than 20% of the content featured on the front page of Digg comes from a surprisingly small group of only about 20 users. Clearly, there seems to a discrepancy between the way Digg attempt to market themselves (self-organizing folksonomy, democracy of opinion, crowdsourcing) and the way their system actually works (“wisdom” derived from a homogenous monoculture, a microscopic “elite” group of privileged individuals). Truth to be told, Digg is well aware of this fact and even makes this information publicly available on their top users’ statistics page. It should be also noted that this problem is in no way particular to Digg only; other popular social bookmarking sites such as Reddit or Delicious (or as a matter of fact, even Wikipedia) exhibit exactly the same type of skewed user contribution statistics.

Future directions

As with all Web 2.0 sites whose success is solely dependent on the input of the people using the service to generate valuable content, the recent massive flock of users from social bookmarking sites to more ‘hip’ services such as Twitter and Facebook begs the inevitable question: are social bookmarking sites here to stay, or are their days already numbered, continuing their slow fade into irrelevancy? According to recent research results, bloggers are getting social media traffic from Facebook and Twitter mainly. As of January 2009, Twitter already had twice as many young users aged 25 to 34 as Digg. The reasons why users move to new services are highly complex and not always rooted solely in the usefulness of a particular piece of technology, but also—and, one could argue, even more so—in societal and fashion trends. At present, no one could tell for sure in what state social bookmarking sites will be in a year from now. Whether Digg and similar sites could regain their former glory, that is yet to seen.

References

What is Digg?
Wikipedia – Digg

Discover and Share Content on Digg
How Digg Works
Wikipedia – Kevin Rose
Harvesting the Collective Intelligence of Social Networks
Top 100 Digg Users Control 56% of Digg’s HomePage Content
Digg loses popularity contest to Reddit
Digg, Reddit, Netscape: The Wisdom of Crowds or Mob Rule?
Twitter Overtakes Digg in Popularity
Can Digg Apologize Its Way Back to Popularity?
Are Social Bookmarking Sites Dying?

Advertisements

9 thoughts on “Harnessing Collective Intelligence

  1. Hi Vivien,

    Nice article! Personally, I’m more partial to Reddit. Are there any big differences between the two services?

    Also, on the point about the ‘elite privileged few’, one could argue that there isn’t anything wrong with these 20 or so people making up 20% of the featured content. The fact that their names are recognisable means that people can source authoritative opinions by paying attention to these users who the collective community has effectively ‘dug up’ and elevated as such.

    The real implication to me, is that these users could use their influence to spread opinions that might be misinformed, and people might sway to it without considering stronger arguments from others.

    Are you a Digg user yourself?

    Cheers,

    Anthony Smith
    http://anthonysmith4it.wordpress.com

    • Hi Anthony, I’m glad you liked my post 🙂 Actually, I can’t say much about the differences between Reddit and Digg since I have used only Digg for a while in the past. By the way, I agree with your remark, misinformed high-ranking users being able to potentially influence public opinion is a real threat to the usefulness of the service.

  2. In my opinion, whether that web 2.0 site will suceed or not depends on how it controls its users. Of course, it will be very difficult to control them because users are never simple as you mentioned. However, it seems to be the most important task to be overcome.

    By the way, it will be more credible if you reveal its source when you use statistics or research results. 🙂

    • Thanks for your suggestion, the statistics and research results were already listed in the references section at the end, but I put a link to the source in the text, nevertheless. You are right, user management is an important issue and is among the biggest challenges to overcome in the web 2.0 applications of tomorrow.

  3. Hi Vivien,

    I really enjoyed reading your blog, particularly the section on future directions. Did you spend a lot of time thinking about this? I had a prediciton last year that Facebook users would slowly leave it for something new and exciting but it still has millions of users…. Like you say, it is a highly complex sciences and I guess if we could figure it out then we would be the owner of the latest and greatest Web 2.0 application. Maybe one soon we will be…… Who knows!

    • Hi Nicole, I’m glad you liked my post and yes, I have spent considerable amounts of time researching this since I’m quite new to this whole Web 2.0 business 🙂 You’re absolutely right, if someone can figure out what users of a particular existing service like Facebook are really missing, then she’s in a very good position to come up with the Next Big Web 2.0 Thing.

  4. Digg.com was a good example of harnessing collective intelligence of an online community, its sad seeing how much Digg’s user engagement has declined since Kevin Rose left in 2011 and amazing seeing the once underdog reddit.com completely crushing them in monthly visits with continuous growth the last few years.

    Reddit 14,449,596 visits per month Jan 2012
    Digg 4,830,311 visits per month Jan 2012

    http://www.quantcast.com/reddit.com
    http://www.quantcast.com/digg.com

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s