September 17th, 2008

Why Every News Site Should Put a Continuously Updated News Aggregation on the Homepage


My post on Drudge beating all other news sites on engagement was an aha for many, which is interesting because the lesson of Drudge has been around for a decade. But the lessons of web publishing are all so utterly counterintuitive that I suppose they take a while to sink in.

That said, a number of commenters took issue with the conclusion that Drudge’s engagement metrics are meaningful, and that there are any useful lessons for other news sites.

I’m going to respond by upping the ante — I think the lesson of Drudge is that every news site should put a continuously updated aggregation of links on their homepage.

Here’s how the argument to my original post went. Several commenters, including  Ian Lamont from The Industry Standard pointed out:

The auto-refresh rate (about once every three minutes, I believe) throws a wrench into comparisons with other sites that don’t auto-refresh. Some advertisers and outsiders may be impressed by Drudge’s high pv/visit numbers, but those are inflated by people who keep their Drudge window open while visiting links in other tabs.

To which I responded:

What news site wouldn’t want to be open in a reader’s browser being refreshed all day?

And then I asked:

Why can’t news sites be a destination for original content AND links?”

To which Tim Buden responded:

Because they perform two essentially different functions. One is a destination, one is a starting point. You’re simply not going to get the same behaviour.

People go to a story page on some news content site after having found the link on some aggregator or portal. They do not hang around on the news site’s front page, refreshing the page and hoping for new links.

And while outbound links on the story page may well be useful, they’re not going to hang out on that page hoping for new links on the story either. That’s a job for aggregators.

And that’s when the lightbulb went on.

Most new sites are at the wrong end of the value chain.

Most news site are just a bunch of individual content pages that people land on FROM SOMEWHERE ELSE and where they quickly leave to go SOMEWHERE ELSE.

That’s why Google derives more economic value on the web than any other media company — because Google figured out how to be the starting point. And they did it by offering the WHOLE web, not just a tiny slice of it.

Most news sites offer a tiny slice of the web, which consists of their own content.

Aggregators offer the WHOLE web.

Most news sites — in fact, most content sites — are just a brief stopping point as users surf the web, neither a beginning nor an end.

I read NYTimes.come every day, but I only visit it once or twice a day, because unless there is breaking news, the homepage doesn’t really change much. There’s no reason to go back.

But just because has no new content on its homepage doesn’t mean news on the web is standing still.

In contrast, I visit Techmeme multiple times per day, because there is ALWAYS new news. In fact, I visit more often via a link on Techmeme than I do going to directly.

And then there are the hundreds of news sites that I visit ONLY when a site like Techmeme or Google sends me there. But I won’t go back to these sites until I get sent there again. But I’ll ALWAYS go back to Techmeme or Google.

OK, so here comes the really counterintuitive part — a news site does NOT have to choose between being a pass-through content publisher and a starting point aggregator.

A news site can be BOTH.

Imagine if the put above the fold on its homepage a continously updated list of links to breaking news around the web — and then set the homepage to auto-refresh, like Drudge and Techmeme.

Instead of checking once or twice a day, I’d probably start checking it constantly… obsessively.

And each time I came… I’d notice any new NYT content, along with new links.

Or what if did the same thing? Instead of going to by default, I’d probably go to… and would become just one of the many places that sent me.

Or… what if,, for example, created this destination for links to news of what’s going in the world? Maybe I’d stop going to either the Post or the Times, because Newsweek would send me to the best stories from both sites… and many more.

Or… what if the Post’s LoudounExtra local site did the national/international news aggregation. Then I could find out what’s going on in the world — from ALL the best sources on the web — AND find out what’s going on in my community, all in one place.

Bottom line is… I’ll ALWAYS go back to the site that constantly updates with links. It’s the way I can derive the most value from the web as a news source.

New sites need to ask themselves — how can we create the most value for news consumers on the web? The evidence is pretty darn clear.

So is anybody going to take my advice? The problem is what I’m suggesting runs so completely counter to traditional publishing that many editors would literally have to put their heads on backwards in order to devote a portion of their homepages to news aggregation.

But I still hold out hope the newspaper editors, for example, will come to the realization that in print they are already in the aggregation business — but on the web, they are merely in the content business.

And on the web, as in print, all the economic value is in the aggregation business.

The tools exist for newsrooms to collaborate in creating a continuously updated news aggregation on the homepage — all that’s required is for a news site to decide it makes both editorial and economic sense to be in the aggregation business, to be a starting point for news.

P.S. Just for kicks, here’s the latest Hitwise ranking for news sites — check out Drudge’s market share:

P. P. S. It just occurred to me after posting this why Drudge has such a huge audience — because Drudge has NO COMPETITION!

Geesh. You’d think some highly trusted traditional news brand would roll up their sleaves and take on Drudge.

Comments (21 Responses so far)

  1. Scott, the piece they are all missing is the cycle.

    It’s not a matter of start and end — it’s where you come back.

    The smart news site/aggregator establishes itself as the reset switch. Not a gatekeeper, but a terminal hub. You know they’ll come back eventually because you’ve got an easy place to switch to the next interesting train.

  2. Like SeekingAlpha where market currents links out and is continuingly refreshing! and other stuff is aggregated on site!

  3. [...] now ups the ante in a new post urging traditional news sites to start automatically aggregating the news on their homegpages. There is, he says, no rival to Drudge and as most homepages don’t [...]

  4. I’ve been reading your past few articles on the matter of linking and so on, and I’d like to agree with you, in part, by offering you a story on the current state of the Romanian blogosphere.

    To put it succintly, I’m doing a PhD on the influence of the internet in Romanian politics, since a number of politicians have started popping up with personal blogs and websites in the past few years. I gathered all the political blogs I could find that were connected, in some way, to politics, I subscribed to them using Google Reader and I exported the different categories, parties and so on as individual RSS’s that people could subscribe to in bulk. At the same time, I placed one of those Google Reader shared items widgets on my blog, where I hand-picked certain posts and articles which I thought were interesting.

    A normal political blog in Romania clocks somewhere around 50-100 visits a day. My average is around 350. People come for information. I know it’s pretty laughable, in terms of traffic, but that’s the reality here. I just wanted to point out that people do like organized information, aggregated content and what not…


  5. It’s a good topic to raise but I think you’re being narrow in your outlook here. Rather than provoking traditional news sites into becoming full-on aggregators, I’d rather see a discussion of how aggregations of outbound links can be tweaked to provide a better service to the news site visitor above and beyond what they might otherwise find on drudge or googlenews etc.

    Surely there’s room for some lateral thinking that involves more than having either continual wall of changing links on the front page OR only publishing reporter articles with internal links.

    I would rather, for instance, read a news article that is followed by an aggregation of other outside news/blog sites (vetted somehow) covering the same topic. That would make me think that the site providing the article and the links was the starting point more often than not I suspect. I almost never click the staple additive internal links – “news topic du jour” – that accompany or are embedded in MSM news articles. Those selflinks are of little value.

    (by the way, NYT refreshes the main headlines/stories *at least* fairly regularly: hourly or so)

  6. I put together a quick demo awhile back of the NYTimes “river of news”-style, called SimplerTimes. It did two things you might find interesting: it shows you what’s been published throughout the day (to address that homepage not changing issue), and it lets you customize the sections of the paper you want to follow.

    It’s very basic on purpose — I just wanted to demo the idea that the NYTime’s articles alone could benefit from a reader-centric, aggregator-style presentation.

    While I 100% agree about the value of curating external links (this is the basis of much of the world’s blogging), I don’t think companies do enough with their own archives. There’s a lot of value to be had there, and it’s already bought and paid for.

  7. Hey all. I’m with NBC 17 in Raleigh, NC. We launched a new network of local websites this year for the area.

    We integrate aggregation into our sites as well as user generated content. Our theory is the users don’t care where the content comes from, they just want an easy way to be informed on their local area.

    Which makes relevancy the ultimate key.

  8. [...] Reader. Drudge Report: News Site That Sends Readers Away With Links Has Highest Engagement and Why Every News Site Should Put a Continuously Updated News Aggregation on the Homepage add to the continued debate around the production of and linking to content and the drawn out [...]

  9. The real thing that the NYTimes, for example, needs to figure out is what is there “linking voice”. What makes Drudge relevant is that it has a specific type of interest in specific types of stories– mostly political, preferably salacious, with a hint of the bizarre and a touch of outrage. If there’s a scandal brewing, everyone knows the important links will be on Drudge. You don’t go there for local news, sports, economic issues, etc.

    The Times or any other news source needs to develop an editorial voice with regard to linking just as they do with their articles’ editorial voice. Simply having updated links is not enough.

    For the NYT, it might be that their links are the “most important” sources of the day’s biggest and most important stories. You would go back to the NYT because you know their link selection will send you to “the source” and to the articles that hold the most weight among the people who matter. For the WaPo, it might be that they have the most in-depth local Washington stories and consistent links to sources you may not read every day, such as Roll Call, etc.

  10. Not specifically related to your overarching point, but here’s an interesting post on Drudge’s importance in the media landscape:

  11. I think you are halfway there – and then Erich gets closer still. The reason people go to Drudge is NOT just aggregation. It is also filtering. Drudge gets it that people don’t want automatic, brainless filtering – they want intelligent, interest-based filtering. Drudge initially got big by doing an O’Reilly-like right wing filter on the news which was unique. Then he got even smarter and now does a still slightly biased but much more oriented to interest level filter – and THAT is why people keep it open. Yahoo News is also popular because it does a GREAT job of aggregating and a decent job of filtering – but it doesn’t do a great job and has actually gotten significantly worse lately.

    The rally amazing thing is that ANYONE can do a great job in a niche with the feeds available and the smart feed manipulation tools out there. Take a look at for a simple but relatively smart health news aggregator in the central yellow column.

    But – I actually believe that the way forward is to do MORE than Drudge – sure do the aggregator thing – it’s easy. But also provide unique commentary, spin, original content, etc.

    The NYT has in place right now all the tools to do this and it amazes me that they aren’t doing better – but a close look reveals why – they actually use the tools they have very, very badly.

  12. I agree with your point that aggregation builds loyalty, and that this may be the reason for the Dridge’s success. But this doesn’t mean that mainstream sites have fewer loyal readers than the Drudge. It only means that loyal readers account for a smaller percentage of their audience, but they are able to monetise both. The important question is which of the two models is more profitable (in terms of rate-of-return) and scalable.

    I’ve written a series of posts about this – see

  13. [...] framed his previous challenge to news sites in general terms: like Drudge, any site could use continuously updated aggregation to [...]

  14. I’d say that Huffington Post is coming damn near to executing your prescriptive model. Not only do they have a frequently updated Drudge-style above-the-fold content block, they have a remarkably capable team of actual journalists led by Tom Edsall.

    They’re very comfortable with constantly update external links while very capably creating original content.

  15. I understand that I am a little late chiming in here, but I wanted to point out a misunderstanding of this data in some of the comments:

    “The auto-refresh rate (about once every three minutes, I believe) throws a wrench into comparisons with other sites that don’t auto-refresh… those are inflated by people who keep their Drudge window open while visiting links in other tabs.”

    The numbers you show are for “unique visits”. Most web analytics programs measure visits using cookies. When a user visits a site that site will attach a cookie to the users browser with a unique id. In general, the cookie does not expire for 30 days or more. Each unique cookie is counted only once. So “unique audience” is truly counting each person only once. “Sessions per person” speaks more to what your readers are talking about. The constant refreshing will skew these numbers. But as you alluded to, what news site wouldn’t want their readers having them open in their browser all day?

    I think there is a lot to be learned from these numbers, for more than just the news and publishing industry. Web retail business would be wise to take notice of this as well.

  16. Good!

    I often come back to this site for the simple reason that all that I have even done better these lasts time came from Scott’s oblique and analytic insights, but more from all of the comments contributed along any single post.
    Thanks for all, as anyone can use whole or part of them to adjust eventually, ease the way things can be done, serve any parties (the publishers, the audience & the site too) for a mutual benefit, avoiding false routes.

    I use to start with an idea, consider what I can manage and build outward by iterations. Someone may find it tricky, but I’m confortable with this.
    For now, I ignore details and it will only be a problem when it’s a problem, so I don’t waste time on problems I don’t have yet. May be by shrink my time, I should go more and more faster and relevant on what I’m doing
    What I learnt from the comments and count on:

    – A (pretender) site does not have to choose between being a pass-through content publisher and a starting point aggregator. It can be BOTH.
    – … someone will always go back to the site that constantly updates with links (as way to derive the most value from the web for news or content source). Brillant! Hope that this happen!
    – … you don’t have to be the dominant aggregator to realize the (economic) value of content (re)distribution.
    – ….people come back to places that send them away (work for sure for the big web moguls :-)
    decide to go for this route, experimenting with a few news (content) sites in different verticals (tech/gadget news, finance, women, fashion, cosmetics, travel, interiors, gourmet/gastronomy, astrology, video, headlines news, auto and sport news,….).
    – … users want their information quick and easy (we all knew that; the real thing is how to figure it out the best way)
    – …all that is needed is a bunch of decent news and content headlines (I added for my case the visual preview) to know if something’s pique’s your interest
    – … there are rooms for Google/Yahoo/MSN-type sites on smaller, more target scales (thank for highlighting)
    – …. people only would like a single content-hub (aggregator) in any given category. (hard to do, but I try in my case to make it relevant)
    – … every site has an opportunity to define its own category whether it is the localized version, unique niches or to simply do it better that the currents Google/Yahoo/MSN,… (this enforces that)
    – …. people want to see the (best) content (may be the more salacious), and they don’t care whether or not you produced it (I hope so :-)
    – ….they value good filtering and recommendation over second rate proprietary content (aha!!!)
    -…. it is difficult to convince of the high value of becoming a place where people (repeatedly) visit and then could walk away. But the value comes to my opinion in not being reluctant to link out. (damn TRUE :-)
    – … there are a bunch of (smart?) publisher who are starting to, not yet at Google News scale but with small steps in narrow verticals (may I pretend to be part of it, doing it my way?)
    -… by sending people away, a site guarantees that they will come back. (I hope so!)
    – …. It’s not a matter of start and end, it’s where you come back, …. people do like organized information, aggregated content (humm! I really hope so!)
    – …. for a site to establishe itself as the reset switch, not a gatekeeper, but a terminal hub, …users will more and more come back, because it should be an easy place to switch to the next interesting train. (Praise God :-)
    – … aggregations of outbound links can be tweaked to provide a better service to the news site visitor above and beyond what they might otherwise find on Google/Yahoo/MSN,… etc. (may be!)

    I’ve got these points, and again, thanks Scott for opening doors.
    Last, I just would like to introduce for those who will be transported to the site to have a look, that I have rolled out the ‘Loung TALKS’ that can be found in the menu TODAY -> Insights-> Lounge TALKS or, at this url

    It is by mean one of the features I have announced and planned to set up to bring life to SlavisQarats allowing users of all types to cast their voices in many topics and subjects even those that are not related to the site editorial structure.
    One of the advantage is to keep it simple, silly!
    Lounge TALKS is zen in discussion forums giving exactly what SlavisQarats needs and leaving out what I would avoid as having to manage a forums with rules, back-end contrainst and platform security concerns .
    Lounge TALKS’s layout is clear and absolutely easy for any user to understand, simple yet effective, what gets people talking about any subject.
    The main advantage in term of efficiency is that that allows it creates by itself an instant communityeven to simple visitors not registered, to interact, take part in discussions, ask questions, give answers and express opinions.

    I have longly searched ways to set up an AJAX discussion mean with these features and a self-moderated community, giving a better performance in terms of maintenance, and Lounge TALKS comes with a lightning fast and efficient database back end to provide real stability.
    Lounge TALKS’s functionality includes editing and tagging topics with a click, and no more waiting, what brings a considerable server load savings.
    It is easy self-ruling and allows users to discuss with each other on anything they like without the need for moderation, to define new topics with tags or keywords (only for those who desire to be registered). The tags system will help users find topics much easier, and for now I have pre-defines tags they can use.

    Quite 30 or more topics in often a salacious, provocative voire outrageous falvors have been set-up, and I hope there will be more (by anyone who feels so.
    To summarize, it makes it easy for users to communicate with each other, with open and no confusing categories, avatars, with the main advantage that it’s just 3-click-away aka community-forum, with a quick overview for users of all topics in a single mouse click.

    The more exciting is that it is very multipurpose with many features that can come along the more I discover stuffs, learn them, assimilate them and implement them!
    I invite you to start writing for example something in ‘anonymous confessions’, more, spread it around, (I really need first comments on those topics).


  17. [...] in Japan (TechCrunch) • Lynn Crosbie chats with uber-blogger Perez Hilton (Globe and Mail) • Content sites vs. aggregators (Publishing 2.0) • 8 tools for better bookmarking [...]

  18. [...] Still, attempting to build an audience for a page of links, as an influential destination, feels like a good bet to be making on the web. [...]

  19. [...] did not, the value of bringing their unique perspectives to filtering the web, of having a “linking voice.” They understood the editorial power of the [...]

  20. [...] Why Every News Site Should Put a Continuously Updated News Aggregation on the Homepage My post on Drudge beating all other news sites on engagement was an aha for many, which is interesting because the lesson of Drudge has been around for a decade. But the lessons of web publishing are all so utterly counterintuitive that I suppose they take a while to sink in. [...]

  21. [...] P.S. If you are further interested in reading about the topic of main stream media in general, I’d highly recommend Scott Karp’s Publishing 2.0 Blog. [...]

Add Your Comment


Receive new posts by email