Wednesday, September 12, 2007

project for excellence in journalism lowers its standards, analyzes social news sites reddit and digg

Thanks to an article by Joe Strupp in Editor & Publisher, I found a recent study on social news sites from the esteemed Project for Excellence in Journalism. OK, I don't really know if they're actually "esteemed," but it sounded like it.

If Steve and I started another company, I think we ought to have the word "Excellence" somewhere in the name. It just oozes credibility.

Anyway, this study looked at a week's worth of news from three "user-driven sites" and compared them to the top stories on various mainstream outlets.

The results were shocking.

j/k!

"The result: online users gravitated toward different topics than those from traditional news outlets."

Citing the study, Strupp points out that "In a week when the mainstream press was focused on Iraq and the debate over immigration, the three leading user-news sites--Reddit, Digg and Del.icio.us--were more focused on stories like the release of Apple’s new iPhone and that Nintendo had surpassed Sony in net worth." Ouch. But the actual report is a bit more nuanced than Strupp would lead you to believe.

During that week, the immigration debate led the coverage, accounting for 10% of all news stories in the News Coverage Index. That was followed by coverage of a major fire near Lake Tahoe (6%), the failed bombings in the United Kingdom (6%), events on the ground in Iraq (6%), Supreme Court decisions (5%), the 2008 presidential election (4%), flooding in Texas (4%), the policy debate in the capitol over the war in Iraq (4%), U.S. domestic terrorism (3%), and the missing pregnant woman in Ohio (3%). In all, the top ten stories that week accounted for 51% of all the stories in the Index. Wow, if half the articles in an entire week were about one of ten stories, it explains that tingly feeling I get when serendipity drops a unique reddit link in my lap a couple times a day.

In the user-generated sites, these stories were barely visible. Overall, just 5% of the stories captured on these three sites overlapped with the ten most widely-covered stories in the Index (13% for Reddit, 4% for Digg, and 0% for Del.icio.us). reddit suddenly appears very mainstream... odd...

The immigration debate in Congress, the biggest single story of the week in the mainstream media, appeared just once as a top-ten story on Reddit, and not at all on Digg and Del.icio.us. Granted, reddit has a unique method for sorting its front page, so most of the stories will never get enough momentum (hotness) to rise all the way up into the top 10, but even with that aside, it's clear the reddit community is not mired in the popular-talking-point-of-the-week. Really, I watched CNN today, Britney is fat, I get it, but what about the immigration debate -- or has it been settled? Similarly, the war in Iraq accounted for 10% of all stories in the Index and seven percent in the Yahoo-user material. Across the three user-news sites, it amounted to about 1%. (bold text mine)
So what do you think? If you're using reddit, you're apparently getting broader dose of news, but is that really the goal for building a better source for finding out what's new.

There is some consistency in this evidently inconsistent world of user-driven news.
For the most part, there were no dominant [stories]. The only story with any real traction was the release of the Apple iPhone, and that was just on one site (it accounted for 16% of the stories on Digg that week)
16% of an entire week? Yikes. OK, Britney, you've still got a few more days.

Oh right, I didn't mention that Del.icio.us was included as one of the newfangled social news sites. Again, I've triple-checked my sources on this and Del.icio.us is still is not a social news website -- it's for bookmarking, which it does quite well. This ought to explain why the study found a "0% overlap" with the top 10 stories in the mainstream press. No one wants to bookmark the latest update on the fires threatening their ski lodge in Tahoe.

Moving on, here are some more select quotes from the study, chosen for optimal amusement:
"Seven in ten stories on the user sites come either from blogs or Web sites such as YouTube and WebMd that do not focus mostly on news."
WebMD? I have a feeling this is from bunch of del.icio.us users bookmarking herpes symptoms now that Paris is out of jail. See, I can make pop culture references.
"The three user news sites differed from one another in subtle ways. Reddit was the most likely to focus on political events from Washington, such as coverage of Vice President Cheney"
I wouldn't really call it subtle. We certainly have days with lots of politics on the front page; I think it was that week that we learned Cheney neither belonged to the executive nor the legislative branch. Today doesn't happen to be one of those days, but I'm a little disappointed they didn't find any evidence of Ron Paul support on reddit. Heh, it's probably just more evidence of the anti-RP conspiracy...
"Digg was particularly focused on the release of Apple’s new iPhone"
I hear it's a popular phone (it can save lives! Use the "911" widget). I guess it was also a slow news week...
"Del.icio.us had the most fragmented mix of stories and the least overlap with the News Index [of mainstream news outlets]"
Hmm, this appears to only be reinforcing the fact that Del.icio.us is indeed not intended to be for news. I'm sorry for belaboring this, but I think I've found a solution.

I've been told that images help convey messages almost as well as catchy slogans or jingles. Since I have neither the energy nor the ability to come up with a good slogan or produce a jingle, I'm settling for a useful graphic. Print it out and put it in a drawer. If you're ever feeling anxious about what, for instance, Del.icio.us is all about, refer to this chart.In all seriousness, it's great seeing the PEJ showing interest and researching this. Don't settle for my synopsis, read the entire report at Journalism.org. I only hope they avoid including Del.icio.us in future studies regarding social news sites. And if they want to study social bookmarking, I can save them the time and cost of conducting the survey by saying that Del.icio.us has no competition there.

Incidentally, if someone at the PEJ would like to send me a shirt as compensation for my above service, I'd be happy to return the favor with some reddit gear. I'd wear it with pride -- and maybe only a bit of irony ;-)