kottke.org posts about journalism
In 2002, Dave Winer of Scripting News and Martin Nisenholtz of the New York Times made a Long Bet about the authority of weblogs versus that of NY Times in Google:
In a Google search of five keywords or phrases representing the top five news stories of 2007, weblogs will rank higher than the New York Times’ Web site.
I decided to see how well each side is doing by checking the results for the top news stories of 2005. Eight news stories were selected and an appropriate Google keyword search was chosen for each one of them. I went through the search results for each keyword and noted the positions of the top results from 1) “traditional” media, 2) citizen media, 3) blogs, and 4) nytimes.com. Finally, the scores were tallied and an “actual” winner (blogs vs. nytimes.com) and an “in-spirit” winner (any traditional media source vs. any citizen media source) were calculated. (For more on the methodology, definitions, and caveats, read the methodology section below.)
So how did the NY Times fare against blogs? Not very well. For eight top news stories of 2005, blogs were listed in Google search results before the Times six times, the Times only twice. The in-spirit winner was traditional media by a 6-2 score over citizen media. Here the specific results:
1) Hurricane Katrina hits New Orleans.
Search term: “hurricane katrina”
3. Top citizen media result (Wikipedia)
13. Top media result (CNN)
56. Top NY Times mention (NY Times).
61. Top blog result (Kaye’s Hurricane Blog)
Winner (in spirit): Citizen media
Winner (actual): NY Times
2) Big changes in the US Supreme Court (Rhenquist dies, O’Conner retires, Roberts appointed Chief Justice, Harriet Miers rejected).
Search term: “harriet miers”
4. Top media result (Washington Post)
5. Top citizen media result (Wikipedia)
8. Top NY Times mention (NY Times)
11. Top blog result (TalkLeft)
Winner (in spirit): Media
Winner (actual): NY Times
3) Terrorists bomb London, killing 52.
Search term: “london bombing”
1. Top media result (CNN)
2. Top citizen media result (Wikipedia)
21. Top blog result Schneier on Security
No NY Times article appears in the first 100 results.
Winner (in spirit): Media
Winner (actual): Blogs
4) First elections in Iraq after Saddam.
Search term: “iraq election”
1. Top media result (BBC News)
6. Top blog result (Iraq elections newswire)
6. Top citizen media result (Iraq elections newswire)
14. Top NY Times mention (NY Times)
Winner (in spirit): Media
Winner (actual): Blogs
5) Terri Schiavo legal fight and death.
Search term: “terri schiavo”
2. Top blog result (Abstract Appeal)
2. Top citizen media result (Abstract Appeal)
4. Top media result (CNN)
65. Top NY Times mention (NY Times)
Winner (in spirit): Citizen media
Winner (actual): Blogs
6) Pope John Paul II dies and Cardinal Joseph Ratzinger appointed Pope Benedict XVI.
Search term: “pope john paul ii death”
1. Top media result (CNN)
3. Top citizen media result (Wikipedia)
58. Top blog result (The Pope Blog: Pope Benedict XVI)
No NY Times article appears in the first 100 results.
Winner (in spirit): Media
Winner (actual): Blogs
7) The Israeli withdrawal from the Gaza Strip.
Search term: “gaza withdrawal”
1. Top media result (Worldpress.org)
31. Top blog result (Simply Appalling)
31. Top citizen media result (Simply Appalling)
No NY Times article appears in the first 100 results.
Winner (in spirit): Media
Winner (actual): Blogs
8) The investigation into the Valerie Plame affair, Judith Miller, Scooter Libby indicted, etc..
Search term: “scooter libby indicted”:
1. Top media result (CNN)
15. Top blog result (Seven Generational Ruminations)
15. Top citizen media result (Seven Generational Ruminations)
43. Top NY Times mention (NY Times)
Winner (in spirit): Media
Winner (actual): Blogs
And just for fun here’s a search for “judith miller jail” (not included in the final tally):
1. Top media result (Washington Post)
3. Top blog result (Gawker)
3. Top citizen media result (Gawker)
No NY Times article appears in the first 100 results (even though there are several matching articles on the Times site).
In covering the jailing of their own reporter, the Times lagged in the Google results behind such informational juggernauts as Drinking Liberally, GOP Vixen, and Feral Scholar.
Winner (in spirit): Media
Winner (actual): Blogs
Here’s the overall results, excluding the Judith Miller search:
Overall winner (in spirit): Media (beating citizen media 6-2).
Overall winner (actual): Blogs (beating the NY Times 6-2).
Some observations:
- My feeling is that Mr. Nisenholtz will likely lose his bet come 2007. Even though the nytimes.com fares very well in getting linked to by the blogosphere, it does very poorly in Google. This isn’t exactly surprising given that most NY Times articles disappear behind a paywall after a week and some of their content (TimesSelect) isn’t even publicly accessible at all. Also, I didn’t look too closely at the HTML markup of the NY Times, but it could also be that it’s not as optimized for Google as well as that of some weblogs and other media outlets.
- “www.nytimes.com” has a PageRank of 10/10, higher than that of “www.cnn.com” (9/10), yet stories from CNN consistently appeared higher in the search results than those from the Times. The Times clearly has overall authority according to Google, but when it comes to specific instances, it falls short. In some cases, a NY Times story didn’t even appear in the first 100 search results for these keyword searches.
- By 2007, it may be difficult to differentiate a blog from a traditional media source. All of the Gawker and Weblogs, Inc. sites are presented in a blog format and are referred to as blogs but otherwise how are they distinguishable from traditional media? Engadget paid to send 12 people to cover the CES technology conference, probably as many or more than the Times sent. The Sundance film festival was heavily covered by paid writers for both companies as well. In the spirit in which this bet was made, I’d have a hard time counting any of their sites as blogs. (And what about kottke.org? I get paid to write it. Am I still a member of the citizen media or have I crossed over?)
- Choosing appropriate news stories and keywords for those stories was difficult in some cases. Katrina was a no-brainer, but was the Terri Schiavo story really one of the top eight news stories of 2005? Resolving the methodology for this bet in 2007 will be tricky. I wonder how the Long Bets Foundation will handle its determination of the victory.
- Wikipedia does very well in Google results for topical search terms. Overall, traditional media still dominates (in first appearance as well as number of results), but blogs and Wikipedia do very well in some instances.
- What do these results mean? Probably not a whole lot. Nisenholtz asserts that “[news] organizations like the Times can provide that far more consistently than private parties can” while Winer says that “in five years, the publishing world will have changed so thoroughly that informed people will look to amateurs they trust for the information they want”. It’s difficult to draw any conclusions on this matter based on these results. Contrary to what most people believe, PageRank has a bias, a point of view. That POV is based largely (but not entirely) on what people are linking to. As someone said in the discussion of this bet, this bet is about Google more than influence or reputation, so these results probably tell us more about how Google determines influence on a keyword basis rather than how readers of online informational sources value or rate those sources. Do web users prefer the news coverage of blogs to that of the NY Times? I don’t think you can even come close to answering that question based on these results.
Methodology and caveats
The eight news stories were culled from various sources (Lexis-Nexis, Wikipedia, NY Times) and narrowed down to the top stories that would have been prominently covered in both the NY Times and blogs.
The keyword phrase for each of the eight stories was selected by the trial and error discovery of the shortest possible phrase that yielded targeted search results about the subject in question. In some cases, the keyword phrase chosen only returned results for a part of a larger news story. For instance, the phrase “pope john paul” was not specific enough to get targeted results, so “pope john paul ii death” was used, but that didn’t give results about the larger story of his death, the conclave to select a new pope, and the selection of Cardinal Joseph Ratzinger as Pope Benedict XVI. In the case of “katrina”, that single keyword was enough to produce hundreds of targeted search results for both Hurricane Katrina and its aftermath. Keyword phrases were not tinkered with to promote or demote particular types of search results (i.e. those for blogs or nytimes.com); they were only adjusted for the relevence of overall results.
The searches were all done on January 27, 2006 with Google’s main search engine, not their news specific search.
Since the spirit of the bet deals with the influence of traditional media versus that of citizen-produced media, I tracked the top traditional media (labeled just “media” above) results and the top citizen media results in addition to blog and nytimes.com results. For the purposes of this exercise, relevent results were those that linked to pages that an interested reader would use as a source of information about a news story. For citizen media, this meant pages on Wikipedia, Flickr (in some cases), weblogs, message boards, wikis, etc. were fair game. For traditional media, this meant articles, special news packages, photo essays, videos, etc.
In differentiating between “media” & citizen media and also between relevent and non-relevent results, in only one instance did this matter. Harriet Miers’s Blog!!!, a fictional satire written as if the author were Harriet Miers, was the third result for this keyword phrase, but since the blog was not a informational resource, I excluded it. In all other cases, it was pretty clear-cut.
The Onion interviews Stephen Colbert. “It used to be, everyone was entitled to their own opinion, but not their own facts. But that’s not the case anymore. Facts matter not at all. Perception is everything.”
Why do journalists drink so much Tab? Futhermore, if, as conservatives would like us to believe, the political and cultural tempo of the country is being dictated by the pulse of the liberal media and they all guzzle fantastic amounts of Tab, why is Tab not more popular?
Email correspondance between members of The New Yorker staff and one of Caitlin Flanagan’s sources in writing this story about Mary Poppins’ author P.L. Travers. The source, Travers biographer Valerie Lawson, wrote a letter to the editor complaining that Flanagan had not properly attributed items in the story to Lawson. “The exchange offers a glimpse at the sausage-factory aspect of how the magazine handles complaints, and raises interesting questions about what journalists owe, in terms of recognition, to their sources.”
Two weeks ago, I wrote:
In terms of editorial and quality, I am unconvinced that a voting system like Digg’s can produce a quality editorial product.
Lloyd Shepherd, Deputy Director of Digital Publishing at Guardian Unlimited, has been thinking along similar lines:
Everything we do to “edit” the [Guardian Unlimited] site seeks to keep a balance between editorial instinct and the desires of the audience, and that, in doing that, we may be reflecting the “community” more fairly, both mathematically and ethically, than the likes of digg.
So how do you reflect the community more fairly? Paging Mr. Surowiecki:
In order for a crowd to be smart, [Surowiecki] says it needs to satisfy four conditions: 1. Diversity, 2. Independence, 3. Decentralization, and 4. Aggregation.
Much of the online media we’re familiar with uses a mix of humans and automated systems to perform the aggregating task. Human editors choose the stories that will run in the newspaper (drawing from a number of sources of information as Lloyd illustrated), blog authors select what links and posts to put on their blog (by reading other blogs & media outlets, listening to reader feedback, and sifting through already aggregated sources like del.icio.us or Digg), and the editors of Slashdot filter through hundreds of reader submissions a day to create Slashdot’s front page. Google News uses technology to decide which stories are important, based primarily on what the publishers are publishing. Digg and del.icio.us rely almost entirely on the crowd to submit and determine by a simple vote what stories go on its front page.
Some of these methods work better than others for different tasks. The product of 50,000 diverse, independent, decentralized bloggers is probably more editorially interesting, fair, and complete than that of 50,000 diverse, independent, decentralized Digg users, but the Digg vote & tally approach is less time-intensive for all concerned and the information flows faster. A site like Slashdot sits in the middle…it’s a little slower than Digg but offers a more consistent editorial product. A hybrid Digg+Slashdot approach (which is not unlike the one used by individual bloggers) would be for Digg to produce a “Digg digest”, a human selected (could use simple voting or let the most highly respected community members choose) collection of the best stories of the day that incorporates what was said in the comments and around the web as well. Actually, I think if you wanted to start a blog that did this, it would do very well.
Khoi Vinh on the move…he’s the new Design Director for NYTimes.com. From the outside, it’s one of the best jobs in web design and it’s been filled well. (via waxy)
Crunks ‘05: The Year in Media Errors and Corrections (and plagiarists). My favorite: “Norma Adams-Wade’s June 15 column incorrectly called Mary Ann Thompson-Frenk a socialist. She is a socialite.”
If Mike Wallace could question GW Bush, he would ask him: “What in the world prepared you to be the commander in chief of the largest superpower in the world? In your background, Mr. President, you apparently were incurious. You didn’t want to travel …. Why do you think they nominated you?”
Ken Auletta explores the recent troubles at the NY Times in the New Yorker (interview with Auletta). As much as people complain about the liberal media, it’s hard to imagine a conservative magazine running a similar story about, say, Fox News.
A just-concluded eGullet conversation with Ruth Reichl, currently editor-in-chief of Gourmet magazine and former food critic for The New York Times.
Reasons bloggers hate the mainstream media. “Bloggers got stood up at prom. By the MSM.”
Steven Shaw doesn’t like a lot of food criticism and he’s not shy about telling you why. In his opinionated new book (which, as a NYC food fan, I enjoyed thoroughly), Shaw devotes much of a lengthy chapter to skewering guidebooks like Zagat’s and Michelin, starred restaurant reviews, and the undercover restaurant reviewer. Ruth Reichl, now editor-in-chief of Gourmet, recently recounted her experiences as food critic for the New York Times in her newest book, Garlic and Sapphires. Reichl employed a number of disguises when going to restaurants in order to ensure she didn’t receive special service because of her job at the Times. Shaw believes this approach is flawed and serves to distance restaurants and their customers:
It sends a signal to the public that restaurants are out to deceive us, and that in order to expose them restaurant reviewers must act as undercover investigative consumer advocates.
He prefers an approach akin to other forms of artistic criticism, with the reviewer taking a more active role in being as close to the action as possible:
There is, to my mind, absolutely nothing wrong with a critic having ties — close ties — to the community about which he writes. In my opinion, it is preferable from the standpoint of providing the best possible coverage. To me, the primary function of restaurant criticism should not be something so prosaic as reporting on the average meal and labeling it with some stars. Rather, restaurant criticism should parallel other forms of criticism — in art, literature, architecture, music — such that critics are champions of excellence who promote the best within the industry while exposing the worst.
This probably sounds like a familar argument to many who follow weblogs and the ongoing conversation about the responsibility of bloggers regarding disclosure of junkets, gifts, free movies, & review copies of books, their relationship to advertisers, who their friends are, and so forth. It’s a question of access vs. independence and objectivity. To get a story, some sort of access is often required, but then the reader might worry about biased reportage.
The key is trust (and I’m sure Shaw would agree with me here). Do you trust a particular source of information to balance her need for access to the story with the desire of her readers for her to remain as independent and fair-minded as possible? I believe that if we want better reviews, we need to be better readers and take a more active role in how we deal with information. Access isn’t necessarily bad, but what individual bloggers/journalists do with that access can be.
And when reading, you should be asking yourself, is the writer being fair here? Have they been fair in the past? If the piece you’re reading appears in the NY Times or the WSJ, how does the political orientation (if any) affect what gets printed in the paper? Are music journalists and bloggers biased in their reviews because they receive free review CDs in the mail? And if so, does that make the reviews completely worthless? If they don’t disclose things like junkets, personal relationships to their subjects, and the like, does that completely negate the review? Or can you adjust your opinion of the reviews to get something worthwhile out of them anyway?
Dealing with information has always been an imprecise science; there’s no such thing as complete objectivity. But as readers, we can encourage the writers whose work we read to be as fair as possible.
Disclosure: I purchased this book in a NYC bookstore with my own money. I have never met Steven Shaw, but I do enjoy eGullet very much. If you click on any of the links to Amazon in this piece and purchase merchandise there, I will get a small percentage (~5%) of the sale.
I had this idea the other day that instead of having to open my laptop or turn on the TV to check the weather report, my toaster could burn that information onto my breakfast toast as a passive information delivery mechanism. I knew that people had wired toasters to print images on them, but I didn’t remember that someone had done the weather thing already. That got me thinking about what other information a toaster could print on bread. A graph of the previous day’s DJIA activity? Photo of your kids? The Red Sox score from last night?
There are constraints, of course. Bread is not exactly a high resolution medium. A course wheat bread would be difficult to print on while a dense rye might give you a couple dozen ppi to work with. But then you run into a contrast problem…toasted rye bread isn’t much darker than untoasted rye bread. Now, if you were to use Pop Tarts, they’re a little more high-res, a finer grained paper. You might even be able to print a few lines of text if the heating elements were precise enough…your stocks, meeting schedule for the day, top news stories, shopping list, the 5-day forecast, or a serial short story that you read over a few breakfasts (you could call them Breakfast Serials™!!). Or maybe toasters will be free in the future, with the toaster companies making their money from advertising printed on your morning toast, not unlike the free newspapers they hand out in the NYC subways.
Though what would be even better is wifi-enabled Alpha Bits. Just connect the box to your local network, pour yourself some cereal, and view the five most recent headlines from your RSS reader floating in your milk. Then right click your bowl to open up links on the screen in your refrigerator. That and a rocket-powered hoverbike, please.
Nobody’s talking about the anal sex portion of a recently released survey on American sexual habits. “Evidently anal sex is too icky to mention in print. But not too icky to have been tried by 35 percent of young women and 40 to 44 percent of young men — or to have killed some of them.”
Here’s the front page of the new Guardian. They’ve completely revamped the paper…it’s smaller and has a new font, among other changes. They changed the format to Berliner because “unambiguous research [showed] that readers increasingly find broadsheet newspapers difficult to handle in many everyday situations, including commuting to work”.
New feature in the NY Times magazine: comics! First up, a six-month-long strip by Chris Ware, on whom I have a non-sexual crush.
The NY Times takes Google to task for blacklisting Cnet over them publishing some publicly available information about Google CEO Eric Schmidt. I wonder if there’s a useful distinction to be made between implicitly available information and explicitly broadcast information?
News.com ruminates about Google building a collection of tools that serve as a replacement OS. Where have we heard that recently? You’re welcome for the story idea and thanks for the non-link, guys…tech journalism at its finest. I hereby institute a policy of not linking to you for a year.
Michael via email: “please tell me you were kidding”. Well, mostly yes, particularly about the no link policy thing (it’s actually going to be two years).
Paul’s really personal Google News. “Paul comes late to work, again”.
“Right and left wing blogs are both crap”. “The left is full of crop circle paranoids. The right is full of stupid angry people. The sheer volume of information in both does manage to strip things to bare bones facts, but not by virtue of intelligence, just volume - like a colony of bacteria feeding on a corpse.”
Newer posts
Older posts
Stay Connected