kottke.org posts about James Surowiecki
James Surowiecki, the author of The Wisdom of Crowds, wrote about what was right and wrong about Reddit's crowdsourced hunt for the Boston bombing suspects.
The truth is that if Reddit is actually interested in using the power of its crowd to help the authorities, it needs to dramatically rethink its approach, because the process it used to try to find the bombers wasn't actually tapping the wisdom of crowds at all -- at least not as I would define that wisdom. For a crowd to be smart, the people in it need to be not only diverse in their perspectives but also, relatively speaking, independent of each other. In other words, you need people to be thinking for themselves, rather than following the lead of those around them.
When the book came out in 2004, I wrote a short post that summarizes the four main conditions you need for a wise crowd. What's striking about most social media and software, as Surowiecki notes in the case of Reddit, is how most of these conditions are not satisfied. There's little diversity and independence: Twitter and Facebook mostly show you people who are like you and things your social group is into. And social media is becoming ever more centralized: Facebook, Twitter, Tumblr, Medium, Pinterest, etc. instead of a decentralized network of independent blogs. In fact, the nature of social media is to be centralized, peer-dependent, and homogeneous because that's how people naturally group themselves together. It's a wonder the social media crowd ever gets anything right.
Writing for IEEE Spectrum, New Yorker staff writer James Surowiecki writes about the development and evolution of money, from that of tribal societies to today's highly abstracted currencies.
It's really in the seventh century B.C.E., when the small kingdom of Lydia introduced the world's first standardized metal coins, that you start to see money being used in a recognizable way. Located in what is now Turkey, Lydia sat on the cusp between the Mediterranean and the Near East, and commerce with foreign travelers was common. And that, it turns out, is just the kind of situation in which money is quite useful.
To understand why, imagine doing a trade in the absence of money-that is, through barter. (Let's leave aside the fact that no society has ever relied solely or even largely on barter; it's still an instructive concept.) The chief problem with barter is what economist William Stanley Jevons called the "double coincidence of wants." Say you have a bunch of bananas and would like a pair of shoes; it's not enough to find someone who has some shoes or someone who wants some bananas. To make the trade, you need to find someone who has shoes he's willing to trade and wants bananas. That's a tough task.
With a common currency, though, the task becomes easy: You just sell your bananas to someone in exchange for money, with which you then buy shoes from someone else. And if, as in Lydia, you have foreigners from whom you'd like to buy or to whom you'd like to sell, having a common medium of exchange is obviously valuable. That is, money is especially useful when dealing with people you don't know and may never see again.
In this same vein, this reply on Reddit to "Where has all the money in the world gone?" is also worth a read.
The thing to remember is that all throughout, from the initial trade to this central-banking system, all of this money is debt. It is IOUs, except instead of being an IOU that says "Kancho_Ninja will give one bushel of apples to the bearer of this bond in October", it says "Anyone in town will give you anything worth one bushel of apples in trade."
The money is not an actual thing that you can eat or wear or build a house with, it's an IOU that is redeemable anywhere, for anything, from anyone. It is a promise to pay equivalent value at some time in the future, except the holder of the money can call on anybody at all to fulfill that promise -- they don't have to go back to the original promiser.
Uniqlo, Costco, and Trader Joe's are among the large retailers that are making more money by hiring more retail employees, which runs counter to the conventional wisdom.
The big challenge for any retailer is to make sure that the people coming into the store actually buy stuff, and research suggests that not scrimping on payroll is crucial. In a study published at the Wharton School, Marshall Fisher, Jayanth Krishnan, and Serguei Netessine looked at detailed sales data from a retailer with more than five hundred stores, and found that every dollar in additional payroll led to somewhere between four and twenty-eight dollars in new sales. Stores that were understaffed to begin with benefitted more, stores that were close to fully staffed benefitted less, but, in all cases, spending more on workers led to higher sales. A study last year of a big apparel chain found that increasing the number of people working in stores led to a significant increase in sales at those stores.
Using Blockbuster and Netflix (and Redbox) as an example, James Surowiecki writes about how big incumbent companies can lose out to smaller upstarts.
The problem -- in Blockbuster's case, at least -- was that the very features that people thought were strengths turned out to be weaknesses. Blockbuster's huge investment, both literally and psychologically, in traditional stores made it slow to recognize the Web's importance: in 2002, it was still calling the Net a "niche" market. And it wasn't just the Net. Blockbuster was late on everything -- online rentals, Redbox-style kiosks, streaming video. There was a time when customers had few alternatives, so they tolerated the chain's limited stock, exorbitant late fees (Blockbuster collected about half a billion dollars a year in late fees), and absence of good advice about what to watch. But, once Netflix came along, it became clear that you could have tremendous variety, keep movies as long as you liked, and, thanks to the Netflix recommendation engine, actually get some serviceable advice. (Places like Netflix and Amazon have demonstrated the great irony that computer algorithms can provide a more personalized and engaging customer experience than many physical stores.) Then Redbox delivered the coup de grace, offering new Hollywood releases for just a dollar.
From Scott McCloud, here's Blockbuster's new logo.
From James Surowiecki, a very interesting piece about procrastination and related phenomena.
A similar phenomenon is at work in an experiment run by a group including the economist George Loewenstein, in which people were asked to pick one movie to watch that night and one to watch at a later date. Not surprisingly, for the movie they wanted to watch immediately, people tended to pick lowbrow comedies and blockbusters, but when asked what movie they wanted to watch later they were more likely to pick serious, important films. The problem, of course, is that when the time comes to watch the serious movie, another frothy one will often seem more appealing. This is why Netflix queues are filled with movies that never get watched: our responsible selves put "Hotel Rwanda" and "The Seventh Seal" in our queue, but when the time comes we end up in front of a rerun of "The Hangover."
The lesson of these experiments is not that people are shortsighted or shallow but that their preferences aren't consistent over time. We want to watch the Bergman masterpiece, to give ourselves enough time to write the report properly, to set aside money for retirement. But our desires shift as the long run becomes the short run.
I was going to post about this yesterday but...you know.
Companies who target the middle of the market (Sony, Dell, General Motors) are losing customers to companies like Apple & Hermes at the high end and Ikea & H&M at the low end. From James Surowiecki:
The products made by midrange companies are neither exceptional enough to justify premium prices nor cheap enough to win over value-conscious consumers. Furthermore, the squeeze is getting tighter every day. Thanks to economies of scale, products that start out mediocre often get better without getting much more expensive -- the newest Flip, for instance, shoots in high-def and has four times as much memory as the original -- so consumers can trade down without a significant drop in quality. Conversely, economies of scale also allow makers of high-end products to reduce prices without skimping on quality. A top-of-the-line iPod now features video and four times as much storage as it did six years ago, but costs a hundred and fifty dollars less. At the same time, the global market has become so huge that you can occupy a high-end niche and still sell a lot of units. Apple has just 2.2 per cent of the world cell-phone market, but that means it sold twenty-five million iPhones last year.
James Surowiecki, who writes the biweekly financial column for the New Yorker, has started a finance blog on the NYer site called The Balance Sheet.
Two quick reviews of Elizabeth Currid's book, The Warhol Economy, which argues that New York's "vibrant creative social scene" is what makes the city go. First, James Surowiecki in the New Yorker:
Of course, everyone knows that art and culture help make New York a great place to live. But Currid goes much further, showing that the culture industry creates tremendous economic value in its own right. It is the city's fourth-largest employer, and generates billions of dollars a year in revenue. More important, New York has no real global rival for dominance in the culture industry. Using an economic-analysis tool called a "location quotient," Currid calculates that New York matters far more to fashion, art, and culture than to finance. To exaggerate a bit, if New York suddenly disappeared, stock markets could keep functioning, but we would not be able to dress ourselves or find art to put on the wall. Currid suggests that, in the fight among cities for business, being the center of fashion and art constitutes New York's true "competitive advantage."
And from The Economist:
New York's cultural economy has reached a critical juncture, argues Ms Currid, threatened by, of all things, prosperity. The bleak economic conditions of the 1970s allowed artists to flock into dirt-cheap apartments and ushered in the East Village scene of the early 1980s. The boom of the past decade, by contrast, has priced budding Basquiats out of Manhattan, pushing them across the water to Brooklyn and New Jersey. Studio flats meant for artists-in-residence get snapped up by bankers. The closure last year of CBGB, a bar that became a punk and art-rock laboratory in the 1970s (and whose founder, Hilly Kristal, died last month) came to symbolise this squeeze.
Ms Currid sees this expulsion of talent as a serious problem. The solution, she argues, lies in a series of well-aimed public-policy measures: tax incentives, zoning that helps nightlife districts, more subsidised housing and studio space for up-and-coming artists, and more.
The first chapter of the book is available on the Princeton University Press site.
One of the causes of feature creep in products like consumer electronics is that when customers are making purchase decisions, they'll likely choose the one with the most features. "But, when they were asked to use the digital device, so-called 'feature fatigue' set in. They became frustrated with the plethora of options they had created, and ended up happier with a simpler product."
Short interview by James Surowiecki of Nassim Taleb about his new book, The Black Swan. "History is dominated not by the predictable but by the highly improbable -- disruptive, unforeseeable events that Taleb calls Black Swans. The effects of wars, market crashes, and radical technological innovations are magnified precisely because they confound our expectations of the universe as an orderly place." Malcolm Gladwell wrote an article on Taleb for the New Yorker in 2002, which Taleb said "put too much emphasis on the far less interesting, more limited -- and rather boring -- applications of my ideas to finance/economic, & less on the dynamics of historical events/philosophy of history, artistic success, and general uncertainty in society". See also an interview in New Scientist, a NY Times op-ed, and a long piece on the Edge site about the black swan idea.
CEOs of companies whose board members are socially well-connected get paid significantly more than those who work at companies with less connected board members. "Academics have found little evidence that higher executive pay leads to better company performance, and the recent study of three thousand companies actually found that the firms whose directors were the most well connected -- and which paid their C.E.O.s most lavishly -- in fact underperformed the market. Markets work best when people make independent decisions about how much a commodity -- in this case, the C.E.O. -- is worth. They stop working well when people simply imitate what others are doing, or when non-market factors (like how well you get along with the boss) intrude."
James Surowiecki discusses the waste of holiday giving. "Waldfogel's main finding is that, in general, people spend a lot more on presents than they're worth to those who receive them, a phenomenon that he calls 'the deadweight loss of Christmas.'" This is one of my big problems with the whole Christmas thing. Related: gift cards worth billions of dollars are left unredeemed each year.
Although Nintendo finds itself in third place in the video game console wars behind Sony and Microsoft, the company is doing really well financially while Sony and MS are maybe breaking even with their efforts. "Nintendo knew that it could not compete with Microsoft and Sony in the quest to build the ultimate home-entertainment device. So it decided, with the Wii, to play a different game entirely."
Tariffs on imported sugar and ethanol imposed by the US government keep our sugar expensive and is keeping the US from using more efficient methods of saving energy and, oh, by the way, helping the environment. This excerpt from the last two paragraphs of the piece is a succinct description of what's wrong with contemporary American politics:
Tariffs and quotas are extremely hard to get rid of, once established, because they create a vicious circle of back-scratching-government largesse means that sugar producers get wealthy, giving them lots of cash to toss at members of Congress, who then have an incentive to insure that the largesse continues to flow. More important, protectionist rules flourish because the benefits are concentrated among a small number of easy-to-identify winners, while the costs are spread out across the entire population. It may be annoying to pay a few more cents for sugar or ethanol, but most of us are unlikely to lobby Congress about it.
Maybe we should, though. Our current policy is absurd even by Washington standards: Congress is paying billions in subsidies to get us to use more ethanol, while keeping in place tariffs and quotas that guarantee that we'll use less. And while most of the time tariffs just mean higher prices and reduced competition, in the case of ethanol the negative effects are considerably greater, leaving us saddled with an inferior and less energy-efficient technology and as dependent as ever on oil-producing countries.
Maddening. Partisan politics is a not-very-elaborate smokescreen to distract us from this bullshit.
How GM and the other big US automakers are hamstrung by their dealers. It was their own fault, though. They misued the dealers and the dealers responded by gaining all sorts of regulatory protection that severely limits what the car companies can do.
Surowiecki on the difficulty of short-term thinking in business. "It's no wonder that management theory is dominated by fads: every few years, new companies succeed, and they are scrutinized for the underlying truths that they might reveal. But often there is no underlying truth; the companies just happened to be in the right place at the right time."
James Surowiecki fills us in on a new investment opportunity, housing futures. "If housing futures work the way they're supposed to, they will shift risk from those who are less able to bear it (individual homeowners with hefty mortgages) to those who are more willing to (speculators looking for a big upside on their investments). In the process, they will effectively provide a form of house-price insurance."
Even with the encroachment of blogs, craigslist, and online stock listings, James Surowiecki says the newspaper business is actually not a bad business to be in these days. "Newspapers are classic cash cows: solidly profitable businesses in a stagnant industry."
CEO pay and perks can be a good indicator of how healthy a business is, so it makes sense that investors are interested in just exactly how much chief executives make. "We shouldn't expect to see a dent in executive compensation anytime soon. But in the long run companies that don't balance pay with performance tend to suffer where it matters most -- in the stock market."
Two weeks ago, I wrote:
In terms of editorial and quality, I am unconvinced that a voting system like Digg's can produce a quality editorial product.
Lloyd Shepherd, Deputy Director of Digital Publishing at Guardian Unlimited, has been thinking along similar lines:
Everything we do to "edit" the [Guardian Unlimited] site seeks to keep a balance between editorial instinct and the desires of the audience, and that, in doing that, we may be reflecting the "community" more fairly, both mathematically and ethically, than the likes of digg.
So how do you reflect the community more fairly? Paging Mr. Surowiecki:
In order for a crowd to be smart, [Surowiecki] says it needs to satisfy four conditions: 1. Diversity, 2. Independence, 3. Decentralization, and 4. Aggregation.
Much of the online media we're familiar with uses a mix of humans and automated systems to perform the aggregating task. Human editors choose the stories that will run in the newspaper (drawing from a number of sources of information as Lloyd illustrated), blog authors select what links and posts to put on their blog (by reading other blogs & media outlets, listening to reader feedback, and sifting through already aggregated sources like del.icio.us or Digg), and the editors of Slashdot filter through hundreds of reader submissions a day to create Slashdot's front page. Google News uses technology to decide which stories are important, based primarily on what the publishers are publishing. Digg and del.icio.us rely almost entirely on the crowd to submit and determine by a simple vote what stories go on its front page.
Some of these methods work better than others for different tasks. The product of 50,000 diverse, independent, decentralized bloggers is probably more editorially interesting, fair, and complete than that of 50,000 diverse, independent, decentralized Digg users, but the Digg vote & tally approach is less time-intensive for all concerned and the information flows faster. A site like Slashdot sits in the middle...it's a little slower than Digg but offers a more consistent editorial product. A hybrid Digg+Slashdot approach (which is not unlike the one used by individual bloggers) would be for Digg to produce a "Digg digest", a human selected (could use simple voting or let the most highly respected community members choose) collection of the best stories of the day that incorporates what was said in the comments and around the web as well. Actually, I think if you wanted to start a blog that did this, it would do very well.
Surowiecki on the sorry state of the US patent system. "Since the [USPTO] is funded by patent fees, as opposed to getting its budget from Washington, it has a financial incentive to process applications as quickly, rather than as diligently, as possible."
Surowiecki on the economics of textbooks, i.e. why they cost so damn much. "You can often buy the same textbook abroad for significantly less than it costs in the U.S., so students have learned to buy directly from places like the U.K., and a host of small businesses have sprung up to import books."
James Surowiecki on insider trading and members of Congress. From 1993-1998, "senators beat the market, on average, by twelve per cent annually". Here's a piece on the same study from the FT early last year.
James Surowiecki, the New Yorker's resident economist, weighs in on the tipping debate. (Previously discussed here.)
Surowiecki on crisis management. "Entrepreneurs are the cockiest of all. It may be that the very qualities that help people get ahead are the ones that make them ill-suited for managing crises."
"The problem with the global village is all the global village idiots."
-- Paul Ginsparg
"You don't do good software design by committee."
-- Donald Norman
"There's no justice like angry-mob justice."
-- Principal Seymour Skinner
"A person is smart. People are stupid."
-- Agent K
The wisdom of crowds you say? As Surowiecki explains, yes, but only under the right conditions. In order for a crowd to be smart, he says it needs to satisfy four conditions:
1. Diversity. A group with many different points of view will make better decisions than one where everyone knows the same information. Think multi-disciplinary teams building Web sites...programmers, designers, biz dev, QA folks, end users, and copywriters all contributing to the process, each has a unique view of what the final product should be. Contrast that with, say, the President of the US and his Cabinet.
2. Independence. "People's opinions are not determined by those around them." AKA, avoiding the circular mill problem.
3. Decentralization. "Power does not fully reside in one central location, and many of the important decisions are made by individuals based on their own local and specific knowledge rather than by an omniscient or farseeing planner." The open source software development process is an example of effect decentralization in action.
4. Aggregation. You need some way of determining the group's answer from the individual responses of its members. The evils of design by committee are due in part to the lack of correct aggregation of information. A better way to harness a group for the purpose of designing something would be for the group's opinion to be aggregated by an individual who is skilled at incorporating differing viewpoints into a single shared vision and for everyone in the group to be aware of that process (good managers do this). Aggregation seems to be the most tricky of the four conditions to satisfy because there are so many different ways to aggregate opinion, not all of which are right for a given situation.
Satisfy those four conditions and you've hopefully cancelled out some of the error involved in all decision making:
If you ask a large enough group of diverse, independent people to make a prediciton or estimate a probability, and then everage those estimates, the errors of each of them makes in coming up with an answer will cancel themselves out. Each person's guess, you might say, has two components: information and error. Subtract the error, and you're left with the information.
There's more info on the book at the Wisdom of Crowds Web site and in various tangential articles Surowiecki's written:
- Smarter than the CEO
- Interview with Bill James
- Blame Iacocca - How the former Chrysler CEO caused the corporate scandals
- Search and Destroy (on Google bombs)
- The Pipeline Problem (drug companies)
- Hail to the Geek (government and information flow)
- Going Dutch (IPOs)
- The Coup De Grasso (fairness in business)
- Open Wide (movies and "non-informative information cascades")