26 4. BUILDING FOR CHOICE: ALLOWING IMPORT AND EXPORT
menu. Sure, you may buy more this time, but after the stomachache hits you’ll make sure your next outing is to Burger King.
Companies rarely try to measure such effects and even if they did, it’s not easy. It’s a piece of cake to serve someone an additional link and seeing if they click on it; keeping track of whether they come back to the store in the weeks and months to come is much harder. Worse, the difference made by any one additional box is subtle, and thus hard to measure. The real test isn’t whether removing one sell box gets Amazon more customers, it’s whether switching to a kindler, gentler layout does. But that would be a radical change for Amazon—and thus pretty hard to test without raising hackles and freaking people out. “Well, if you can’t measure people,” the MBAs say, “you can at least ask them.” And thus the dreaded “focus group,” whose flaws can dwarf even the most bogus statistical study. At least with the clickthrough games you’re measuring what people actually do; with focus groups you find out what people want you to think they say they do, which is a very different thing.
For one thing, people are notoriously bad observers of themselves. For the most part, we don’t know why or how we do things, so when we’re asked we make up rationalizations on the spot. This isn’t just carelessness—it’s how the brain works. To accomplish tasks of any complexity, we need to make their component parts automatic—you’d never get to the store if you had to think about which thigh muscles to move to get your leg in the right position—and automatic behavior is exactly behavior we don’t think about (this is why athletes’ memoirs are so boring.[1]
So not only are you asking people a question they don’t—can’t—know the answer to, you’re also asking them in a nice conference room, filled with other people, after giving them some cash. It doesn’t take much reading in social psychology to realize this isn’t exactly an ideal situation for honesty. People are, of course, going to say what they think you want to hear, and even if you have the most neutral of moderators asking the questions, they’re going to be able to make some educated guesses as to what that is.
Which is why watching focus groups is such an infuriating experience: like a girl pretending to play dumb in a bar, you’re watching people act the way they think people expect people like them to behave.
But if you can’t measure people and you can’t ask them, what does that leave? Well, good old-fashioned experience. As is usually true in life, there’s no shortcut
- ↑ See D. F. Wallace, “How Tracy Austin Broke My Heart” in Consider the Lobster (2005).
