economics

The Rise of the Y-Axis-Zero Fundamentalists

Justin Fox 35 Comments

On Friday, I read a Natalie Kitroeff Businessweek.com story on the declining appeal of law school, and was so struck by this chart that I shared it on Twitter:

law schools

The chart tells a dramatic story: all the gains in law school enrollment since the mid-1970s have been wiped out in just three years. Twitter responded to that drama with lots of retweets and favorites — but also with lots of disapproving remarks like this:

And this:

There were many, many more responses like that. A couple of them wielded the name of Edward Tufte, today’s leading authority on the visual presentation of data. Which is interesting, because after about five seconds of Googling I found Tufte’s actual views on the practice:

In general, in a time-series, use a baseline that shows the data not the zero point. If the zero point reasonably occurs in plotting the data, fine. But don’t spend a lot of empty vertical space trying to reach down to the zero point at the cost of hiding what is going on in the data line itself. (The book, How to Lie With Statistics, is wrong on this point.)

For examples, all over the place, of absent zero points in time-series, take a look at any major scientific research publication. The scientists want to show their data, not zero.

The urge to contextualize the data is a good one, but context does not come from empty vertical space reaching down to zero, a number which does not even occur in a good many data sets. Instead, for context, show more data horizontally!

Thanks to one of the offended responders on Twitter, Abhinav Agarwal, we can see what the Businessweek.com chart would have looked like with a zero base:

I love that he went to the effort to make that (thanks, Abhinav!) but … it is less informative than the original chart. Yes, in the new version it’s now crystal clear that law-school enrollment hasn’t gone to zero. But who looked at the original chart and thought it had? (Well, this guy says he did, but I think he’s kidding.) And the contrast between the herky-jerky rise of the past four decades and the straight-line drop since 2010 is much less clear in the zero-base chart. It hides the precipitousness of law schools’ change in fortunes.

Such arguments seem to carry little weight, though, among the legions of what BuzzFeed’s Matthew Zeitlin has dubbed y-axis-zero fundamentalists. I had somehow missed out on their rise, I guess because all of my HBR time-series charts over the past few years have for various reasons (the main one being that my Excel skills are so limited that I don’t know how to truncate the axes) featured y-axes that go to zero. But apparently now this is a thing. The Huffington Post‘s Ben Walsh reported a similar experience with a recent (non-zero-based) chart on taxi medallions in New York. According to Walsh, “all the responses were like ‘rule violated. i refuse to consider your thesis’.”

When I checked the Twitter bios of the people who objected to Businessweek.com chart, most of them were software programmers, so I wondered if it was some weird coder obsession. It might be, but a simpler explanation was that prominent programmer Jeff Atwood had retweeted it to his 152,000 followers.

Instead, I think it’s mainly just that more and more people have acquired some amount of statistical literacy, and have learned along the way that not basing your y-axis at zero is can be misleading. As Duke sociology professor — and believer in non-zero-based charts — Kieran Healy Tweeted when I asked him where he thought the reaction came from:

“Narrow axes can make small and inconsequential changes seem big,” Healy went on, “but—symmetrically—zero-axes can make big and real changes seem small. What matters isn’t some iron rule like ‘Always have a zero-base axis!’, it’s your prior commitment to being honest with the data.”

It is easy enough to find examples of people using broken y axes to mislead. From a Media Matters compendium of Fox News chart outrages:

fbn-cavuto-20120731-bushexpire

This isn’t much of a time series, and I really can’t think of any good reason why the y-axis on a bar chart shouldn’t go to zero. But more important than any simple rule is that this chart was obviously crafted to deceive — there’s really no other reason to draw the chart this way.

The Businessweek.com chart, on the other hand, was crafted to show the data as fully as possible. Facebook “data visualization guru” Andy Kriebel recommends adding a note to any non-zero-based-y-axis chart explaining why you didn’t base it at zero. That’s not a bad idea, but I also think the overwhelming majority of those who read a chart like this one online (as opposed to those who see a chart flitting by on the TV screen) are able to figure out what’s going on. I love that so many people online are on the lookout for dodgy charts. But focusing on the data isn’t really dodgy.

Update:  My brilliant colleague Scott Berinato, who is working on a book on data visualization for the HBR Press and created the cool Vision Statement “How to Lie with Charts” in the December issue of HBR, emailed me with his thoughts, which I don’t entirely agree with but seemed worth sharing given that he knows more than I do:

I have to agree with them about the Y axis. Not because it should be a hard and fast rule but because of the metaphor problem. Our brains create 0 when your line begins or ends at the bottom — a metaphorical zero as in “no one is going to law school because the line’s at the bottom.” This is exacerbated by the headline “Empty Classrooms,” which creates a textual cue that “empty” is what matters. 

There’s also the slope problem. Tufte is right and wrong. He’s right about just show the data but a truncated axis doesn’t actually show the data. The data is not the line, the line divides space that represents the proportion of a (those enrolling) and b (those not). So by truncating the axis we not only create a more severe-looking slope, we literally hide representative space, and more on one side than the other.

Having said all that, this kind of thing is rampant, because of web design. This chart would be very tall otherwise. So we have to think about the tradeoffs. My developing sense for these situations is to go even simpler. The data that matters here is:

‘74: low

‘74-‘10: Steady, rolling climb.

‘10-‘13: precipitous fall off

In theory we could build this same chart with three data points — ‘74, ‘10, ’13 — unless those three small humps on the climb matter to the story, which I don’t think they do. Basically start with as few data points as possible then add as necessary. Don’t even connect the lines necessarily; use points.

‘A New Golden Age for Media?’ Annotated

Justin Fox 1 Comment

My latest Atlantic column, headlined “A New Golden Age for Media?” online and “Start the Presses!” in the May print edition, is about how newspapers made money back before they became profit-spewing monopolies — the idea being that the future of the news media will probably look more like that than what media people got used to over the past half century.

I relied on a bunch of books in writing the piece, but only had room to mention two. And while it is possible to insert links into Atlantic print articles, the last time I tried it didn’t stick, and I was in rush this time so I didn’t bother. So here are the links and other annotations I would have liked to include.

pmarca twitterFirst, to the left, is a screen shot of the beginning of Mark Andreessen’s Twitter list of the ways digital media companies can make money (read it from bottom to top). He was later nice enough to compile these and a zillion other media-related tweets in one handy blog post.

Andreessen’s tweeting didn’t inspire me to write the piece — that was Atlantic deputy editor Don Peck’s idea — but it certainly helped organize it.

I mention two books in the column, Richard John’s Spreading the News: The American Postal System from Franklin to Morse and James McGrath Morris’s Pulitzer: A Life in Politics, Print, and Power. They’re both pretty great.

Spreading the News is based on John’s doctoral dissertation, so it’s academic. But it’s good academic, offering lots of I-did-not-know-that moments, plus a framework for understanding why U.S. communications networks developed the way they did. John continues the story in Network Nation: Inventing American Telecommunications, which I cited in a previous Atlantic column. By the number of reviews the two books have gotten on Amazon, it appears that they haven’t attracted a general readership. They should. They’re both relevant to current issues such as the troubles of the USPS and the monopolization of broadband, they’re not hard to read, and they change how one sees the world.

John begins Spreading the News with a quote from Alexis de Tocqueville’s 1831-1832 travel journals, subsequently published as Journey to America:

There is an astonishing circulation of letters and newspapers among these savage woods … I do not think that in the most enlightened rural districts of France there is intellectual movement either so rapid or on such a scale as in this wilderness.

John points out that a few years later in Democracy in America, Tocqueville quantifies this observation, showing that postal revenue per person is higher in backwoods Michigan and Florida than in the Departement du Nord, “one of the most enlightened and manufacturing parts of France.” The Frenchman also offers this wonderful assessment of the inhabitants of the rude wood huts of America’s frontier:

Who would not suppose that this poor hut is the asylum of rudeness and ignorance? Yet no sort of comparison can be drawn between the pioneer and the dwelling which shelters him. Everything about him is primitive and unformed, but he is himself the result of the labor and the experience of eighteen centuries. He wears the dress, and he speaks the language of cities; he is acquainted with the past, curious of the future, and ready for argument upon the present; he is, in short, a highly civilized being, who consents, for a time, to inhabit the backwoods, and who penetrates into the wilds of the New World with the Bible, an axe, and a file of newspapers.

But I digress.

Morris’s Pulitzer is an entertaining, engrossing biography. So is David Nasaw’s The Chief: The Life of William Randolph Hearst, the source for most everything Hearst in the column. The stuff about James Gordon Bennett came mainly from Richard Kluger’s The Paper: The Life and Death of the New York Herald Tribune, a book I read back when it came out in 1986, and loved. I even visited Kluger at his house near Princeton and I thought I had written up the interview for The Daily Princetonian, but I can’t find the article in the archives. This recent Prince article about Kluger looks pretty cool, though.

Most of the other information about how newspapers made money in the 19th century came from Gerald Baldasty’s The Commercialization of News in the Nineteenth Century, which is not exactly a fun read, but is full of useful facts. My brief mention of radio’s success as an advertising medium was based on a skimming of Laurence Bergreen’s Look Now, Pay Later: The Rise of Network Broadcasting, which looked pretty interesting although I was mainly just digging for numbers. Such as, 1931 advertising revenues for the main radio networks and a couple of leading magazines:

  • Saturday Evening Post, $35 million
  • NBC, $25.9 million
  • Ladies Home Journal, $12.8 million
  • CBS, $11.6 million

For the rest of the 1930s, revenues just kept rising for the radio networks while they presumably stagnated or even fell at the magazines. NBC and CBS, Berggreen wrote, “were purveying a commodity desperately needed by a Depression-ravaged nation: free entertainment.” ABC, by the way, wasn’t created till 1943, after the FCC ordered NBC to divest a big chunk of its operations and the man behind Lifesavers candies, Edward J. Noble, bought it.

Still, print publications could make tons of money even in the broadcast era. The quote from Warren Buffett about newspapers gushing profits comes from Berkshire Hathaway’s 2006 shareholder letter. Sandwiched between the two sentences I quoted in my column was this:

As one not-too-bright publisher famously said, “I owe my fortune to two great American institutions: monopoly and nepotism.”

Buffett then launches into an extended (it starts on page 11 and takes up all of page 12) and very good explanation of why newspapers used to be a great business but aren’t anymore. It’s too long to quote, so just read the letter. Fun bonus: Margaret Sullivan, now the excellent public editor of The New York Times, rates an admiring mention.

Finally, while my Atlantic piece was in the editing process, Buzzfeed founder Jonah Peretti posted a fascinating memo to staff in which he too compares the current media environment to that of before 1950. At one point he writes:

These days, media companies don’t have natural monopolies or oligopolies where one or two newspapers dominate a local market or a handful of broadcasters are the only options on a limited dial. There is more competition, the market is more fragmented, and gatekeepers have less power. Advertisers have more ways to reach consumers so they aren’t as dependent on publishers and can negotiate lower rates. In fact, some smart people in our industry don’t think it is possible to build a huge new media company anymore, that the golden age is over, and that all the growth now will be limited to pure technology companies.

Which is basically the argument of my column, although I wouldn’t say it’s impossible to build a media empire, just really, really hard. “This pessimistic view is wrong,” Peretti continues,“because it is focused entirely on what has been lost (monopoly pricing power, etc) and ignores what has been gained. This is a common psychological trap: People tend to be overly focused what is lost, while under-appreciating gains.”

The gains Peretti sees have to do with technology, scale, and diversity of talent. I totally get how those three factors all help Buzzfeed now. And I wouldn’t put it past Peretti to become the new Henry Luce. I wouldn’t bank on it either, though — and I really wouldn’t bank on Buzzfeed continuing to thrive long after Peretti and Ben Smith have retired, as Luce’s Time Inc. did. But then I do prefer history to futurism, which I guess makes me a backward-looking guy.

Hirschmania

Justin Fox Leave a comment

AdelmanMy review of Jeremy Adelman's wonderful new biography, Worldly Philosopher: The Odyssey of Albert O. Hirschman, was finally published in the New York Times Book Review on Sunday.

I say finally because I turned the thing in on March 18, and I think the editors only changed one word in the interim (switching out "doorstopper" for "doorstop," which seemed like a good move). I'm not complaining at all — one knows going in that a NYTBR review may take months to appear, and the Times paid me long ago. But it was still maddening to write something with great enthusiasm and then not see it for months and months and months. It took great willpower not to write Barry Gewen at the Book Review e-mail after e-mail asking, "When's it coming out? Huh? When?"

Anyway, the publication led me to go back and read some of the other things written lately about Hirschman and the Adelman biography. I had avoided other reviews at first because I feared they might taint mine, then because I was so frustrated that mine hadn't been published yet. Now I could finally savor them, and it turns out they're all worth savoring.

The first I read was an essay on Hirschman's legacy by Daniel Drezner. My review — and to a certain extent Adelman's book — focused on why Hirschman wasn't as influential as other economists of his generation such as Milton Friedman and Paul Samuelson. Drezner, though, is a political scientist, and from the perspective of a political scientist Hirschman was pretty danged influential. "Anyone working on issues of economic power, economic development or economic ideas cannot do so without either building on or tangling with Hirschman’s legacy," Drezner writes, concluding that Hirschman's work "is the purest example of political economy since the days of Adam Smith."

After that I moved on to Roger Lowenstein's essay in the Wall Street Journal. Lowenstein was the "journalist friend" mentioned in my review who urged me to read Exit, Voice, and Loyalty, and his piece focuses mainly on that brilliant book. Lowenstein makes a heartfelt plea for more voice and less exit, or at least a healthy balance between the two. "Hirschman saw that when organizations make it easy to exit, voice is weakened," he writes. "Yet, for voice to be effective, a possibility of exit must be present." (The "business-school professor" who pushed The Passions and the Interests on me was Rebecca Henderson of HBS.)

In The New Yorker, Malcolm Gladwell got a lot more space than Drezner or Lowenstein or I, and he makes good use of it. When I first saw the headline ("The Gift of Doubt") in the magazine a few weeks ago, my initial reaction was of course, &^%#$@£ Gladwell, coopting Hirschman to say some facile thing or other. But it actually turns out to be a great and complex essay. It cannot, thus, be summed up in a paragraph, but I especially loved this passage about Hirschman and his brother-in-law and mentor, Eugenio Colorni:

Hamlet shouldn’t have been frozen by his doubts; he should have been freed by them. Hamlet took himself too seriously. He thought he needed to be perfect. Colorni and Hirschman didn’t.

Then there's Cass Sunstein's take in The New York Review of Books. Sunstein may be the most Hirschmanesque (Hirschmanian?) of modern public intellectuals. He crosses disciplinary boundaries, he writes for scholars and for laypeople, he mixes hope with skepticism, he doesn't think the fact that most plans go awry is a reason not to make plans. "Hirschman’s work changes how you see the world," Sunstein writes. "It illuminates yesterday, today, and tomorrow. His categories become your categories." Sort of like nudging.

I finished Adelman's book totally enamored of Hirschman and his way of looking at the world, but dubious of how much impact such a doubt-filled approach could have in an intellectual arena dominated by dueling certainties. After reading what everybody else had to say, I'm a bit more hopeful. Hooray for that!

One other thing that I thought about saying in the review but decided not to was that, for the first couple hundred pages or so, I thought Adelman's book might be the best biography I had ever read, period. It mixed the politics and the personal so smartly and so wisely that I was in awe of the accomplishment. I'm still in awe, but the second half of the book (about Hirschman's academic career) just couldn't be as compelling as the first, so T. Harry Williams' Huey Long retains its place at the top of my list. Although one of these days I should probably reread that to see if it's really as good as I remember.

Milton Friedman on the efficient market hypothesis

Justin Fox 4 Comments

In the orgy of eulogization and evaluation that has followed the death of Milton Friedman, a couple of odd myths have been promulgated.

One is that his monetary theories have been discredited. What has been discredited is Friedman’s belief that monetary policy could be run on a purely automatic basis, by simply allowing the money supply to grow at a certain pace. It turns out the money supply is far too slippery a thing to measure accurately enough for such a policy. But Friedman’s main messages — that inflation is always and everywhere a monetary phenomenon, and that central banks should focus all their energies on keeping prices stable — have been accepted at the Federal Reserve and pretty much every other respectable central bank on the planet.

Another myth, central to a Michael Kinsley essay in Slate this week that I dissected in the Curious Capitalist today, is that Milton Friedman believed that financial markets were efficient. Shawn Tully at Fortune does a great imitation of Friedman saying something along the lines of, “Yes, over the long run, markets might be efficient, but in the short-term …”

I didn’t get anything quite like that out of Friedman when I interviewed him on the subject in 2004, but he did, as always, deliver an interesting quote. I had mentioned that Friedman’s friend and long-time intellectual ally George Stigler had told then-Chicago-grad-student Baruch Lev at a cocktail party in the 1960s that he didn’t believe in the efficient market hypothesis.  Friedman responded:

You don’t have to believe it. I don’t believe it. We all know the market is not efficient in a descriptive sense. But that doesn’t mean that the efficient market is not the best approximation if you don’t have anything else to use. …Warren Buffett proves that there’s not an efficient market, and yet Warren Buffett is what makes the market efficient, and both statements are right. If the market were 100% efficient, nobody could make any money making it efficient, and then it wouldn’t be efficient again. So in a way it’s self-contradictory to suppose that there really is an efficient market.

Friedman didn’t believe markets were perfect. He just thought that they were better, and more accommodating of human liberty, than government. He may have oversold that argument on occasion. But don’t go calling the man an efficient marketeer.

Glenn Hubbard, video star

Justin Fox 2 Comments

This is a Columbia Business School student, playing Columbia Biz Dean Glenn Hubbard as Sting (or is it Sting as Glenn Hubbard?), complaining about Ben Bernanke getting the Fed chairman job (instead of him) to the tune of the Police’s “Every Breath You Take.”

It’s brilliantly done, with some nice Fed jargon slipped in and even some pretty good singing. And it’s one more indication that the new era of user-generated content (or, as I’ve seen one blogger more eloquently if tendentiously put it, “authentic media”) holds untold riches in store. In the past, something like this would have been done at a talent show seen by a couple hundred people. By putting it up on YouTube, the creators have immediate access to a global audience. They posted it last Thursday. As of mid-morning Monday, it had been viewed more than 108,000 times. And I’m betting that on this one the viral contagion is just getting going (I first heard about it this morning). [I had to switch to Google video because the YouTube link stopped working.]

Of course, it’s only a parody. It’s interesting that so much of the “authentic media” being created these days consists of commentary on or parody of the creations of those of us in what I guess has to be called the inauthentic media. But maybe this is just the transitional phase. The popular music industry seems to be starting to come out the other side of the great Internet destroying-and-reinventing machine, and I’m mostly of the impression that the new iTunes/MySpace/podcast infrastructure is a lot better at getting interesting music to people than what went before. It’s just that, not having any clear idea of what the rest of the media is going to look like — and being employed by a big media company — I can’t help but worry about my paycheck.