Why do people overrate the punt?

My last post tried to show that the criticism of Belichick's fourth-down decision is largely misguided - that a very realistic value of the probability of the Colts scoring from a short field causes the decision to break even, even if all other assumptions are made so as to favor the punt. If I am correct, this would fit the general trend in NFL practice and commentary: most coaches choose to punt far more often than a probabalistic analysis indicates that they ought to, and most pundits compound the situation by criticizing coaches on the rare occasions where they correctly forego a punt (but almost never vice versa.) So perhaps we should follow up by looking at the meta-question: why are people so eager to believe in the punt? After all, if people are going to go around spouting irritating nonsense about sports, we can at least use this as a natural sociological experiment to identify the sorts of conceptual errors to which human beings are particularly prone. Here is a letter I sent to Boston Metro on Monday morning following the Colts game, in which I offer a possible answer for this in the case of punting:

--------------------------------

The mad scramble by fans and pundits alike to condemn Belichick's 4th-and-2 gamble on Sunday (believe it or not, the numbers suggest that he probably made the right decision) is evidence of a more general issue worthy of discussion: the punt is currently the most overrated strategy in American sports. Contrary to what the "Trust your defense!" crowd seems to believe, punting does not send the ball across a magical Rubicon that ensures that the other team will fail to score. It adds some distance - generally around 40 yards - to how far they will have to go, making it somewhat less likely that they will succeed. This can make it a valuable tactical option in certain game scenarios. But in a great many cases, it is merely a foolish waste of an opportunity to do something productive with an offensive possession, whether scoring points, or, in this case, running out the clock without the other team having a chance to touch the ball. The fact that NFL teams punt far more often than they ought to has been mathematically demonstrated several times: the ground-breaking analysis was done by David Romer of UC Berkley in a paper called Do Firms Maximize? Evidence From Professional Football, and espn commentator Gregg Easterbrook (aka TMQ) routinely exposes the folly of the automatic fourth-down punt in his columns.

So why is it so firmly ensconced in conventional wisdom that punting (in all but a tiny minority of fourth down situations) represents "playing the percentages," when in fact the opposite is often true? The answer seems to lie in a phenomenon known by economists as "risk aversion" - we are hard-wired to prefer the certainty of some fixed amount of most things (food, money, etc) rather than an uncertain outcome where we will either end up with a great deal, or none at all. This is true even if the weighted average of the uncertain outcome is higher than the certain one: when signing a ten-year contract, most of us would choose a guaranteed income of $50,000 rather than a possibility of either $5000 or $100,000, to be decided by a coin flip. (Risk aversion is one of the major reasons why people purchase insurance.) This is a sensible response to uncertainty when making economic decisions: personal well-being, not money, is the ultimate currency here, and for most people, being rich is not that much better than being comfortable, while poverty can be ruinous. But risk aversion is not rational when the uncertain quantity is win probability in sports, as this probability itself is the ultimate standard by which the decision is to be evaluated. (As such, it is not subject to decreasing marginal utility - a strictly linear relationship is maintained at all values.) Here, ten times out of ten, we should prefer a decision that will leave us with either a coin-flip 5% or 100% of winning to one that will guarantee us a 50% chance of winning.  While the specific numbers differ widely from case to case, this general template underlies the various possibilities arising from a fourth down decision. A punt will guarantee a relatively negligible impact on win probability - we don't score, and we lower the probability that the other team scores. Nothing much has changed. An attempted fourth-down conversion, on the other hand, will have a much greater impact on win probability in one direction or the other: either we keep the ball, with a good chance to put points on the board, or we give the other team the ball with good field position. We instinctively balk at this latter scenario, treating a coach who chooses it as though he were an investment manager exposing his client's 401k to ruin in pursuit of a big payoff. In fact, we should be celebrating coaches like Belichick who are bold enough to look past the irrationality of the conventional wisdom, and make decisions that put their team in the best position to win.