Tuesday, February 8, 2011

This is an extract from a letter I wrote to espn.com's TMQ, in which I defended Mike Tomlin's decision to attempt a two-point conversion late in their Wild Card playoff game against the Jacksonville Jaguars following the '07 season. The Steelers had scored just a touchdown, reducing their deficit to five point, with 10:25 remaining in the game. A holding penalty was called on the two-point attempt, moving the line of scrimmage back to the 12-yard line. At this point, many commentators (including TMQ) apparently believe that Tomlin erred in repeating the two-point try, rather than kicking an extra point. An exception was footballcommentary.com, whose calculations suggest that the break-even probability of success on the two-point try was 0.17 - a relatively low threshold. Unfortunately, the process used by footballcommentary.com is rather opaque, using a dynamic programming model based on league-average probabilities of possible game events. As a result, I have found in the course of my discussions of this issue that casual fans do not hesitate to dismiss footballcommentary.com's conclusions out of hand, whenever they violate conventional intuition on the subject, as was certainly the case here.

In order to vindicate footballcommentary.com's two-point conversion chart - and also to show how the break-even point can be modified based on our knowledge of the specific teams in question - I will attempt to work through the issue in a more step-by-step manner. This will undoubtedly sacrifice some degree of accuracy in comparison with footballcommentary.com, but should serve to validate their conclusions by showing how they can be approximated in a thoroughly transparent manner. While I will refer only to the particular case faced by the
Steelers in this game, the general intuition can be applied to many late-game situations where a two-point try must be considered.

Our approach will be to partition all possible outcomes of the game by conditioning over the possible future scores by our opponents after the possible two-point attempt, and then evaluating how the two-point conversion attempt holds up in comparison with an extra point in each case, with each case weighted by its estimated likelihood of occurring in the game. This would ordinarily be a very tedious (and very imprecise) process; in this case, however, we can safely assume that if the opponent scores twice more, our win probability is so small that it can safely be disregarded. (This may seem a controversial assumption, but I believe it is justified: even if the opponent scores two field goals, this requires that we score twice more as well - and their scoring possessions presumably consume a large amount of the time remaining in the game. So while not 0, the probability of winning if the opponent scores twice more following the two-point try is of an order of magnitude that its effect on the final conclusion will be negligible.) This leaves us only needing to consider three cases:

a) The opponent does not score for the rest of the game. Then the
two-point conversion attempt is clearly superior to the extra point,
as if successful, it allows us to tie the game with a field goal. So
it is superior from a win probability standpoint by a margin of P(2) x
P(FG) x 0.5.

b) The opponent scores a field goal later in the game. This is more complicated, as the extra point after this touchdown would allow us to tie the game with a subsequent TD + XP, while a two-point attempt, depending on its success or failure, would either leave us needing a TD + 2-pt in order to tie, or allow us to take the lead after a TD + XP. So the win probability value of the extra point, holding other things equal, is P(TD) x 0.5 (I'm assuming throughout this discussion that all extra points succeed, as this assumption works unequivocally against the case I am attempting to make in favor of the two-point try), and the win probability value of the 2-pt attempt is [P(2) x P(TD)] + [(1-P(2)) x P(TD) x P(2) x 0.5]. So the net value of the two-point conversion attempt rather than the extra point is the second expression minus the first.

c) The opponent scores a touchdown later in the game, and kicks an extra point (regardless of our two-point decision and its success/failure, they would have no reason to attempt a two-point
conversion of their own at this point.) Then it makes virtually no difference whether an extra point or two-point was attempted after this touchdown, as one (and only one) two-point conversion will be required in order to allow a field goal to tie the game, and it can be equally well attempted after the next touchdown.

So in case (a), the two-point try is unequivocally superior, and in case (c), the two-point and XP attempts are of equal value. This leaves only case (b) in which the XP could come out ahead. However, we can easily see that if we use a conservative approximation of 0.4 for P(2) in
these expressions, the net value of the two-point conversion is still positive, as it simplifies to 0.52 x P(TD) - 0.5 x P(TD). In fact, for any value of P(2) greater than 0.38, going for two strategically dominates the extra point, as it provides equal or greater win probability regardless of what the opponent does for the rest of the game. Thus, it is undoubtedly the right call in a normal situation.

Things become more interesting when one assumes that P(2) is significantly less than 0.38, as was presumably the case when Pittsburgh attempted their conversion from the 12-yard line. Then one needs to estimate values for P(TD) and P(FG), as well as for P(Opp. No Score) and P(Opp. FG), which gives the respective weight to be assigned to cases (a) and (b). Plugging in league-average values gives a break-even point for P(2) (i.e. where the gain from case (a)
exactly offsets the loss from case (b)) of 0.156 - quite close to the footballcommentary.com value of 0.17. However, one can pursue the question still further by using team-specific and game-specific values for the various probabilities. For example, using FootballOutsiders' Drive Stats, with a simple average taken between the values for Pittsburgh Offense and Jacksonville
Defense, or vice versa, gives a more appropriate value for the break-even point than using league average values does. (There may well be an even more accurate method of estimating these team-specific probabilities, but this suffices as a first approximation.) By this method, since the Pittsburgh defense was generally so good at preventing the other team from scoring, the break-even point for this particular P(2) is actually reduced all the way to 0.12. Surely this threshold is low enough that Tomlin's decision was justified. [In a discussion of Bill Belichick's decision to attempt a 4th and 13 conversion rather than kick a FG in SB XLII, footballcommentary.com uses a success probability of 25% in their analysis - the probability here is lower, since a defensive penalty would not result in an outright success, but certainly cannot be pegged below the 12% threshold.]

A simplified version of the "two-point conversion equation" for this game situation (trailing by 5 following the TD, and where the amount of time remaining is such that we can safely disregard situations where the opponent scores two or more times) is the following:

P(2) x P(Opp. No Score) x P(FG) x 0.5 - [P(Opp. FG) x P(TD) x [0.5 - (1.4 x P(2))] = 0.

This establishes the break-even probability for P(2). Analogous equations could be constructed for other game situations, and empirical testing of the degree of inaccuracy introduced by the assumptions could refine the conclusion still further. Regardless, the primary purpose of this discussion was to show that footballcommentary.com's chart of break-even probabilities makes good logical sense, and by obtaining a close correspondence with its result in one of the most apparently counter-intutive cases, I think that it succeeds in this endeavor.