NBA Playoff Win Probabilities

Live win probabilities for NBA playoff games are available at wp.advancednflstats.com/nba. Previous final games are here. The WP graphs include new interactive features and stats. The new stats are explained in this post. Check them out and see what the NFL graphs will look like this fall.

May 26, 2009

Are NFL Coaches Too Timid?

Risk is at the heart of football strategy. Aggressive, risky gameplans should result in boom-or-bust high-variance outcomes, sometimes scoring lots of points but sometimes scoring very few. Conservative gameplans result in relatively consistent low-variance outcomes. Teams would more likely score close to their average score.

In this post, I’ll look at what high and low variance strategies would look like in terms of point totals and how they affect each team’s chances of winning. I’ll also compare the theoretical strategies to the actual distributions in the NFL. We'll see why NFL coaches should be more aggressive when they're the underdog.

High Variance Strategy in Basketball

Some time ago, I came across an article posted by basketball researcher Dean Oliver that analyzed high and low variance strategies for the NBA. Oliver calculated the win probability of each opponent according to the mean and standard deviation (SD) of each team’s scoring tendencies. SD represents the degree of variance. The more aggressive and riskier the strategy, the higher the SD will be. For example, a basketball team that shoots lots of 3-pointers would have a high variance.

The key to accurately modeling basketball is realizing that each team’s score is correlated with that of its opponent. The pace of a basketball game ties each team’s score together, and there is a high level of covariance. When one team scores a high number of points, the other team will tend to score more too. Game scores are interdependent.

In Football

Recently the Smart Football blog illustrated the advantage of high variance strategies for underdogs. A high variance strategy increases an underdog’s chances of winning but comes with the cost of also increasing its chances of being blown out.

In the NFL as a whole, visiting teams average about 19 points with a SD of 10 points while home teams average about 23 points with a SD of 10 points. But unlike basketball, football opponent scores are negatively correlated. This makes intuitive sense because the better one team does, the worse the other should do. If one team gets lots of first downs and doesn’t commit turnovers, its opponent will usually start drives with poor field position, and vice versa. The covariance between NFL opponent scores is -1.9 points-squared.

If NFL scores were normally distributed, this is what the typical score distribution would look like. The visitor scores are in red and the home scores are in blue.


We can calculate each team's chances of winning by summing all the probabilities with these distributions and factor in the covariance using Dean Oliver’s method. This estimates that the home team wins 56.5% of the time, which happens to be exactly the NFL actual home field advantage.

Disclaimer

There’s one problem. NFL scores are not normally distributed, primarily due to its unique scoring, which typically comes in chunks of 3 or 7. Here is what the actual distribution of scores looks like.


The good news is, if we group the scores into bins of 7 points, we get a quasi-normal distribution. (Technically, it may be more of a gamma or Poisson distribution.) I’m going to stick with normal distributions to simplify the math and to better illustrate the concepts I want to convey.



Demonstration

Here’s why underdogs should play aggressive and risky gameplans. Take an example where one team is a 7-point favorite over its underdog opponent. Say the favorite would average 24 points and the underdog would average 17 points. With a SD of 10 points for each team, the underdog upsets the favorite 31.5% of the time. The favorite’s scoring distribution is blue and the underdog’s is red.


But if the underdog plays a more aggressive high-variance strategy, increasing its SD to 15 points, it would upset the favorite 35.3% of the time.


Note that I haven’t increased the underdog’s average score in any way, just its variance. The increase in its chance of winning results due to more of its probability mass moving to the right of the favorite’s mean score of 24. In fact, the higher the variance, the wider the probability mass will be spread. Consequently, more mass will be to right side of the favorite’s average score. But more mass will also be to the left, meaning there is a higher risk of an embarrassing blowout.

Even if employing a high-variance strategy is non-optimum, it can still help an underdog. In other words, even if an aggressive gameplan results in an overall reduction in average points scored, it often still results in a better chance of winning.

The next graph plots the scoring distributions of just such a scenario. Like before, the favorite’s average score is 24 with a SD of 10. But this time the underdog’s average is reduced from 17 to 16. The increase in variance still results in a slightly better chance of winning despite its overall reduction in average points scored. In this case, it's 33.2% for the underdog.


What about the favorite? Should it increase its variance in response to an aggressive underdog? No. Ideally it should play as consistently as possible. The lower the variance the better for the favorite. The next example shows a favorite playing a low-variance game with an average of 24 points and a SD of 5 points. The underdog is playing conventionally with a 17 point average and 10 point SD. The result is an increase in the favorite’s chances of winning from 69.5% in the original example to 73.0%.


And if the underdog plays an aggressive high-variance game, the low-variance strategy is still better for the favorite. In this case the favorite still improves its chances of winning from 64.7% to 67.8%.


In Practice

So what does any of this mean in the real world? Simply put, to win more often underdogs should employ a high-variance strategy from the beginning of the game. It shouldn’t wait until the 4th quarter and become desperate. Go for it on 4th and short, run trick plays, throw deep, and blitz more often. Roll the dice from the get-go.

The real question is, what is the optimum level of risk? I’m not sure, but I do know NFL coaches are operating far from it.

Looking at games from the ’02 through ’06 seasons (a total of 1280), underdogs do not increase their variance. For example, for games in which the point spread is between 6 and 7.5 points, the underdog’s SD is 9.8 points, slightly less than the overall league average. Ideally, it should be higher. The favorite’s SD is 10.4 points when ideally
it should be lower.

The table below lists the SDs of points scored for the favorite and underdog according to the most common point spreads.









SpreadFavorite SDUnderdog SD
0 - 1.59.610.5
2 - 3.59.89.4
6 - 7.510.49.8
10 - 11.5
10.58.7



If anything, there appears to be slight trends in the exactly wrong directions. The bigger the spread, the smaller the underdog’s variance and the bigger the favorite’s variance. It appears underdogs may get less aggressive while favorites may get more aggressive.

Conclusions

This is more evidence coaches do not coach to maximize their team’s chances of winning. My theory is coaches are delaying elimination until the latest point in the game—that is, trying to “stay in the game” for as long as possible. Underdog coaches minimize risk all game long hoping for a miracle along the way. They seem to be reducing the chances of being blown out, but this is not consistent with giving their team the best chance to win.

But if you think about it, this kind of approach might be good for the NFL as a whole. It keeps games entertaining as long as possible, and keeps viewers tuned in.

Coaches of favored teams could be accused of the same crime. They might be playing with too much variance. But there is certainly a limit to just how consistent a team can be, no matter how hard it tries. There will always be random variation in team performance. I suspect a SD of 10 points may be near that limit, and that coaches of both favorites and underdogs simply play the least risky game they can consistent with accepted conventions.

24 comments:

Mark M said...

Interesting stuff. As a question, have you looked whether there's any change in SD among underdogs in division games?

It often seems that underdogs are happy to lose by a few points if they don't mind losing to their opponents. No-one wants to lose to a division rival though, and teams may be more willing to risk being blown out in an effort to get the win that no-one thinks you'll get.

Justin said...

Going for it on fourth and short is a statistically sound maneuver based on how easy it is to gain one or two yards. I wonder what would happen to projected outcomes if one team just decided not to punt unless they were buried in their own territory?

More apropos to this post, though, and coaches typically not maximizing their teams' chances of winning any given game, is it possible that this is less that they're stringing the game along, hoping for a mistake, or is it that they're stringing the game along, knowing that a close loss they were never really doing to win anyway is less likely to get them fired than a blowout where they really gave it a good shot?

Justin said...

...never really going to win anyway...

JMM said...

Should then the underdog play the "high variation" strategy until ahead, then becoming the favorite and going conservative?

drichters said...

As you alluded, whether this strategy is sound depends on what happens to the distribution when they switch to "high-variance" mode. I doubt it is safe to assume that the mean does not change, or that it only changes by 1 point. If "high-variance" mode shifts the mean by 3 instead of 1, is it still helpful?

Dan said...

I'm with you on the theory, but I don't think your "in practice" data (on the SD of favorite & underdog scoring) tell us what we need to know, because a team's strategy influences the distribution of their opponent's points scored as well as their own points scored. Risky defensive tactics (like blitzing) directly affect the opponent's offense, and other decisions (like going for it on 4th down instead of punting, or running plays with a higher risk of turnovers) seem like they should influence both teams' scoring distributions.

Brian Burke said...

Mark-no, I don't have division games coded in the database.

Justin-I agree. It's probably all about job security. That's certainly rational, plus fans don't like blowouts either.

JMM-Yup. I suppose if an underdog gets a considerable lead, it's not really the underdog anymore.

drichters-Good point. I don't know the break even point. My examples of SDs of 5, 10, and 15 points are just used to illustrate the concept. In reality, the answer is dependent on how much of an underdog you are, and how an increase in risk will affect your mean score. With some aggressive strategies, like going for it on 4th down, would actually be more optimum and improve the mean score.

Dan-Good point also. Perhaps we really want to know the SD of point difference, not points scored. But defenses don't really have many ways of increasing risk/reward compared with the offenses. I really doubt it's symmetrical like that. Plus, the covariance was -1.9 points-squared, which is actually quite small. So looking at just points scored is going to hopefully be a pretty good indication.

Phil Birnbaum said...

I really liked this post. Well explained, good graphics, empirical evidence ... made my day!

DoubleB said...

It is nearly 100% about job security which is more valuable than gold in the coaching profession.

Two examples:

Minnesota loses a meaningless bowl game to Texas Tech at the end of December by a score of 44-41. Two days later Glen Mason, Minnesota's football coach, is fired out of the blue. The AD admits it was because of that football game, one in which Minnesota lost a 31-point lead in the 3rd quarter. The RESULT was irrelevant to Mason, who had just signed a long-term extension the year prior. It was the unfolding, or the HOW of that result that got him canned.

Using an NFL example, is Bruce DeHaven, a well-respected coach, fired if the Music City Miracle occurs at any other time in his career? I would guess no. The HOW and the WHEN of the result is what got him canned.

These are obviously extreme examples, but the old canard that it's all about wins and losses just isn't true. HOW you win and lose matters.

Jon said...

Brian, Dean covered this in more depth in Basketball On Paper. Have you read that book?

Brian Burke said...

No, but I take it you recommend it. I'll put it on my list. Can you believe his article that I linked to above has survived online since 1995? Amazing.

Chase said...

Good stuff, Brian.

One thought -- while the covariance between points scored and points allowed may be negative normally, I'm not sure that would remain consistent if the underdog adopted a risky strategy. If they are throwing deep often, that leads to short drives and not taking time off the clock, which gives the opponent more possessions. Similarly, blitzing more often leads to shorter drives.

I agree that coaches in general and underdogs specifically are too conservative, though.

Anonymous said...

A lognormal distribution is probably best for this exercise given points can't drop below zero.

This is similar to a gambling argument. If you gamble in vegas, you're the underdog. so your best chance is to use your entire bankroll on one bet for a 45% chance of doubling your money, vs. a much lower probability on repeated small bets.

I wonder if coaches applied this strategy historically in the Super Bowl, and that's why we used to get so many blow outs. i.e. I wonder if super bowl scoring differential volatility is higher than the regular season.

Anonymous said...

Interesting analysis. I think there's more psychology behind current behaviors than just job security, though. For example, if a coach abandons his team's regular game plan / strategy and goes high-risk, he's basically saying "you know what, our usual approach is inadequate." I would think that even a coach who's secure in his job wouldn't want to send that message to his players. Perhaps the "delaying elimination" strategy is actually better for player morale. Players on a bad team might feel better about 3 close losses than it would about 1 lucky win and 2 blowout losses.

With all that being said, it still makes no sense to me that the Lions didn't go no-huddle and run more trick plays once that Thanksgiving blowout put them at 0-12. The D couldn't stop anything, so obviously the O needed to put more than 21 points on the board most weeks to have a chance.

Anonymous said...

The 2001-04 Steelers make an interesting example on this topic. In '01, they were #1 in rush yds and #21 in pass yds. They lost to the Patriots in the AFC championship game, and Cowher admitted that this loss made him rethink how aggressive his offense should be. In '02 and '03, the Steelers switched from Kordell to Maddox at QB, threw more (~90 attempts/year more), and got worse. After a lousy '03, they put in a rookie QB, threw a lot less, and won the Super Bowl.

Obviously the issues are complicated (did they win because they run more, or did they run more because they won?), but Cowher definitely made the offense more aggressive in 02-03 and less aggressive in 01 and 04, and there's no question that the Steelers were a better team with the more conservative offense. (In this Steelers fan's eyes, anyway)

Justin said...

Steelers fan here also. I'm pretty sure they run when they're winning and pass when they're behind to catch up, but most teams do that.

And not to pick nits, but Roethlisberger went to the AFC Championship (again against the Pats) in his rookie season and won the Lombardi the following year.

parker said...

steelers fans,
Since they won the championship wouldn't you say they had the superior teams and therefore proved the article right.

Brian,
I was always looking for a way to put into numbers why rivalry games seem to have more unexpected outcomes. Thanks for this article.

Anonymous said...

Oops, you're right Justin. I should've said that the Steelers won the Super Bowl in 2005.

While it's true that the Steelers, like most teams, run more when they're winning, I really think they tried to throw more in general with Maddox in 2002-2003, and Cowher did say that he wanted to build a more pass-heavy offense during that time. With the arrival of Big Ben, they switched back to a more conservative offense. Whether that was a choice based on the theoretically ideal strategy or a practical one based on having a young QB, I can't say. Probably some of each.

Justin said...

They definitely scaled back when Maddox got hurt because of Ben's inexperience. I could be wrong, but I don't view Cowher as having much of a head for numbers. :)

@parker
Yeah, maybe. I think they definitely had the superior teams in the Super Bowls, but I'm not so sure it was the case in all the playoff games leading up to the championships, especially in 2005.

Justin said...

More to the point, if you go back and watch those playoff games in 2005, Whisenhunt had the Steelers come out throwing the ball, unlike their regular season strategy. If this is an indication of being an underdog (and they were, given they were the 6 seed), then maybe we're on to something for sure.

parker said...

I seem to remember them being much better than a "normal" sixth seed. They had superior personel and were coming off a 15-1 season.

Jim A said...

In order to observe the variance in strategy based on point totals, I'd think you'd have to adjust for the fact that among mismatched teams, the strategies will likely change due to the current score as the game goes on. So it may be that if heavy favorites become more conservative later in the game as they build a lead (and thus underdogs take more risks) the numbers shown in the last table may actually understate the early-game strategic inefficiencies. However, as others have suggested, there might be other in-game strategy changes that affect the results.

Also, while I agree that in theory favorites should be more conservative off the bat and underdogs should take more chances, I don't think anyone has a clue how to modify your team's level of risk-taking without significantly changing your average point-scoring ability. In other words, by increasing its SD to 15 by employing a high-risk strategy, the underdog in your example may be reducing its average points to something so far below 17 that it becomes counterproductive. Recall that the Sackrowitz Chance paper on ball control offense found that the probability of winning is so sensitive to changes in scoring efficiency that underdogs attempting to deviate from optimal pace will likely reduce its ability to score too much to make the strategy worthwhile.

If you somehow defined "conservative" and "risky" decision making, you should be able to show using historical data that underdogs that took more chances did indeed increase their odds of winning (even unintentionally). And vice versa for favorites. But I'd bet that such an effect wouldn't show up empirically because the margin for error is so thin.

And of course, this is all complicated by overwhelming evidence that, when it comes to 4th down decisions anyway, teams don't make optimal choices in neutral situations to begin with.

Michael Schuttke said...

One thing I find interesting about this research is how, in spite of many coaches inability to go against their own tendencies, it does actually reinforce certain strategic trends.

For example, how often do we hear during a season where a particular team is having an explosive offense that they "should consider running the no-huddle more often" so as to put pressure on the defense, gain favorable mismatches due to limiting personnel/formation adjustments that the defense can make, etc. I would say that this research actually enforces this line of thinking; if one team is across-the-board more talented than another, then increasing the number of opportunities to display that personnel/execution advantage (i.e. run more plays) would logically cause each team to move more towards its performance mean if you will.

So, in that, I think it is wise for an underdog to almost NEVER run a no-huddle offense (unless their offense is considered better than the other teams defense). With that said, the same said offense should then, as you said Brian, throw deep more often, run trick plays, perhaps use more unusual formations/motions/formation shifts, etc. but that the actual tempo of the game should be fairly slow.

I would be interested to look at teams during what would be deemed a dynasty era that "overachieved" early in their dynasty run (and see how their use of the no-huddle compared with that of their more dominant years later on. I am willing to beat we would see a faster tempo from the more talented team than the early team.

For example, the 2001 Patriots were, by most standards, considered a team that had mediocre, 8-8 type talent on their roster that went on to win a Super Bowl. I would be willing to beat that the 2007, 16-0 Patriots who were eventually upset by the N.Y. Giants ran the no-huddle significantly more than did the 2001 Patriots...ditto the 49ers of 1989-1990 versus 1981 (first championship of that dynasty, also won by a team considered not overly talented).

As both Bill Walsh understood and Bill Belichick understand the value of analysis of tendencies (both their own and the opposition's) and metrics in determining strategy, I bet both dynasties would provide interesting support for the no-huddle variance strategy I am speaking of.

Brian Burke said...

Michael-I agree. That was my recommendation for the Giants in the SB 2 yrs ago. The more iterations in a contest, the more likely the better opponent will come out on top.

Underdogs and Reducing PossessionsSuper Bowl XLII and Team Possessions

Post a Comment