CSFB

Thought Leader Forum
2003
Paul DePodesta

The Genesis, Implementation, and Management of New Systems
Related Links
  Biography

Concept Cards


As an introduction to the second edition of his pamphlet, Common Sense, Thomas Paine wrote, “perhaps the sentiments contained in the following pages, are not yet sufficiently fashionable to procure them general favor; a long habit of not thinking a thing wrong, gives it a superficial appearance of being right and raises at first a formidable outcry in the defense of custom.” Well, welcome to the world of baseball.

When I joined the Cleveland Indians in 1996, the baseball world was really rich for reform. Fans were still holding a grudge from the strike, salaries were exploding and small market teams were disappearing from the competitive landscape. In short, crisis was emerging and the existing operating paradigm in baseball was totally incapable of solving these new problems.

All of this was very bad for baseball at the time, but as it turned out, very good for me. In retrospect, I had a distinct advantage over everybody else in the industry at the time in that I knew absolutely nothing. I'd played baseball in college but that was about it. Because I knew nothing I observed everything critically and took nothing for granted. I spent my first few years with the Indians analyzing all of their systems, from contracts to player development and scouting. Because I had no preconceived notions over how an organization ought to be run, this was an education for me.



I realized very quickly that subjectivity ruled the day in evaluating players. I sat in a major league staff meeting at the end of 1996 after we had traded for Jeff Kent and listened to one of our staff members say, “Jeff Kent has the weakest hack I have ever seen.” So we traded Jeff Kent and watched him become the most explosive second baseman in the entire game—and it was with the Giants, not the Indians. I sat in scout seats behind home plate and listened to scouts rave about the five tools of one of the players—how he could hit, hit for power, run, throw, field—and I'm watching him swing and miss at another slider in the dirt for strike three.

To the untrained ear, these scouts were unbelievably convincing. Some of their subjective opinions almost sounded like they were objective. If you had worn a major league uniform at some point in your life, you were somehow qualified to make these judgments despite a complete lack of empirical evidence to support your claims. Don't get me wrong, subjectivity by itself isn't really a sin, and complete objectivity isn't perfect either. In our industry we make a lot of educated guesses on the future performance of people under very stressful situations. Subjectivity will be an element in any decision we make.

The incredible thing is that in subjectivity there are a lot of biases that come into play—emotional opinions or focusing just on outcomes, or even worse, focusing on the most recent outcomes. In baseball it can even take into account the player's physical appearance or worrying about what the press is saying all the time.

Evaluation is really at the core of decision—making whether the field of endeavor is baseball or picking stocks. It was clear to me that using clearly subjective evaluation was shoddy at best. The psychological biases I mentioned, and more, were all in play. Imagine if you made a huge investment in a company after just meeting the management and never even glancing at a financial report. Your entire evaluation would be something like, “the CEO seems smart; he's got a good body on him; and I'm still really angry with that last company that lost all of our money so I'm going to do something and I'm going to do it now.”

Major league teams were basically using this model and if you did, you were eligible to be a GM. We were making multi—million dollar bets on the future performance of the players. Baseball is an industry that is run by old time baseball people. If you weren't a baseball guy, then you better consult with baseball people before you made any kind of decision. But these people weren't always right.

The problem itself wasn't the people. Our scouts were very loyal, passionate, industrious people. The problem was the operating system. The industrial inertia was leading them further and further away from the truth. The operating system at the time, which I'll refer to as Subjective 1.0, was incapable of providing solutions to all the new problems the game was facing. Former Red Sox pitcher Bill Lee may have summed up the baseball operating system when he said, “In baseball you're supposed to sit on your ass, spit tobacco, and nod at stupid things.” That's America's pastime right there.

Despite this situation, I was grappling with a significant issue: the Indians were very successful at this time. We kept winning the division year after year, selling out every game in our stadium and the owner took the team public at one point and was making more money than any other owner. Thomas Kuhn wrote in The Structure of Scientific Revolutions, “As in manufacture so in science—retooling is an extravagance to be reserved for the occasion that demands it.” There was no crisis in Cleveland, at least not on the surface.

How was I supposed to innovate a supposedly smooth running machine? There was, however, a crisis underlying our success. Our lofty expectations had stifled our innovative spirit. Everything we had done to be successful, we stopped doing. We were hanging on instead of trying to move forward. We signed veteran, big name players who everybody knew. Our team got a lot more expensive and started growing older. Though I was seeing all this, I didn't have much of an audience in Cleveland.

Then the Oakland A's called and offered me the assistant GM job. At the time the Indians had one of the top payrolls in the game, about $71 million. The A's were near the bottom at $20 million. The A's were really in full crisis mode. In the past six seasons the A's had finished 161 games under 500. Attendance was in a freefall. To add insult to injury, about a year and a half earlier, the A's had traded Mark McGuire, the greatest power hitter of our generation. Then the next year they watched McGuire chase and break the single season home run record. They were at rock bottom.

What did I decide to do? I moved to the Bay area. This was the perfect opportunity because losing had become the expectation in Oakland. If we tried something really innovative and it didn't work, all we'd be doing is fulfilling expectations. To use a scout's term, there was a lot of upside. If somehow we figured out how to put a playoff caliber team on the field for pennies on the dollar, the baseball world would have to take notice.

It wouldn't be easy for us. First of all, no small market team had ever made the playoffs in the post—strike era. The A's like everybody else in baseball had ceased to do one very critical thing—to ask the naïve question: “If we weren't already doing it this way, is this the way we would start?”

Management guru Peter Drucker introduced this simple test decades ago and yet our public and private institutions are replete with things as they are because that's pretty much the way things have always been. Why is the workday 9—5? Why do we have the Electoral College? In baseball, why do people still believe that trying to bunt and steal bases helps in scoring runs?

Jim Pinkerton wrote a book called What Comes Next, and in it he wrote, “It's human nature to stick with traditional beliefs, even after they outlast any conceivable utility.” It was as if he were writing this specifically for baseball. So once I got to the A's I began a subtle, under-the-radar mission to ask the naïve question all over the A's organization. As you can imagine, some people didn't like it. Remember that the baseball industry is run by these old time guys with leathery skin who chew tobacco. Imagine Jack Palance in a baseball cap. And here's this young guy asking all of these questions like, why is our scouting system the way it is, what about our contractual strategies?

Pinkerton also wrote, “systems of any kind tend to degrade over time. Bugs accumulate, people figure out how to cut corners, and eventually they go through the motions and a lowest-common-denominator mentality prevails. And as the original purpose is forgotten, reflexive self-perpetuation becomes the only goal.” This is the world of player evaluation in three sentences.

At first it may have seemed charming that this young guy was asking all these questions—he seemed to really want to learn how things were done. But the questions became a little more detailed and that must have seemed threatening because that reflexive self-perpetuation goal definitely kicked in. Previously there had been no accountability at all in player evaluation. As you can imagine, the scouting community prized that tradition. How were they to feel free otherwise about giving their opinions and keep getting raises at the end of the year despite how often they were right or wrong? I was still waiting for the time I would be sitting in an organizational meeting and I would ask a scout, “well Bob, what do you think of this player,” and he would answer, “well Paul, he had a good beat, I could dance to it and I'd give it a nine!”

There were some moments where the need for reform was self-evident. The baseball salary structure was unusual, but one of the things that we do is that the club has the right to pay the player the minimum salary. One club decided they'd give their players raises even though they didn't have to. Then some other clubs joined the suit. Suddenly all 30 teams were negotiating these contracts where the players had 100% of the leverage.

When I was first in Oakland I was sucked into this practice. I was negotiating a contract for a player who had hit .220 the previous year. I sat there for hours during spring training negotiating his contract. We had narrowed it down to about $1,000. The entire conversation centered on one other player on our team who was pretty bad himself, who was scheduled to make $500 more. The agent gave his 45-minute plea one last time. I summed it up this way for him, “so you know your guy is awful but he's not as awful as the other guy?” Again, the need for reform was self-evident.

The response to all this questioning was somewhat expected. There were opinions layered on top of opinions. A lot of people were saying, “I think it's because,” or “maybe it's this,” or even “that's the way we've always done it.” My industry is comprised of human capital—the players are our assets. So subjectivity plays some role. But the enormity of the subjectivity was staggering. Our scouts even started making up vocabulary like “pitchability” to describe players.

Opinions are great—don't get me wrong. They're great for starting research projects. Then you go study and see if you can prove the opinion or not. But when placing multi—million dollar bets on future outcomes, opinions are wholly unsatisfactory. Opinions as conversation starters are fine. Opinions as conclusions are very bad. I started research projects to discern the objective “why.” I wanted to know why certain teams won and why other teams lost; why certain drafts produced big stars and others didn't. This was the naïve question at work.

Then I encountered another problem in baseball—information overkill. Between stats, scouting reports and ESPN, there's too much information and it's difficult to decipher what was important and what didn't matter. Naturally, our brains go searching for cause and effect relationships, but there was too much noise. The problem was that baseball people would draw conclusions from baseball stats that just didn't matter. It was difficult to distill what was important. We started making up relationships and people bought into them and the myth was perpetuated.

Miguel Tejada is our shortstop and last year he was the American League MVP. He's going to be a free agent at the end of the year. We sat him down at spring training and told him that we probably wouldn't be able to afford him. He agreed that he would love to keep playing in Oakland but understood the economics of the game. His announcement to the press was a big deal. After about six weeks he was hitting .160. The media said it was because he was worrying about his contract. Finally I said, “Miguel grew up in the Dominican Republic. There was a time in his life when there was no floor in his house. I think he has played baseball with more on the line than whether he was going to be really rich or obscenely rich.” We told the media that by the end of the year he would have better numbers. There is so much randomness that people draw conclusions from. It leads to a lot of inefficiencies.

I was on a quest to find relevant relationships. Usually it wasn't as simple as “if X then Y.” I was looking for probabilistic relationships. I christened the new model in the front office: “be the house.” Every season we play 162 games. Individual players amass over 600 plate appearances. Starting pitchers face 1,000 hitters. We have plenty of sample size. I encouraged everyone to think of the house advantage in everything we did. We may not always be right but we'd be right a lot more often than we'd be wrong. In baseball, if you win about 60% of your games, you're probably in the playoffs.

One of the other problems is that the traditional metrics and stats used in baseball are muddied with so much noise that just didn't matter that I was having a tough time distilling all the information. I decided to throw it all out and start all over with no assumptions. I built a Markov model, or actuarial table, for the last five or ten years that recorded what had actually happened in the course of every major league baseball game.



From that research I was able to figure out that a man on first with nobody out is worth “X” runs and a man on second with two outs is worth “Y” runs. From there I was able to jump to understanding what it means to have someone who can hit a lot of doubles. What was the value of that event and others? I went a step further and asked who the people were who could add these value—enhancing skills to our team. Finally I was able to figure out what the cost of each of those activities was and what the margins were. This was process versus outcome. I just didn't believe the outcomes that the traditional stats were giving us.

Once the research was complete, debated and stress—tested (which took years) we had considerable new knowledge, and a lot of it was pretty startling. Now remember that we hadn't really invented anything. We had only discovered relationships that were already there. Fortunately for us, most of them were contrary to popular opinion. These discoveries ranged from broad philosophical ideas, such as the fact that 90% of the player population in major league baseball is replaceable by someone who makes less to the very minute detail, such as pitch counts or control of the strike zone. What I ended up doing was creating a whole new set of metrics around this objective core. When I was done we had stats but not in the traditional sense. It was an entirely new operating system. It wasn't an upgrade from Subjective 1.0 to Subjective 2.0. It was more like “Winning Baseball 1.0.”

Our next challenge was to implement the new systems now that we had the new knowledge. We fought this battle at two fronts. First, how were we going to use this knowledge to become better decision makers ourselves in the front office? Second, how were we going to overhaul our current systems and processes?

Dealing with our own decision-making was relatively easy. As long as we maintained some kind of emotional detachment that comes with running a casino and knowing that in the end it's going to work out, we would be able to maintain a high level of confidence and discipline. We would avoid making lazy decisions. This new confidence went a long way. Our new decision making process became so streamlined that rather than taking days to make a decision on a player we were able to make a decision on a player in about 10-15 minutes.

Michael Lewis in Money Ball referred to us as crazed K-mart shoppers who would take anything we could fit in our cart in under 15 minutes. I prefer an analogy with Warren Buffett or Charlie Munger. [laughter] Either way it became clear to us that the inefficiency in decision-making in baseball was vast.

Overhauling our systems and the though processes of our people was a bigger problem. Thomas Kuhn also wrote, “the emergence of new theories is generally preceded by a period of pronounced professional insecurity.” A lot of the people at the A's had been in place for 10-20 years. The good thing was that the relationships and trust existed. Unfortunately the reprogramming of the thought processes was going to be incredibly difficult.

Most of you probably know Malcolm Gladwell's book The Tipping Point. He talked about the spread of epidemics and he said there were three different types of personalities that need to be present to spread an epidemic. The first is a maven or information hound. The second is a connector, or someone who knows everyone from all walks of life. The third is a salesman. We had the information and we knew all of our people, so we needed to sell.

We still had two critical decisions to make. First, there was a fine line between drowning in data and providing data for proof. What exactly should we sell to our people to get them to buy in? Second, there are moments in time that carry with them paramount influence. When would be the opportune time for us to start selling? We knew that the right combination of sales pitch and timing would strengthen the epidemic. Remember we had just completed a six-year swoon. Finding the right time wasn't really easy.

Why would anyone believe us when we had no credibility? Many of us share a common psychological deficiency. We judge decisions based on the outcome instead of the time and the circumstances under which they were made. This happens all the time in baseball. They make trades and say things like, “we'll see in three or four years if it was a good decision.” That doesn't work for me because you can't go back and learn from the decisions because of all the variables that occurred in the intervening time. It makes replication of an outcome impossible.

I was in Las Vegas for a weekend playing blackjack. A person at the table to my right had 17 and said they wanted a hit. The whole table stopped and even the dealer asked if he was sure he wanted a hit. Finally he said he wanted a hit. The dealer deals the card and of course it was a four. What did the dealer say? “Nice hit.” But I'm thinking, you're kidding me. It was a terrible hit. Even though it ended up working out, it wasn't a good decision.

Outcome-based myopia actually gave us an opportunity in selling our concept that we could take advantage of. I realized that all we really needed to do was win some games and find a way to get into the playoffs. Then we could leverage this success by introducing all of these changes in our systems. We wouldn't need to go into in-depth analysis of how we came to all of our conclusions. If we won, people would buy into it. This is how the game had operated for 100 years.

So we made some aggressive changes during the 1999 season and up until the end of the free agent period we signed a bunch of players. We acted like we were a first place team even though we were far out of contention. When it was done we had transformed our roster to 88 wins and 74 losses. It was good for second place that year. The epidemic began to spread. As each season passed and we won more games and division titles and playoff games, we targeted another area of our operations that we could reform.

Once the system was in place, we needed to manage it. We implemented incentive systems and salary structures that were built upon our philosophies. We started a guerilla education program for our players, our coaches and our staffs. We knew that we couldn't just tell people they were going to start thinking a different way. Dictating thought process doesn't really work. We needed them to truly believe that they were going to use this new knowledge to develop our players and help us find new ones. It took time. We continually refined and retooled our systems.

It wasn't until the middle of 2002 that we were able to put our arms around the whole baseball operation when we went after our scouting system. During this time we never stopped asking the naïve question because the market kept shifting in the meantime. After we had asked the naïve question to identify those underperforming systems that we had, after we researched relentlessly, after we implemented and then refined each new process, and after our people bought into the direction, what happened? Over the next four years we won 383 regular season games. The Yankees won 383 regular season games. But better for us, the Yankees spent over $433 million on player payroll and the A's spent about $134 million. Off the field the industry called us lucky or heretics. That was the blue ribbon because you're only being truly innovative if you're really aggravating someone.

So is our current operating system 1.0 the answer? Probably not. We still have a lot of questions and solutions that are over the horizon that we haven't gotten to. Fortunately for us, the bulk of the baseball world is still working on upgrades to Subjective 1.0 rather than what we're doing or even finding a completely different system. Being innovative doesn't mean searching for upgrades over inefficient systems. It means searching for entirely new ways of doing things. We don't spend a lot of energy tweaking current systems that are inefficient.

Thomas Kuhn wrote, “the proliferation of competing articulations, the willingness to try anything, the expression of explicit discontent, the recourse to philosophy and to debate over fundamentals, all these are symptoms of a transition from normal to extraordinary research.”
CSFB | EMPOWERING CHANGE
Sitemap Contact Us Important Legal Information