Route Two Soccer: Replacement Level in the NWSL – Part 1

Sports analytics has evolved by leaps and bounds over the past three decades. Arguably the single most important concept undergirding that explosion has been the idea of ‘replacement level.’ Indeed, replacement level is so significant that it has gained usage outside of the sports world in a variety of business and economic contexts.

What is replacement level?

Put simply, replacement level defines the level of performance that is readily and freely available. In the baseball world, where it first gained widespread usage, replacement level is roughly the quality you would expect if you promoted a career minor leaguer to fill a roster spot.

Importantly, replacement level doesn’t mean zero value. A team full of replacement level players would still pick up the occasional point here and there; these players are still very good, just not quite good enough to make a normal roster. What’s more, performance is always within a range. A replacement level player won’t turn in exactly-replacement level performances every game. They’ll sometimes play well, and sometimes play dreadfully, but over time the median result will center on replacement.

To get a rough estimate of what a team full of replacement level players would look like, the 2016 Boston Breakers and 2018 Sky Blue FC seem to fit the bill. Now, both of those teams had players that are clearly better than replacement level. But they also likely had some below that level. And they also both performed toward the bottom of their potential.

Why replacement level is important

Replacement level is critical for understanding team building and player value. While it is not the only way to assess a player, it is probably the single most important way of characterizing their utility to a team. This is because it exists in a goldilocks position between two extremes: measuring the positive function of simply showing up, without overvaluing mediocre performance.

On the one side, it is preferable to a pure counting-stat approach, which merely adds up a player’s positive contributions. It is of course true that a player who scores three goals in a season has added value over what would have happened if no one had filled that position. But that’s not how it works. A player who scores three goals from several dozen excellent chances that were created by her teammates is actually hurting the team by squandering the chances that someone else might have converted. This is a critical opportunity cost of occupying a valuable active roster spot—the value lost because the spot couldn’t be occupied by someone else. Replacement level attempts to quantify that lost value by identifying what could reasonably be expected from freely-available talent.

This concept of ‘freely-available’ talent also shows why replacement value is superior to a second intuitive model: value above average. It’s admittedly tempting to measure players against average. A player who is below average is a drag on their team in some sense. In order to win more than you lose, you need your players to be above average.

The problem is that, by definition, not everyone can be above average. In fact, roughly half the players will be worse than average. So if one of your regulars is injured or suspended, it is very unlikely that you will be able to replace them with another above-average player. Far more likely, your replacement will be, well, replacement level. Which demonstrates that the ‘merely average’ player is actually providing significant value. Simply by showing up and performing at a sustainable level, they are helping their team.

Limitations of replacement level analysis

Having laid out the general theory of replacement level, it’s worth taking a moment to acknowledge three big caveats, and provide some explanation for why the concept is still worth using.

First, replacement level is an abstraction. In the real world, it depends on context. In particular, if your team has several excellent left backs, the actual ‘replacement level’ is much higher than it would be for a team with no cover. Nevertheless, in terms of league-wide player assessment, the abstraction is helpful for characterizing the value that a given player can contribute. Each individual team will want to make its own assessments of positional scarcity, but for comparing players across teams, replacement level is consistent.

Second, soccer is a dynamic game, and player performances can’t be easily disentangled. To start with an obvious example: Alex Morgan is one of the world’s best strikers, but if you don’t provide her any service, she isn’t going to contribute much value. That’s not her fault, and it would be strange to decide that Morgan was not a valuable player simply because the team around her was bad. And more broadly, there’s the reality that some players will play well in certain systems but flail in others. Some players will excel if given a limited responsibility, but fall apart if they’re asked to do more. A dedicated defensive player might be quite valuable in a team set up to play with a true #6, but terrible on a team that needs creativity from every midfielder. A good striker might be hopeless if forced to play as a fullback. Or vice versa.

None of that is a reason to disregard the concept, but it is a reason to be careful about drawing assessments too broadly. Value over replacement is best set up as a framework for assessing what actually did happen, but it won’t necessarily provide much perspective on what would have happened if things had been set up differently.

Third, soccer simply isn’t as amenable to analytic assessment as many other popular sports. The state of soccer analytics is miles ahead of where it was a few decades ago, but even at the top levels of the men’s game, where data is much more plentiful and quite a bit of money is being spent, things are still far hazier than in sports like baseball or basketball.

Those complexities mean that it will never be straightforward to assign a single value to a given player, and any assessment along those lines should be taken with a huge grain of salt. Nevertheless, as a matter of theory-crafting, the concept is still useful, and it’s worth taking seriously as a framework for considering player performance.

Having laid out the framework, I’ll post the second part of this piece next week, which will discuss how replacement-level can help organize our thoughts on the league.

#NWSL