Jack Coles’ piece “Rise of Data Analysis in Football” was named in A Football Report’s ‘Best of Football Writing 2013‘ list. The second year in a row that Outside of the Boot has featured in the list. He returns with another fantastic piece..
‘Our results suggest that coach turnover does not produce any statistically significant positive effect on team performance’
The Effects of Managerial Turnover: Evidence from Coach Dismissals in Italian Soccer Teams
Maria De Paola and Vincenzo Scoppa, University of Calabria – September 2008
Allow yourself, for a moment, to forget about the chores you imagine await a football manager on a day-to-day basis and enter a world in which that individual manager is in charge only of flipping a coin. The manager’s boss, the chairman or CEO, states that the coin must land on heads roughly 50% of the time and tails roughly 50% of the time (allowing for a 7% margin of error, say), when the manager flips the coin every weekend. In week one the manager gets a tails, week two a heads, and weeks three through ten he gets a tails every time. The manager, employed to achieve an even split, has obtained one heads and nine tails. The manager is fired. He wasn’t doing his job.
We know this isn’t true, however. The coin will naturally (over thirty-eight weeks, say) land on heads roughly 50% of the time and tails roughly 50% of the time, given a long enough period. It’s just a case waiting for the coin tosses to average out. But the manager’s boss doesn’t care, and brings in a new manager all the same. The new manager tosses the coin and gets a heads in weeks eleven and twelve. The new manager’s boss has done a great job, he thinks, and sits back until week thirty-eight, and sees the new manager has achieved exactly what was asked of him, and where his predecessor had failed. The coin fell 56% of the time on tails, and 44% of the time on heads. What a great season, the new manager’s boss thinks, ‘I was right to fire the first manager.’
But we all still know this close-to-50/50 split would have happened anyway.
Whilst the sport of coin tossing would be presumably less lucrative than football, its allegorical value is immeasurable. For some football teams, their manager’s task is very similar to tossing a coin every weekend this season.
In the 2013/2014 Premier League season, Martin Jol, Malky Mackay, Steve Clarke, and Paolo di Canio have been fired, whilst Andre Villas Boas and Ian Holloway parted from their clubs officially by mutual consent. This figure (six) is currently just about consistent with the average number of managers that lose their jobs over a single Premier League campaign in the last ten years. Although last year, whilst the total amount of sacked managers stood at eight by the end of the 2012/2013 campaign, that the figure only stood at two by New Years Day. In the last 8 seasons, just under 4 managers have lost their jobs in the second half of the Premier League tournament, or in the off-season. With this in mind, this season is set to be one of the highest for managerial turnover in Premier League history.
But, crucially, does firing your manager make the slightest bit of difference, in terms of improving results? Five of the above managers have been replaced, and only West Bromwich Albion at the time of writing have not employed another head coach (in fairness, they have a slightly different management structure to most other Premier League sides). Aside from the fact that at least two clubs clearly did not have replacements lined up immediately (something which is frankly impossible to understand – if a manager is fired with such regularity, and the person responsible (presumably) has been deliberating for some time, then why do clubs who fire their managers not have their replacements identified and installed straight away, à la Meulensteen, if they are so important to the team?), have the clubs who have re-employed managed to help the situation that their owners felt so despairing that dismissing the manager was the only option?
The answer is probably not. The topic of managerial sackings has interested reams of academics, all of whom have performed various tests on the many football leagues throughout the world, and concluded, by and large, one consistent observation – managerial turnover is largely ineffectual when trying to enhance sporting achievement.
In layman’s terms, sacking your manager mid-season does not improve results.
Bas ter Weel of Maastricht University in the Netherlands, who is perhaps the best-known authority on the subject of the utility of firing football managers, used data from the Dutch Eredivisie in his paper, ‘Does manager turnover improve firm performance? New evidence using information from Dutch soccer, 1986 – 2004’, and found ‘manager turnover is not followed by significant improvements in firm performance and that manager quality does not seem to predicted manager turnover. These results are robust to different specifications’. Ter Weel did state, in an article with the BBC in the summer of 2013, that ‘changing a manager during a crisis in the season does improve the results in the short term,’ but, ‘this is a misleading statistic because not changing the manager would have had the same result.‘
Although this seems to run counter to the narrative spoon-fed to us by footballing television media outlets, who contrast a montage of Paolo di Canio skidding on his knees at St. James’ Park to an upbeat rock song with a black and white, grainy image of Martin O’Neil rubbing his face in resignation to a melancholy, acoustic melody, the central theme is that teams who stick by their manager generally see an improvement in results anyway. Essentially, no matter who is in charge, our metaphorical coin just starts to flip the other way again – known as regression to the mean.
Hannah Barnes, who used Ter Weel’s data in her afore-cited piece on the BBC website, stated ‘while Ter Weel’s research focused on Dutch football, he argues that this finding is not specific to the Netherlands. Major football leagues in Europe, including England, Germany, Italy and Spain also bore out the same conclusion – teams suffering an uncharacteristic slump in form will bounce back and return to their normal long-term position in the league, regardless of whether they replace their manager or not.‘
And Barnes and Ter Weel are correct. The conclusions of a study on the Italian league are quoted above the article; academics Andres Giraldo, Juan Mendoza, Andrés Rosas, and Dayana Tellez found that ‘using data from the Colombian first-division soccer league between 2003 and 2010 we do not detect any improvement in team performance following the hiring of a new coach. We find that a new coach does not statistically change the number of points, the score difference, or the numbers of goals scored and allowed. Our results call into question the commonsensical view that managerial turnover is efficient.‘ Sandra Maximiano from Purdue University analysed data from the Portuguese League in a March 2012 study, and found ‘firing the coach does not improve teams’ performance. In fact, it seems that teams that decided not to fire the coach, even when they also experienced a spell of underperforming results, recover better than the ones that replaced the coach.’ Sören Salomo and Kai Teichmann of theUniversity of Kiel concluded, ‘although the dismissal of the manager is supposed to improve team performance this effect does not seem to be supported by the data on the German premier soccer league,’ in their 2000 paper, ‘The Relationship of Performance and Managerial Succession in the German Premier Soccer League’, and Anne-Line Balduck and Marc Buelens of the University of Genk agreed their analysis ‘reveals no evidence to attribute the performance recovery following a change of coach to his/her successor and rejects the hypothesis of the effectiveness of coach turnover,’ and finally Selman Kaldiroglu of Trinity University in San Antonio, Texas adds further weight to the argument based on data from Turkish football, found in the paragraph below.
So why do teams fire their managers at all? As promised, Selman Kaldiroglu sums up the probable reasons well, in the excellent concluding remarks in the economists’ 2013 paper ‘Statistical Evaluation and Efficiency of Mid-Season Managerial Turnovers: Signs from Turkish Soccer Between 1998-2012‘ by stating, ‘The scapegoat hypothesis asserts that dismissals take place in order to find a person to blame for the performance. Since we see that the mid-season dismissals do not matter from our results, scapegoating seems to be a high possibility‘. Kaldiroglu’s research also finds, ‘dismissals occur quite frequently in businesses and in soccer clubs… the constructed control group shows the regression to the mean effect that casted doubt on the effectiveness of manager terminations. Comparing the results to the regression group data, it was found that firing coaches does not help team performance’.
André Villas-Boas has seen himself become something of a monument to the principle of scapegoating. Now that he has left Tottenham Hotspur, Villas-Boas has had no joy at two London clubs, without performing badly at either. When he was fired as Chelsea FC manager in March 2012 Chelsea were 5th, and by the end of the season under his replacement Roberto di Matteo, Chelsea FC had dropped a place to 6th. Also, when Andre Villas-Boas’ time had come to an end at Tottenham Hotspur, he had achieved more points this season than he had the season before, and still has the third highest win percentage of any Tottenham manager since 1898.
The scapegoat idea is nothing new though. Although perhaps only obvious from undertones in club press releases concerning a managerial dismissal, managers are usually fired simply to give a message to the fans that the club are addressing poor results, and the narrative of the season then changes in the eyes of the players, the media, and the fans for the better, even if it does nothing to the results. In this sense, maybe firing the manager is a good idea. All of the above parties are usually willing to give the new incumbent a free pass, until the end of whatever season he has intruded into, as was the case with Roberto di Matteo.
So next time a manager is fired because of poor performances, try not to get to excited about the replacements that are found. If your team are in crisis due to results, the best thing the chairman or CEO could usually do is……..nothing. Without investment, the team will almost always carry on exactly as they could have expected to anyway. If you are getting a lot of tails, the coin will always start to flip the other way. As Hannah Barnes of the BBC states, the, ‘theory seems to work if you look at what happened to other clubs in the English Premier League last season. The same week in March which spurred Sunderland to change the personnel in charge, Aston Villa were sitting at 17th in the table, struggling against relegation. Villa too had been struggling to find form. Instead of dispensing with the services of manager Paul Lambert, however, the Villa board hung on and witnessed a turnaround in form. Over the same period of five matches Villa notched up two wins, a draw and two losses – exactly the same results that Paolo Di Canio achieved in his first five games in charge.‘
