How well do FIFA’s ratings predict World Cup success?

Written by Ray Stefani on . Posted in Sports

The first match has yet to kickoff, but it’s not too early for the weeping and gnashing of teeth to begin. Fans of the 32 World Cup teams already worry that their group is too hard and their hated rival’s group is too easy, based on FIFA’s team ratings. Just how much of a bellwether have those ratings been? FIFA changed the original 1993 system in 1999 (for the worse as we shall see) and again in 2006 (for the better).

Two WCs (1994 and 1998) were played under the original 1993 system, two (2002 and 2006) under the 1999 system while the 2010 WC was played under the 2006 system, as will the upcoming 2014 WC. Are there other rating systems that might better serve FIFA for the men’s WC? As in life, men should look to women for the answer.

How the FIFA World Ranking system works

The three FIFA systems have the same general form. Match points are calculated for each game using five factors, Match points are summed over a period of years.

match points = result x opponent strength x importance x continent x ageing

Table 1 below shows the history of those five factors, along with a system I suggested in a letter to FIFA in 1996, which is quite close to the eventual 2006 FIFA system. FIFA combined my suggestion with others into a serious of options given to their consultants. In 1993, FIFA’s result points were based on 2,1,0, for wins, draws and losses respectively, even though FIFA mandated 3,1,0 points for all domestic and international matches. Those points were assigned to each team in some mysterious way that they would not publish, based on relative opponent strength.

Table 1: History of the five factors for FIFA rating system
Factors FIFA 1993 Stefani 1996 FIFA 1999 FIFA 2006
Result Win = 2,
Tie = 1,
Loss = 0
Win = 3,
Tie = 1,
Loss = 0
Win = 2,
Tie = 1,
Loss = 0
Win = 3,
Tie = 1,
Loss = 0
Opponent Strength Points shared based
on ranking difference
Step-down table 10 x 1993 points 100 x Step down equation
Importance (Friendly-WC) 1 - 1.5 1 - 3 1 - 2 1 - 4
Continent 0.8 - 1 0.8 - 1 0.85 - 1 0.85 - 1
Ageing over 6 years 4 years 8 years 4 years

In 1996, I suggested 3,1,0 points multiplied by a step-down table, so that progressively fewer points are gained when defeating or tying progressively worse-ranked opponents. In 2006, FIFA changed to 3,1,0 multiplied by 100 times a step-down equation, 2 - rank/100, which is more precise than the table I had suggested and which (coincidentally) went through the points in my 1996 step-down table.

In 1996 I suggested making a WC match count at least 3x that of a friendly, while FIFA undervalued a WC match as being only 1.5x as important as a friendly. They now count a WC match 4x as much as a friendly. The continental factors have been adjusted only slightly.

In 1993, FIFA used a six year window. I suggested using a more logical four-year WC cycle in 1996. FIFA moved to an unusually large eight-year window in 1999, which appears to have resulted in worse predictive accuracy, and then to a four year window.

Table 2 below shows the predictive success of the three FIFA systems over the last five WCs, based on pre-WC ratings. Ties were not counted in the group phase to facilitate comparison with the knockout phase and with other systems. The 1993 system was 69.2% successful in predicting winners in the 1994 and 1998 WCs. The 1999 system only predicted 64.1% of the 2002 and 2006 matches correctly, with an abysmal 37.5% accuracy in the 2006 knockout phase. On the other hand, the 2006 system resulted in 72.5% success in 2010 with an excellent 87.5% success in the knockout phase. The 2006 system is headed in the right direction, but might other systems be a better choice for FIFA?

Table 2: Predictive success of FIFA’s higher ranked teams
  Group Phase Knockout Phase All
WC Games Won Lost Tied % Games Won Lost % %
1994 36 19  8  9 70.3 16 12  4 75.0 72.0
1998 48 23  9 16 71.9 16  9  7 56.2 66.7
2002 48 22 12 14 64.7 16 12  4 75.0 68.0
2006 48 26 11 11 70.3 16  6 10 37.5 60.4
2010 48 23 12 13 65.7 16 14  2 87.5 72.5

Some better rating systems

World-wide attention focuses on the rugby and cricket WCs too; but, the rating systems are more predictive. The International Rugby Board, IRB, and International Cricket Council, ICC, use the same basic scheme for adjusting ratings after each match.

New rating = old rating + K [result – prediction]

The IRB and ICC both score a result 1, 0.5, 0 for a win, tie and loss respectively. The prediction (the probability of winning) is also limited to a 1-0 basis depending on rating difference, using a simple equation (0.5 + 0.5 diff /10 for rugby and 0.5 + 0.5 diff /50 for the David Kendix system used in cricket). K depends on importance and score difference in rugby and on importance and games played in cricket. The rugby system includes an adjustment for home advantage. The higher-rated team won an average of 83% of all matches in the 2007 and 2011 rugby WCs, with 90% success in 2011. David Kendix’ system was 79% predictive in the 2007 and 2011 One Day International WCs and 76% predictive in the 2014 T20 WC. Both systems outperformed FIFA’s men’s system.

FIFA need only look to their women’s Elo system for improvement. The form is the same as for rugby and cricket. FIFA cleverly converts game score to a 1-0 scale, instead of making every win the same and every loss the same. The prediction (again the probability of winning) is calculated from rating difference using the well-known Elo equation from chess ratings. The Elo system, which adjusts for home advantage, was 82% successful for the women’s WCs of 2007 and 2011. The rugby, cricket and women’s FIFA systems more precisely adjust after each win, tie or loss and each is more strongly weighted toward pre-WC strength. FIFA would do well to use the women’s Elo system for men’s competition as well.

The four top-ranked teams

Table 3 contains the success of the four top-ranked teams in the last five men’s football WCs using pre-WC rankings. Also shown are the top four as of 8 May, 2014. One more set of ratings will be published prior to the 2014 WC. Of the 20 teams, 75% reached the knockout phase, meaning that 25% of those supposedly dominant teams were eliminated in the group phase. 55% reached the quarter finals and 30% (6) reached the semi-finals. Three of those six won the WC and three were second. No top rated team won, presenting a challenge for Spain in 2014. The second-rated teams won twice, good news for Germany. The fourth-rated teams were second twice, with host nation Brazil currently being fourth. Brazil is particularly well placed to win, given a host-nation advantage. 18th ranked France won as the host nation in 1998. The average winner was ranked 8th, four positions worse than Brazil.

Table 3: Success of the four top-ranked teams
World Cup Host FIFA ranked 1 Finish FIFA ranked 2 Finish FIFA ranked 3 Finish FIFA
ranked 4
Finish
1994 USA Germany QF Neth. QF Brazil 1 Italy 2
1998 France Brazil 2 Germany QF Spain Group England Round of 16
2002 SK-Japan France Group Brazil 1 Argentina Group Portugal Group
2006 Germany Brazil QF Czech Group Neth. Round of 16 Mexico Round of 16
2010 South Africa Brazil QF Spain 1 Portugal Round of 16 Neth. 2
2014 Brazil Spain TBC Germany TBC Portugal TBC Brazil TBC

Please excuse me as I return to gnashing my teeth. Landon Donavan was not chosen for the US team. Silly me: I thought that loyalty, dedication and experience were reasons to be selected. Let the serious gnashing now begin.

We use cookies to understand how you use our site and to improve your experience. By continuing to use our site, you accept our use of cookies and Terms of Use.