Terminal Models

Fairness Audit

Model AModel B

Overlay Model B

Within-league signed gap · Model A

+0.093

95% CI +0.063 to +0.124 · about +10%

Model A+0.093

Model B+0.164

A − B−0.071

Model A − Model B: −0.071
Accuracy gap (MAE): +0.011
Held-out players: 5,820

06 / 07 · Robustness

Does it survive attempts to make it go away?

The gap under every checkwithin-league signed gap, log scale

Primary estimateplayer-level bootstrap+0.093
Club-clustered intervalresampling whole clubs+0.093
Within predicted-value decilessame predicted value+0.095
Within league and decile+0.101
900-minute thresholdsensitivity population+0.084
Clubs the model never sawclub-grouped CV, point estimate+0.112
Ridge regressionpoint estimate+0.132
Random forestpoint estimate+0.134
Gradient boosting, defaultpoint estimate+0.106
Gradient boosting, tuned (primary)point estimate+0.093

Points without whiskers are point estimates

Show as a table

Robustness checks
Row	Estimate	95% CI low	95% CI high
Primary estimate (player-level bootstrap)	0.0933	0.0629	0.1240
Club-clustered interval (resampling whole clubs)	0.0933	0.0612	0.1254
Within predicted-value deciles (same predicted value)	0.0951	0.0657	0.1247
Within league and decile	0.1008	0.0704	0.1309
900-minute threshold (sensitivity population)	0.0837	0.0524	0.1142
Clubs the model never saw (club-grouped CV, point estimate)	0.1122	n/a	n/a
Ridge regression (point estimate)	0.1320	n/a	n/a
Random forest (point estimate)	0.1340	n/a	n/a
Gradient boosting, default (point estimate)	0.1059	n/a	n/a
Gradient boosting, tuned (primary) (point estimate)	0.0933	n/a	n/a

Where the largest errors fallModel A, share of non-UEFA players

All players25.7%
Top 5% under-predicted35.7%
Top 5% over-predicted21.0%

Residual correlation within clubs: 0.08

Players at the same club are not independent, which is why the club-clustered interval is shown alongside the player bootstrap.

The sign of the gap survives resampling whole clubs, comparing players with the same predicted value, a 900-minute threshold, clubs the model never saw, and every algorithm tried.

Row

Estimate

95% CI low

95% CI high

Primary estimate (player-level bootstrap)

0.0933

0.0629

0.1240

Club-clustered interval (resampling whole clubs)

0.0933

0.0612

0.1254

Within predicted-value deciles (same predicted value)

0.0951

0.0657

0.1247

Within league and decile

0.1008

0.0704

0.1309

900-minute threshold (sensitivity population)

0.0837

0.0524

0.1142

Clubs the model never saw (club-grouped CV, point estimate)

0.1122

n/a

Ridge regression (point estimate)

0.1320

n/a

Random forest (point estimate)

0.1340

n/a

Gradient boosting, default (point estimate)

0.1059

n/a

Gradient boosting, tuned (primary) (point estimate)

0.0933

n/a

Fairness Audit

Does it survive attempts to make it go away?

The gap under every checkwithin-league signed gap, log scale

Where the largest errors fallModel A, share of non-UEFA players

How to read the observatory

Fairness Audit

Does it survive attempts to make it go away?

The gap under every checkwithin-league signed gap, log scale

Where the largest errors fallModel A, share of non-UEFA players

How to read the observatory