Stories · Research · Methods, the technical account
How FairValue works
How the data was chosen, how the models were trained and judged, and where the results stop applying. Figures come from the generated research outputs; method details follow the repository's research design and reports.
Abstract
FairValue estimates the Transfermarkt market value of 5,820 players at clubs in 14 European first divisions from two seasons of club match data. Two tuned gradient-boosting models are compared: Model A sees league and club context, Model B does not. Neither sees nationality. Every metric uses out-of-fold predictions. Comparing signed residuals within leagues, non-UEFA players are under-predicted relative to UEFA players by +0.093 log units under Model A (95% CI +0.063 to +0.124). The finding is observational, specific to this selected population, and not evidence of discrimination.
01Data and provenance
Valuations and match records come from Transfermarkt, compiled by the open transfermarkt-datasets project and downloaded from its public bucket, with each file checksummed. The snapshot is frozen: upstream updates paused in 2026, with valuations ending on 12 June 2026.
Citizenship is used only to label players for the audit. It is stored in a separate table that the training code never reads, and a test fails if any demographic column reaches a feature matrix.
02Population
From every player in the dataset to the modelling population. Each bar splits UEFA and non-UEFA players.
- Every player in the dataset50,14935.4% non-UEFA
- With a valuation in the last 12 months19,36446.3% non-UEFA
- Valued while at a club in a covered league7,37825% non-UEFA
- With at least 450 club minutes in two years5,82025.7% non-UEFA
UEFA Non-UEFA
The final population is 5,820 players valued between June 2025 and June 2026 while at a club in one of 14 European first divisions, with at least 450 club minutes in the previous two years. It is a selected population: results describe these players, not the global transfer market. A 900-minute threshold is checked as a sensitivity analysis.
03Features
All computed as of each player's own valuation date, from matches up to that date only.
| Input | Definition | Model A | Model B |
|---|---|---|---|
| Age | Years at the valuation date; the linear baseline also gets age² | Yes | Yes |
| Height | As recorded; missing values imputed | Yes | Yes |
| Position | Position, role (sub-position) and preferred foot, one-hot encoded | Yes | Yes |
| Appearances | Club matches in the 730 days up to the valuation date | Yes | Yes |
| Minutes (2 years) | Club minutes in the same window, and minutes per appearance | Yes | Yes |
| Share of games started | Share of appearances started rather than entered as a substitute | Yes | Yes |
| Goals per 90 | Goals, assists, their sum, yellow and red cards per 90 minutes | Yes | Yes |
| European-cup minute share | Share of minutes in European club competitions | Yes | Yes |
| Minutes (last 365 days) | Club minutes in the last 365 days: recent form or injury signal | Yes | Yes |
| Days since last match | Days from the last club match to the valuation date | Yes | Yes |
| Minutes at current club | Share of window minutes played for the club at valuation | Yes | Yes |
| Club points per game | Valuation club's league points per game over 365 days | Yes | No |
| League | League of the club at valuation time | Yes | No |
Never features: citizenship, confederation, audit group, birthplace, national-team identifiers, caps and international goals, the player's highest past value, and earlier valuations.
04Training and validation
The target is the natural log of market value. Models are compared with five-fold cross-validation, stratified on value tier. Hyperparameters are tuned with Optuna inside each outer fold (inner three-fold CV, 25 trials), so no score is computed on data used for tuning.
Every reported metric, and every number in the fairness audit, uses out-of-fold predictions: each player is scored by a model that never saw them. Audit group is never used for splitting, training or tuning. The served models are refits on the full population, with versions recorded in a registry.
05Model selection
Rule fixed in advance: lowest mean 5-fold CV log RMSE under Model A (D-019).
- Gradient boosting, Optuna0.525
- Gradient boosting0.539
- Ridge (baseline)0.604
- Random forest0.646
06Evaluation
| Model | Algorithm | Log RMSE | Log MAE | Log R² | Median abs. error |
|---|---|---|---|---|---|
| Model A | Ridge (baseline) | 0.604 | 0.471 | 0.856 | €683.51k |
| Model A | Random forest | 0.646 | 0.502 | 0.836 | €717.59k |
| Model A | Gradient boosting | 0.539 | 0.416 | 0.885 | €624.15k |
| Model A | Gradient boosting, Optuna | 0.525 | 0.406 | 0.891 | €617.99k |
| Model B | Ridge (baseline) | 1.026 | 0.830 | 0.585 | €1.26M |
| Model B | Random forest | 0.986 | 0.791 | 0.617 | €1.13M |
| Model B | Gradient boosting | 0.922 | 0.728 | 0.665 | €1.08M |
| Model B | Gradient boosting, Optuna | 0.917 | 0.725 | 0.669 | €1.06M |
A log MAE of 0.406 means the typical estimate is off by a factor of about 1.5. Errors shrink as minutes grow, are largest for goalkeepers and in Scotland, and pull estimates towards the middle: cheap players are over-predicted and expensive ones under-predicted.
| Model | Accuracy | Balanced accuracy | Macro F1 | Within one tier | ROC AUC |
|---|---|---|---|---|---|
| Model A | 79.3% | 72.1% | 0.737 | 99.9% | 0.946 |
| Model B | 66.5% | 56.8% | 0.590 | 96.9% | 0.862 |
07Explainability
SHAP values are computed with TreeExplainer on the served gradient-boosting regressors, on the log scale, with one-hot columns summed back to their input. They add up exactly to each prediction. The studio computes them live for any profile; the bars below are averages over the training population and say nothing about a single player.
Model A
- League0.67
- Age0.47
- Club points per game0.32
- Appearances0.25
- European-cup minute share0.18
- Minutes (last 365 days)0.18
- Days since last match0.10
- Goal contributions per 900.10
Model B
- Age0.47
- Appearances0.38
- European-cup minute share0.32
- Days since last match0.26
- Minutes (2 years)0.16
- Goal contributions per 900.14
- Minutes (last 365 days)0.12
- Height0.05
Counterfactuals (DiCE) asked how much match output would need to change to move a player up one tier, holding age, position, league and club fixed. Across 720 queries, 469 found a valid counterfactual. No consistent difference between UEFA and non-UEFA players emerged. These are counterfactuals on match statistics, never on nationality, which the model cannot see.
08Fairness definitions
The primary measure is the signed residual gap: mean (actual − predicted) log value for non-UEFA players minus that for UEFA players, computed within leagues. Accuracy gaps (MAE, RMSE), tier error-rate gaps (ER1a), per-tier true-positive-rate gaps (equalised-odds components, ER1b) and selection rates are reported alongside. All use held-out predictions with 2,000-replicate bootstrap intervals.
None of these is treated as a pass/fail test. The full results are in the fairness audit.
09Monitoring
With no live traffic and a frozen source, drift is simulated: the previous year's snapshot ((2024-06-12, 2025-06-12]) is compared with the training year ((2025-06-12, 2026-06-12]). The retraining policy returned NO ACTION.
| Feature | PSI | Median, before → after |
|---|---|---|
| Age | 0.002 | 26.35 → 26.39 |
| Minutes (2 years) | 0.002 | 2645.00 → 2592.00 |
| Goal contributions per 90 | 0.001 | 0.16 → 0.16 |
| Club points per game | 0.041 | 1.33 → 1.26 |
| ln(value) | 0.007 | 14.51 → 14.51 |
| Model | CV log RMSE | Next year | Change |
|---|---|---|---|
| Model A | 0.538 | 0.570 | +6.0% |
| Model B | 0.939 | 1.059 | +12.9% |
10Limitations
- Coverage
- Match data exists only for 14 European leagues; 79% of currently valued non-UEFA players play in leagues without it and are excluded.
- Survivorship
- Non-UEFA players who reach these leagues are positively selected (median value €2.5M vs €2M for UEFA).
- The target
- Transfermarkt values are crowd estimates. The model can reproduce any bias in them; the audit measures model against target only.
- Proxies remain
- Allowed features still predict UEFA membership (AUC 0.67). Nationality-blind is not group-blind.
- Context
- League and club strength are reputation proxies. Model B exists to measure their effect.
- Thin defensive data
- No tackles or passes: defenders and goalkeepers are described mostly by minutes and context.
- Frozen snapshot
- Upstream updates paused in 2026; drift is simulated from history, not observed live.
- Unseen clubs
- Error rises 8.7% for clubs absent from training; headline metrics describe known clubs.
- Small tiers
- Only 94 World Class and 246 Elite non-UEFA players; per-tier fairness metrics are imprecise.
- Intervals
- Bootstrap intervals ignore refit variance and are somewhat too narrow.
Not suitable for scouting, recruitment, contracts or any decision about a person. The full list is in docs/KNOWN_LIMITATIONS.md, with the model card and data card alongside it.
11Reproduce
From the repository root, with Python 3.12 and Node 20 or later.
python -m src.data.acquire # download raw tables, verify checksums
python -m src.data.build_dataset # processed tables
python -m src.models.train # 5-fold CV, MLflow, held-out predictions
python -m src.fairness.audit # fairness audit on held-out predictions
python -m src.models.final # refit the served models
python -m src.product.export # research JSON for this site
uvicorn src.api.app:app --port 8000 # the API
cd web && npm ci && npm run dev # this site