Current replication
Spearman rank correlation
0.48Measured 2026-09-27
View this run ↗Public Policy · 2018
British Nationals' Preferences Over Who Gets to Be a Citizen According to a Choice-BasedConjoint Experiment
Britons penalized Muslim applicants and those without a job most.
Read the paper ↗01 / Human finding
What the study found
Britons penalized Muslim applicants and those without a job most.
Answer-key review: AI-reviewed
02 / Experiment design
Who chose, and what varied
- Country
- United Kingdom
- Human respondents
- 1597
- Attributes
- 8
- Task
- discrete choice
03 / Synthetic replication
Registry replication results
Paper-era reference
2023 CLM mean Spearman ρ
0.44Historical results use their original estimator and reference. They are not a matched comparison with the current run.
Detailed replication evidence · 2026-10-02
Right direction on 12 of 14 significant effects, weak ranking (ρ = 0.48); effect sizes not yet settled (1.19× human).
Use it to screen which attributes move choice; calibrate sizes against a small human study before forecasting, and confirm the 2 reversed effects with people.
- direction
- mixed
- rank
- fails
- size
- inconclusive
Supported uses
- Nothing yet without human confirmation.
Limits to keep in view
- Ranking close levels against each other; the order diverges from humans.
- Decisions that hinge on the attributes whose direction flipped (listed below).
- Forecasting the size of a shift without a human calibration anchor.
Effects on a common scale
Human effects: change in choice probability. Synthetic effects are multiplied by the record’s calibration factor, k = 0.8404. Calibration aligns magnitude; it does not establish agreement in unadjusted effect sizes.
| Attribute / level | Human | Human 95% half-width | Calibrated synthetic |
|---|---|---|---|
| Gender: The applicant is male (reference) | 0.000 | Not measured | 0.000 |
| Gender: The applicant is female | -0.006 | 0.010 | 0.032 |
| Length of residence: The applicant has lived in the UK for 4 years (reference) | 0.000 | Not measured | 0.000 |
| Length of residence: The applicant has lived in the UK for 6 years | 0.022 | 0.025 | 0.101 |
| Length of residence: The applicant has lived in the UK for 10 years | 0.083 | 0.025 | 0.089 |
| Length of residence: The applicant has lived in the UK for 20 years | 0.113 | 0.025 | 0.213 |
| Occupation: The applicant works as a corporate manager (reference) | 0.000 | Not measured | 0.000 |
| Occupation: The applicant works as a doctor | 0.043 | 0.030 | 0.068 |
| Occupation: The applicant works as an IT professional | 0.019 | 0.030 | -0.023 |
| Occupation: The applicant works as a language teacher | -0.012 | 0.030 | -0.002 |
| Occupation: The applicant works as an administrative worker | -0.052 | 0.030 | -0.005 |
| Occupation: The applicant works as a farmer | -0.070 | 0.030 | -0.044 |
| Occupation: The applicant works as a cleaner | -0.094 | 0.030 | 0.037 |
| Occupation: The applicant is unemployed | -0.363 | 0.030 | -0.088 |
| Occupation: The applicant is a stay-at-home parent | -0.174 | 0.030 | -0.041 |
| Ancestry: The applicant has a parent who was born in the UK | 0.094 | 0.020 | 0.124 |
| Ancestry: The applicant has a grandparent who was born in the UK | 0.060 | 0.020 | 0.082 |
| Ancestry: The applicant has no British relatives (reference) | 0.000 | Not measured | 0.000 |
| Country: The applicant was originally from Germany (reference) | 0.000 | Not measured | 0.000 |
| Country: The applicant was originally from Italy | 0.009 | 0.100 | -0.020 |
| Country: The applicant was originally from India | 0.081 | 0.100 | -0.029 |
| Country: The applicant was originally from Poland | 0.052 | 0.070 | -0.010 |
| Country: The applicant was originally from Pakistan | 0.015 | 0.100 | -0.038 |
| Country: The applicant was originally from Syria | 0.005 | 0.100 | 0.026 |
| Country: The applicant was originally from Somalia | 0.015 | 0.120 | -0.024 |
| Country: The applicant was originally from Nigeria | -0.012 | 0.120 | -0.031 |
| Country: The applicant was originally from Ireland | 0.080 | 0.070 | 0.002 |
| Country: The applicant was originally from Australia | 0.068 | 0.070 | -0.046 |
| English Proficiency: The applicant only speaks very basic English (reference) | 0.000 | Not measured | 0.000 |
| English Proficiency: The applicant can speak English well | 0.051 | 0.100 | 0.267 |
| English Proficiency: The applicant is perfectly fluent in English | 0.108 | 0.080 | 0.375 |
| Refugee Status: The applicant is not a refugee (reference) | 0.000 | Not measured | 0.000 |
| Refugee Status: The applicant is a refugee | 0.043 | 0.070 | -0.031 |
| Religion: The applicant is a practicing Christian (reference) | 0.000 | Not measured | 0.000 |
| Religion: The applicant is a practicing Muslim | -0.161 | 0.120 | -0.068 |
| Religion: The applicant is not religious | -0.015 | 0.040 | -0.005 |
04 / Evidence and limitations
Follow the evidence
Missing results mean unmeasured, not zero agreement.