Current replication
Spearman rank correlation
Not measuredNo current measurement is available in this registry.
Public Policy · 2016
Americans preferred Syrian refugees who are female, English-speaking, and Christian on the eve of Donald Trump's election
Americans preferred female, English-speaking, Christian Syrian refugees.
Read the paper ↗01 / Human finding
What the study found
Americans preferred female, English-speaking, Christian Syrian refugees.
Answer-key review: AI-reviewed
02 / Experiment design
Who chose, and what varied
- Country
- United States
- Human respondents
- 1800
- Attributes
- 5
- Task
- discrete choice
03 / Synthetic replication
Registry replication results
Paper-era reference
2023 CLM mean Spearman ρ
0.91Historical results use their original estimator and reference. They are not a matched comparison with the current run.
Detailed replication evidence · 2026-10-02
Right direction on 6 of 7 significant effects, partial ranking (ρ = 0.65); sizes not comparable (the paper uses other units).
Use it to screen which attributes move choice; sizes need the paper's respondent data, and confirm the reversed effect with people.
- direction
- mixed
- rank
- mixed
- size
- not comparable
Supported uses
- Nothing yet without human confirmation.
Limits to keep in view
- Ranking close levels against each other; the order diverges from humans.
- Decisions that hinge on the attributes whose direction flipped (listed below).
- Effect sizes in the paper's units; only direction, rank and shape are compared.
Effects on a common scale
Human effects: other. Synthetic effects are multiplied by the record’s calibration factor, k = 1.1703. Calibration aligns magnitude; it does not establish agreement in unadjusted effect sizes.
| Attribute / level | Human | Human 95% half-width | Calibrated synthetic |
|---|---|---|---|
| Religion: Religion is Christian (reference) | 0.000 | Not measured | 0.000 |
| Religion: Religion is Muslim | -0.486 | 0.090 | -0.223 |
| Previous occupation: Previous occupation is Farmer (reference) | 0.000 | Not measured | 0.000 |
| Previous occupation: Previous occupation is Teacher | 0.108 | 0.090 | 0.120 |
| Previous occupation: Previous occupation is Doctor | 0.255 | 0.090 | 0.293 |
| English fluency: This applicant speaks fluent English (reference) | 0.000 | Not measured | 0.000 |
| English fluency: This applicant speaks broken English | -0.241 | Not measured | -0.334 |
| English fluency: This applicant speaks no English and needed an interpreter | -0.425 | Not measured | -0.553 |
| Age: Age is 20 Years old (reference) | 0.000 | Not measured | 0.000 |
| Age: Age is 40 Years old | 0.098 | 0.090 | -0.071 |
| Age: Age is 60 Years old | 0.049 | 0.090 | -0.233 |
| Sex: Sex is female (reference) | 0.000 | Not measured | 0.000 |
| Sex: Sex is male | -0.188 | 0.090 | -0.023 |
Respondent model comparison
Each row retains its recorded answer-key version and matched comparison. Scaled RMSE is shown with its corresponding scale factor.
| Model | Answer key | Spearman ρ | Scaled RMSE | Scale k |
|---|---|---|---|---|
| ensemble:bedrock-human-v5 | 1 | 0.60 | 0.201 | 0.7639 |
| google.gemma-3-27b-it | 1 | 0.56 | 0.192 | 0.8175 |
| zai.glm-5 | 1 | 0.52 | 0.220 | 0.6278 |
| databricks-claude-sonnet-4 | 1 | 0.45 | 0.217 | 0.6398 |
| mistral.mistral-large-3-675b-instruct | 1 | 0.45 | 0.219 | 0.6131 |
04 / Evidence and limitations
Follow the evidence
Missing results mean unmeasured, not zero agreement.