Direct Answer
A structured reference check uses the same standardized, job-related questions and rating scales for every candidate. An unstructured reference check — the traditional phone call or open-ended letter of recommendation — lets the reference say whatever comes to mind, with no consistent format across candidates.
This distinction matters far more than most people realize. It is the single biggest factor in whether a reference check produces useful information or not.
Why It Matters
Imagine two hiring managers checking references for the same role. One calls a previous employer and asks, “So, what was she like to work with?” The other sends every reference the same set of specific, behavior-focused questions with a consistent rating scale.
The first conversation might be friendly. It might even feel informative. But the information it produces is difficult to compare across candidates, prone to personal bias, and shaped by whatever the reference happens to think of in the moment.
The second approach generates data that can be meaningfully compared, statistically analyzed, and used to make fairer decisions.
The Science Behind It
The research on this is clear. Early studies on reference checks reported low predictive validity — a correlation of just ρ = .26 between reference information and job performance (Hunter & Hunter, 1984). For decades, this gave reference checking a reputation as little more than a formality.
But those early studies measured unstructured methods. When researchers examined structured reference checks — with standardized questions, behavioral anchors, and consistent rating scales — the picture changed dramatically. Hedricks et al. (2013) found that structured, web-based multisource reference checks achieved internal consistency reliability of α = .96–.98 and criterion-related validity of r = .35 with supervisory performance ratings (p < .001, N = 223). That level of validity rivals personality tests, assessment centers, and biographical data as predictors of job performance (Hedricks et al., 2019).
Structure also reduces bias. In a sample of nearly one million job applicants and over four million reference providers, Fisher et al. (2022) found no meaningful gender bias in structured, quantitative employment references — including in stereotypically masculine and feminine occupations.
Common Misconceptions
People often assume that a casual phone call gives a more “honest” picture than a formal questionnaire. The evidence suggests the opposite. Unstructured letters of recommendation suffer from well-documented inflation: in one study, only 0.23% of 6,854 reference ratings fell below average (Aamodt & Williams, 2005, as cited in Nicklin & Roch, 2009). When every candidate sounds exceptional, references cannot distinguish between them. Structure counteracts this ceiling effect by asking specific, behavioral questions that produce meaningful variation in responses.
How This Connects to Better Hiring
Understanding this distinction is the foundation for improving how you gather and use reference information. The takeaway is simple: how you ask matters more than whom you ask. Structured formats produce more reliable, more valid, and fairer information — regardless of who is providing the reference.