Direct Answer
Criterion-related validity is the degree to which scores on a selection tool — a test, an interview, a reference check — actually predict a meaningful outcome, such as job performance. It is measured by calculating the correlation between the tool’s scores and the outcome it is supposed to predict. The higher the correlation, the more useful the tool is for making hiring decisions.
Why It Matters
Every hiring tool makes an implicit promise: “This information will help you identify who will perform well.” Criterion-related validity is how you test that promise. Without it, you are making decisions based on information that may feel relevant but has no demonstrated connection to the outcome you care about.
This is not an abstract concern. Organizations invest significant time and money in selection processes. A tool with low criterion-related validity is not just unhelpful — it can actively harm decision quality by introducing noise or bias into the process while creating the illusion of rigor.
The Science Behind It
The foundational work on criterion-related validity in personnel selection comes from Schmidt and Hunter (1998), who synthesized 85 years of research to rank selection methods by their ability to predict job performance. Their meta-analysis established a hierarchy: cognitive ability tests, structured interviews, and work sample tests ranked among the strongest predictors, while unstructured interviews and years of experience ranked much lower.
More recently, Sackett et al. (2022) revisited these estimates and found that systematic overcorrection for range restriction had inflated some validity coefficients. Their reanalysis placed structured interviews as the top-ranked selection procedure and revised the cognitive ability estimate downward from ρ = .51 to ρ = .31. This sparked active debate — Bobko et al. (2024) argued that the revised methodology introduced its own confounds — but the broader conclusion remains: the predictive power of any selection method is an empirical question, and structured approaches consistently outperform unstructured ones.
For structured employment references specifically, Hedricks et al. (2013) reported criterion-related validity of r = .35 with supervisory performance ratings (p < .001, N = 223). This places structured references in the same range as many well-established selection methods, including personality assessments and biographical data inventories (Van Iddekinge et al., 2023).
Common Misconceptions
A common misunderstanding is that face validity — whether a tool looks like it measures something useful — is the same as criterion-related validity. A question that seems obviously job-related may have no predictive power, while a question that seems unrelated may predict performance quite well. The only way to know is to collect data and compute the correlation. Intuition about what “should” predict performance is often wrong.
How This Connects to Better Hiring
Criterion-related validity is the standard by which all selection tools should be judged. When evaluating any hiring practice — from cognitive tests to reference checks to AI-scored video interviews — the first question should always be: “What is the evidence that this predicts the outcomes we care about?” Tools that can answer that question with data deserve a place in your process. Tools that cannot should be scrutinized carefully.