Contact
Bias does not appear at a single point, it can enter through multiple layers of the AI pipeline, from the data used to train models to the design of the interview itself.
Yes, every AI assessment model learns from historical data, and if that data reflects past hiring decisions made by humans who favored certain demographics, educational backgrounds, or communication styles, the model will risk replicating those patterns. Candidates who don't match a historical profile that was itself the product of bias may be systematically disadvantaged before a single interview question is asked.
Signal to look for: Ask vendors how their training data was sourced and reviewed. Were answers assessed without knowledge of the demographics of the candidate? Were calibration sessions conducted to remove unreliable evaluations? If a vendor cannot answer these questions in detail, bias mitigation was not a design priority.
Inconsistency is, by definition, unfair, and research published in 2025 demonstrates that LLMs used for candidate assessment produce meaningfully different rankings across demographic groups for otherwise identical candidates (Seshadri et al., 2025), and diverge significantly from human expert judgment depending on contextual conditions (Varshney & Ganuthula, 2025) If two candidates give the same response and receive different scores depending on when or how the model was queried, the process is arbitrary, and arbitrary hiring decisions are unfair hiring practices. Also, inconsistency provides a missed opportunity for the business as a good candidate may have been unfairly scored by the LLM, and not passed on to the shortlist or subsequent steps in the recruitment process.
Signal to look for: Ask vendors whether their scoring models are deterministic, meaning the same input always produces the same output. If the vendor's response involves language about "probabilistic outputs" or "model variation," that inconsistency risk has not been addressed.
The problem is that a post-hoc explanation cannot be used to defend a hiring decision in an audit, and in LLM-based systems, the explanation shown to recruiters is often generated after the score has been assigned: a plausible-sounding rationale rather than the actual scoring logic. This means bias embedded in the scoring process can be obscured by a surface-level narrative that appears reasonable.
Signal to look for: Ask vendors whether the explanation shown to recruiters is mechanically tied to the scoring process, or generated separately after the fact. Vendors should be able to describe this distinction clearly.
Yes, if different candidates for the same role are asked materially different questions, the comparison is not valid regardless of how the scoring model performs. Structural unfairness in interview design will produce biased shortlists even from a well-calibrated algorithm. Also, the more dynamic the interview template is, the more the interview will become an unstructured interview. And research has clearly demonstrated how bad unstructured interviews are at predicting future performance at work (e.g. Sackett et al., 2022; Schmidt & Hunter, 1998). So, the more dynamic and variable interview design may risk creating both bias and lower quality.
Signal to look for: Verify that every candidate receives the same structured interview: same questions, same competency framework, same scoring criteria. Also confirm that interview content has been reviewed for questions that may systematically disadvantage candidates based on cultural background or communication style.
The following questions should be part of any vendor evaluation.
On training methodology: Were models trained against expert human evaluations, or optimized against historical hiring outcomes? What behavioral frameworks, for example, Behaviorally-Anchored Rating Scales, structured the evaluation process?
On scoring consistency: Are assessment models deterministic? Are models versioned and locked for the duration of a hiring cycle so every candidate is assessed under identical conditions?
On explainability: Is the explanation shown to recruiters faithful to the scoring logic, or generated separately or post-hoc? Can the vendor trace a specific score back to specific scoring criteria without narrative interpretation added after the fact?
On bias testing: Has the model been tested for adverse impact across protected groups? How frequently are bias audits conducted post-deployment? What precautions have been taken to mitigate model drift?
On human oversight: Does the system make automated rejection decisions, or does the final decision remain with the recruiter? Under the EU AI Act, AI in recruitment is classified as high-risk. High-risk systems require effective human oversight.
Fair AI interview screening is automation designed around structured interviewing science, with bias mitigation built into every layer: training data, model architecture, scoring consistency, explainability, and post-deployment monitoring. The practical markers are consistent across vendors who take this seriously:
At Hubert, these principles are the design architecture. Hubert's assessment models are deterministic and proprietary, uses Behaviorally-Anchored Rating Scales and have been validated against 1,000,000+ expert human evaluations. The explanation a recruiter sees is the exact same logic that generated the score, not a narrative produced after the fact. The result is a shortlist recruiters can stand behind: faster, fairer, and legally defensible by design.
Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262.