The AI Interview Summary That Invented a Candidate Weakness

A sentence that was never actually said

An interviewer asks whether the candidate has led a migration. The candidate says they supported the rollout but did not own the architecture. An automatic summary later says: 'Candidate lacks architecture ownership and struggled with ambiguity.' The first clause compresses a factual boundary; the second may be an invented interpretation. If the panel reads only the summary, a model-generated inference becomes a scored weakness. This is a plausible synthetic scenario, not a documented incident.

Separate recording, transcription, summarization and evaluation

These are four distinct operations. Audio is the original observation where recording is lawful and authorized. A transcript is an imperfect representation of speech. A summary selects and compresses content. An evaluation judges job-relevant competencies. Every transition can lose context. Do not allow the output of one stage to impersonate the evidence from an earlier stage. Store provenance: source type, timestamp, speaker, reviewer, and any correction.

The four error classes worth auditing

Check (1) speaker swaps, where the interviewer's hypothetical example becomes the candidate's claim; (2) negation loss, where 'I did not own it' becomes 'I owned it'; (3) unsupported trait inference, such as 'not collaborative'; and (4) omitted caveats, where a limitation disappears. A fluent summary can be wrong without containing obvious nonsense. Review high-impact claims against a permitted primary record before scoring.

A source-linked evidence standard

Every scored claim should point to a job competency and a traceable interview observation, with the candidate's actual example and context. When recordings are not retained or recording is inappropriate, the interviewer should confirm contemporaneous notes and mark uncertainty. An AI tool may help organize evidence, but it should not manufacture an assessment scale or fill in missing candidate behavior. Use the same rubric across candidates, consistent with structured-interview guidance.

The privacy problem is not solved by turning off training

Before sending interview audio or notes to a vendor, confirm the lawful basis, transparency, access controls, retention and processor terms applicable to your jurisdiction. Recording a person, using a transcript and scoring them are not automatically the same authorized purpose. Restrict sensitive and irrelevant information. Do not upload raw candidate interviews to a personal chatbot just because a browser extension makes it easy.

A blind 12-claim spot check

Take a permitted sample of twelve material summary claims across interviews. Have two trained reviewers classify each as directly supported, supported with caveat, unsupported or not verifiable from retained evidence. Reconcile disagreements by checking the primary record where lawful. Report unsupported and unverifiable separately. One bad claim is a prompt to investigate the workflow, not proof that all automated notes are unusable.

The counterintuitive rule: correct the note, not the person

When a summary is wrong, preserve a minimal audit trail of the correction and ensure downstream scorecards no longer rely on the inaccurate claim. Do not create a permanent 'candidate was misrepresented' flag in the profile. Track the error type, system version, affected process stage and whether a decision was revisited. If an outcome was affected, escalate through the organization's established review and candidate-rights process.

A practical release gate for AI notes

Do not release an auto-summary into the decision packet until a responsible reviewer confirms speaker attribution, negations, competency links and high-impact statements. The reviewer must be able to mark 'unverified' rather than forced yes/no. The check below illustrates how to quantify review coverage, not how to decide whether a person should be hired. A perfect score on twelve claims does not establish a universal model accuracy rate.

Run the operational check

This calculator uses only synthetic or aggregate values and runs entirely in your browser. It cannot score an individual candidate.

Implementation checklist and guardrails

Assign a named owner, agree the data definitions, document exclusions and check a small representative sample before changing the workflow. Review outcomes by job family and process stage. Protect candidate privacy, provide accessible alternatives and do not automate adverse decisions from these calculators. Recheck the process after two cycles and retain the results that contradict the initial hypothesis.

Research basis and further reading

Twenty topic-specific and adjacent methodological references. Some describe regulations or assessment standards, others are vendor studies or operational definitions. They do not all validate the proposed framework; source scope and jurisdiction must be checked before applying it.

1. NIST — AI Risk Management Framework
2. NIST — Generative AI risk profile and confabulation
3. NIST — AI Risk Management Resource Center
4. NIST — AI RMF resources
5. OPM — Structured interviews
6. OPM — Structured interview guide
7. OPM — Assessment strategy
8. OPM — Hiring process analysis
9. OPM — Interview scoring principles
10. OPM — Competency selection
11. EEOC — ADA job applicants
12. W3C — WCAG 2.2
13. W3C — Preliminary accessibility checks
14. EDPB — Data protection principles
15. ICO — Recruitment and selection data
16. Greenhouse — 2024 state of job hunting
17. Greenhouse — 2025 AI in hiring report
18. EEOC — Employment tests and selection procedures
19. CIPD — Recruitment overview
20. CIPD — Resourcing and talent planning

The decision worth changing this week

Pick one measurable, reversible change, state what success and harm would look like, and publish the result with the denominator. Recruiting quality improves when evidence survives the handoff from recruiter to hiring manager.