AI in Recruitment and Hiring: A Bias Test Checklist for Small Agencies

A small recruitment agency checks an AI screening tool for bias by running a sample of at least 100 CVs through it before go-live, comparing pass rates across groups, questioning the vendor in writing, and having a person review every rejection it makes. Then it repeats a smaller version of that test monthly and logs the results.

None of this makes a tool "unbiased". It tells you whether the tool behaves badly on your roles and your candidates, which is the only question you can answer for yourself. A vendor's own bias audit was run on its test data and its settings, not on your candidates or your configuration, so the testing has to be yours. It is a practical routine, not legal advice; ask a qualified adviser what your obligations are.

Follow me on Instagram@sagnikteaches

What can go wrong with an AI screening tool

Screening tools rank, score or filter candidates from their CVs and application answers. Bias enters in a few ordinary ways, and knowing them tells you what to test.

Connect on LinkedInSagnik Bhattacharya

• Proxy signals. Home addresses, school names, gaps in employment, or a first-job title can stand in for protected characteristics even when the tool never sees age or gender.
• Training-data echo. A tool trained on past hires reproduces who got hired before, including old patterns you would not choose today.
• Format sensitivity. Two-column CVs, career breaks, non-standard job titles and CVs written in a second language can be scored lower because they parse badly, not because the candidate is weaker.
• Prompt drift. If you configure a general assistant with instructions such as "find culture fit", the model fills the vague phrase with its own assumptions.

The test below is built around those four. If you want the wider picture of how fairness shows up in CV screening, read fairness checks every recruiter should run; this tutorial turns that into a repeatable small-agency routine.

Subscribe on YouTube@sagnikteaches

Step 1: the vendor questionnaire

Send these questions in writing before you pay or switch anything on. A vendor who answers vaguely, or says "our AI is bias-free", has told you something useful. Keep the replies; they go in your log.

The last row matters more than it looks. Without an export of scores and outcomes, you cannot run your own monthly test, and the tool should not be on your shortlist. For related buying questions, see whether to bolt AI screening onto your ATS or switch systems.

Step 2: build a test set you can trust

You need a set of CVs where you already know the right answer, or at least where you can compare groups. There are two ways to get one, and a small agency can do either in an afternoon.

Option A: real CVs, anonymised

Pull 100 to 200 past applications for one role family, such as administrators or warehouse supervisors. Remove names, photos and contact details. Record, in a separate sheet, what happened to each candidate (shortlisted, interviewed, placed). Any group information you hold, such as self-declared gender from an equal-opportunities form, stays in that sheet and is not given to the tool.

Option B: paired CVs

Take 10 to 15 real or realistic CVs, anonymised, and make near-identical copies with one detail changed: a typical name from a different background, a career break of eight months, a different university, a two-column layout. Run both versions. Any difference in score is caused by that detail alone, which makes the signal easy to read even with a small set. Do not use real candidates' data in a way that identifies them, and do not post these test CVs anywhere public.

Use a mix of both where you can. Paired CVs find specific sensitivities; the larger set shows overall pass rates.

Step 3: run the test and compare pass rates

Run the whole set through the tool with the configuration you plan to use. Note the version or date, the role criteria you entered, and the cut-off that counts as a "pass". Then compare pass rates between the groups you can identify.

A common rule of thumb used in fairness audits is the four-fifths rule: if one group's pass rate is below 80% of the highest group's pass rate, treat it as a flag for investigation. It is a screening heuristic, not a legal test, and with small groups it throws up false alarms, so use it only to decide where to look harder.

A worked example with numbers

An illustration. A five-person agency tests a screening tool for an office-administrator role with 120 anonymised past applications. The tool passes any CV scoring 70 or more out of 100. From the CVs themselves the agency can split the set into two groups: candidates with a career break of six months or more (30 CVs) and candidates without (90 CVs). A career break is not a protected characteristic, but it often stands in for caring responsibilities or illness, which is why it is worth testing.

The break group's rate is 30 divided by 60, which is 50% of the other group's rate, well under the 80% line. With 30 CVs the result could partly be chance, but the gap is large enough to investigate rather than shrug at. The agency does three things. It reads 10 of the 21 failed break-group CVs by hand and finds that six of them show relevant experience that a person would have shortlisted. It checks the tool settings and finds an "employment continuity" weighting switched on by default. It switches that off, reruns the test, and the break group's pass rate rises to 47%, a ratio of about 78% against a 60% rate. Better but not clean, so the agency adds a rule that every break-group fail gets a human read, and logs the change.

The point of the example is the sequence: flag, read examples by hand, find a cause, change one thing, rerun. A number alone never tells you why.

Reading the paired-CV results

For paired copies, look at score differences, not just pass or fail. In an illustrative run, ten CV pairs differing only by name show an average score gap of 0.4 points, which is noise. Ten pairs differing by a two-column layout show an average gap of 11 points, because the tool misreads the second column. That is not bias in the legal sense, but it unfairly penalises candidates whose CVs come from certain templates, and it is easy to fix by converting CVs to plain text before scoring.

Step 4: human review of rejections

The single most protective control is a rule that the tool never rejects anyone alone. Its output is a ranking or a note, and a person decides. For a small agency, a workable version is:

1. The tool scores and sorts the batch.
2. A recruiter reads every CV in the top band and a random sample of the bottom band: for example one in five, and at least ten per role.
3. If the recruiter would have shortlisted a bottom-band CV, they record it as an override and why.
4. Any candidate who is rejected receives their outcome from a person, not an automated message that implies a human judged them.

Count overrides. If recruiters overrule the tool on more than one in ten of the sampled bottom-band CVs, it is not ready to be trusted for that role. The override reasons are also the best source of changes to the configuration. A related setup is described in AI CV screening for recruitment agencies: setup and safeguards.

Step 5: the monthly log

Tools change, roles change, and the pool of applicants changes with the job market. A once-only test goes stale. A one-page log, filled in monthly, keeps the evidence current and takes under an hour once the habit is set.

Two columns of this log do most of the work: the override count, which tells you whether the tool matches human judgement, and the change made, which proves you acted on what you saw. If you stop filling in the second column you have a report, not a control. In the illustrative month above, the 13% override rate is over the one-in-ten line from Step 4, which is why the job-title synonym list went in and a rerun was booked before the tool screens that role again.

Scoring the vendor's reply: a filled-in example

A questionnaire only helps if you decide in advance what counts as a pass. Give each answer 0 (no answer or evasive), 1 (partial) or 2 (specific and checkable), and set a threshold before the replies arrive. An illustrative reply from a vendor might score like this, one line per question in the table: inputs named clearly (2); no way to switch off the address field (1); a bias audit mentioned with a date and method but no results shared (1); explanations of each score shown (2); auto-reject can be turned off (2); retention period stated as 12 months (2); no change-notice promise (0); export to a spreadsheet available (2). The total is 12 out of 16.

Set your bar in advance, for example 13 or above to proceed, with a mandatory 2 on the export and auto-reject rows. This vendor falls short on the total and could be asked to fix the address field and give change notices in writing. If they agree, you proceed; if they will not, you have your answer. Deciding the bar first stops a good demo from talking you past a weak reply.

Candidates whose CVs are not in standard English

Agencies placing people into varied workplaces often handle CVs in a second language, or written by candidates whose first language is not English. Screening tools tend to score these lower because keyword matching expects particular phrases. Add a small group to your test set: eight to ten CVs from strong candidates you placed, written in plainer or non-standard English, and see where they land.

In an illustrative run, seven of ten such CVs land in the bottom band even though all ten candidates were placed successfully. The cause is usually keyword matching against the job advert. Two fixes work: ask the tool, or the assistant prompt, to judge described work rather than matching phrases, and make sure the human sample includes bottom-band CVs with this profile. Log the result like any other group.

How long the routine takes

For one role family, an illustrative budget: questionnaire and scoring, 90 minutes; building the test set, three hours the first time; running and comparing results, one hour; reading flagged examples, one hour; writing the first log entry, 30 minutes. That is about seven hours once. The monthly routine is then roughly an hour: export scores, read the sample, rerun the paired set, update the log. For a five-person agency, that is a modest cost against the risk of using a tool you cannot explain to a client or a candidate who asks why they were not shortlisted.

A mistake that shows up in small agencies

A common one is testing the tool on the roles it is good at. An agency tries screening on a high-volume admin role with thirty near-identical CVs, sees a clean result, then switches it on for a senior finance role with fewer, more varied CVs. Different roles produce different failure modes. Run the test per role family, even if the sets are small, and do not carry a pass from one family to another.

The same applies after a configuration change. Changing the criteria from "three years' experience" to "relevant experience" can move pass rates for groups that tend to have shorter or broken careers. Rerun the paired set after any change to criteria, even a wording tweak.

When the tool is a general assistant, not a hiring product

Many small agencies do not buy a screening product. They paste CVs into ChatGPT or Claude with a prompt such as "rank these against the job description". The same test applies, with a few extra cautions. The ranking varies between runs, so run the same CV set twice and compare; if the order moves a lot, the output is not stable enough to rely on. Use a plan with business data terms, and read how to stop AI tools training on your business data before pasting candidate details. Ask for evidence against each requirement, not a single score, so the recruiter can check the reasoning.

Compare the CV below with the job requirements. For each requirement,
write: MET / PARTLY MET / NOT EVIDENCED, plus the exact line from the
CV that supports it. Do not infer age, gender, ethnicity, health,
family status or nationality from any detail. Do not penalise career
breaks, non-standard layouts or job titles that differ from the
requirements; judge the described work. Do not give an overall score.

A realistic output is a short table of requirements with quoted CV lines. If a line it quotes is not in the CV, you have found a fabricated quote; that run goes in the log as a failure. Because the model gives no score, the recruiter forms the judgement, which is what you want. For writing the role description that feeds this prompt, see writing job adverts with ChatGPT without biased language.

Record keeping and what to keep

Keep, for each tool and each role family: the vendor questionnaire and replies, the test set description, the configuration used, the results table, the changes made and the monthly log. Keep them for as long as you keep the candidate records they relate to, and agree the period with your adviser. Store them where candidate data is protected, because the test set may contain personal details.

Tell clients and candidates plainly when AI is part of your process. A short line in your privacy notice and your client terms is enough: which stage uses it, that a person reviews rejections, and how someone can ask for a human re-read. If you have candidates in the EU, AI used for recruitment or selection is classed as high-risk under Annex III of the EU AI Act, which brings duties such as human oversight and record keeping. The date those duties start has already been pushed back once (published timetables now show 2 December 2027 for this category), so check the timetable in force with an adviser rather than relying on a tutorial.

A one-page checklist to print

• Vendor questionnaire sent and answered in writing; replies filed.
• Input fields switched off: name, photo, age, address, graduation dates.
• Auto-reject disabled; a person decides every rejection.
• Test set built per role family: at least 100 CVs or 10 to 15 paired CVs.
• Pass rates compared by group; any ratio under 80% investigated by reading examples.
• Paired CVs run for name, career break, layout and university; gaps over 5 points investigated.
• Reruns after any criteria or version change.
• One in five bottom-band CVs read by a recruiter each month; overrides counted.
• Monthly log completed, with a "change made" entry each time.
• Candidate notice and client terms mention AI use and human review.

For a tool-cost view of the same decision, the free AI tool options for a small agency tutorial covers what a no-cost setup does and does not give you; the bias routine above applies to those as much as to paid products. If you would rather map this with someone, that is what my AI implementation consultation covers; see the contact page.

Questions agencies ask about testing screening tools for bias

How many CVs do I need for a meaningful bias test?

More than you think, and fewer than a statistician would like. For a quick agency test, run at least 100 real or realistic CVs through the tool, with 20 or more in any group you compare. Small groups swing wildly, so treat results as a flag to investigate, not proof either way.

Is a vendor's own bias audit enough?

No. A vendor audit describes their test data and their model version, not your candidates, your roles or your configuration. Ask for it, read it, then run your own sample test on your own data before using the tool on live candidates.

Do I need to store the test results?

Yes. Keep the test set description, the settings, the results and what you changed as a dated record. If a client or a candidate later asks how you checked the tool, that record is your answer. Store it where your normal compliance records live.

Does the EU AI Act affect a small agency?

If you place candidates with employers or screen applicants in the EU, AI used for recruitment or selection is listed as high-risk in the Act's Annex III, which brings obligations around oversight and records. The date those obligations start has already been pushed back once (published timetables now show 2 December 2027 for this category), so check the timetable in force with an adviser.

Further reads

• AI Bias in Small Business Decisions: Hiring, Pricing and Credit — Bias in hiring, pricing and credit decisions.
• How Recruiters Use AI to Write Candidate Summaries Clients Read — Candidate summaries that clients will actually read.
• Can AI Screen CVs Fairly? What Small Employers Need to Know — What small employers need to know about fair screening.
• How Staffing Agencies Automate Candidate Follow-Up With AI — Automate candidate follow-up without losing the personal touch.
• Questions to Ask a Recruitment Agency About Its AI Screening — The same questions, from the client's side of the table.
• AI Hallucinations Explained for Business Owners: Causes and Fixes — Why a model may quote a CV line that is not there.
• AI Interview Software for Video Interviews: A Hiring Checklist for Trades — A hiring checklist for trades firms: which AI interview features to use, which to avoid, and the notice, consent and bias checks to run first.
• HVAC Technician Job Description: Draft It With AI, Test Fairly — A master job description, an AI-drafted advert, an honest on-call rota, and a one-hour practical test with a scorecard that treats applicants alike.
• Plumber Job Description: Write the Ad and a Practical Test With AI — Write the plumber ad from the real working week, then add a short practical test scored by rubric, with AI drafting and a qualified plumber checking.
• AI Talent Recruiting Software: Manatal, Loxo or Bullhorn Amplify? — Manatal from $15 a user, Loxo from $149, Bullhorn from $99 on its small-agency plans: what each costs a five-person agency and what the AI does.
• AI Agents for Recruitment Agencies: Sourcing, Screening and Review Points — What an AI agent can take off a recruiter's desk, the four points where a person must check its work, and what agent credits cost per role.
• AI Boolean Search Generator for Recruiters: Prompts and Checks — How to brief an AI for a Boolean search string, what a hiring manager should review, and how to test the string on three sites and log false positives.
• AI Staffing Best Practices: A Checklist for Placements and Candidate Data — A checklist for staffing agencies using AI in placements: who decides at each gate, what candidates are told, how long data is kept and what to log.
• AI Staffing Tools for Skill Assessment: What to Test and Trust — How a staffing agency can use AI skill tests fairly: job-related tasks, rubric marking, an adverse impact check, a validity check and candidate notice.
• How to Use Claude for Recruitment at a Small Agency: Specs and Outreach — Claude for job specs and candidate outreach at a small agency: working prompts, sample outputs and the rules that keep rejection decisions with a human.
• ICO AI Recruitment Report: A Checklist for UK Recruitment Agencies — An agency-facing checklist that turns the ICO recruitment findings into evidence, owners and review points.
• AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.

Sources: EU AI Act Annex III, point 4(a) (recruitment and selection) and the Act's published implementation timeline; the four-fifths guideline in the Uniform Guidelines on Employee Selection Procedures (section 1607.4(D)); ChatGPT Business and Anthropic privacy pages on training; all checked 4 November 2026. This tutorial is not legal advice.