An occupational therapy practice can audit documentation monthly by sampling a few de-identified notes per clinician, scoring them with AI against a written rubric, and compiling a gap list for each clinician. A senior clinician then checks the scores, gives feedback, and re-audits the same gaps the next month.
This is review work, not note writing. The AI never sees names, never writes a note, and never judges clinical reasoning. It checks whether the things your standard requires are present and consistent, which is the dull part of an audit that tends to slip when the diary fills up. Run the same way each month, it also leaves a paper trail of what you reviewed, what you found and what changed.
Checklist 1: set the standard you are auditing against
An audit without a written standard is a conversation about taste. Before the first sample, pin down what "complete" means in your practice, using your professional body's documentation guidance, your payer or funder requirements and your own policies. Keep it to what you could point to in a note.
- Identity and context present: date of service, session length, setting, who attended. Why: funders reject notes missing these. Check: field is populated, not "see above".
- Goal linked: the note names the goal the session addressed. Why: a session without a linked goal is hard to defend. Check: goal wording or reference number appears.
- Intervention described: what was done, in enough detail that another therapist could understand it. Check: a verb and an activity, not "continued programme".
- Response and progress: what the client did and how it compares with last time. Check: a measurable observation or a clear statement of no change.
- Plan: next step and date. Check: a concrete action, not "ongoing".
- Consent and safety: consent recorded where required, any incident or concern noted. Check: a line exists.
- Originality: the note is not a copy of the previous one with the date changed. Check: at least one detail differs in a way that fits the session.
- Legibility and tone: plain, factual, no judgement language about the client. Check: no unexplained abbreviations.
Eight items is enough. A rubric of thirty items becomes a form-filling exercise and clinicians stop caring about the result. If your standard has more, group them and score the group.
Checklist 2: draw the monthly sample
Sampling is where an audit becomes fair. Decide the method once and write it down.
- List each clinician's notes from the month in the practice system, exported to a spreadsheet or PDF by the person who has access.
- Number them. Pick five per clinician using a random number generator or a spreadsheet
RANDBETWEENcolumn, not "the ones that look typical". - Ensure the five cover at least two different caseload types (for instance a paediatric block and an adult rehabilitation case), by re-drawing if all five are the same.
- Record the sample on a sheet: clinician initials, note date, case type. No client names on this sheet.
Say a practice owner runs a seven-clinician practice. Five notes each gives 35 notes a month. At about three minutes per note to de-identify and about ten seconds of AI time each, plus review, the whole audit is a half-day for a senior clinician in the first month and drops towards two hours when the routine settles. For a two-clinician practice, ten notes takes under an hour.
The Complete Excel Course with AI Integration
From formulas and pivot tables to Power Query, DAX, VBA and AI with Copilot, ChatGPT and Claude — hands-on from the first lesson.
By Sagnik Bhattacharya and Virginia Thorn, Primetime Emmy Award winner
Buy nowChecklist 3: de-identify before AI sees anything
The note goes into the prompt as plain text, with every identifying item removed. Do it by hand or with a find-and-replace routine, and keep the rules identical every month.
| Remove or replace | With |
|---|---|
| Client name, nickname, initials | "Client" |
| Parent, carer or sibling names | "Parent", "Sibling" |
| Date of birth | Age in years |
| School, employer, therapist at another service | "School", "Workplace", "Other provider" |
| Address, phone, record number, funding reference | Delete |
| Exact dates | "Session 1", "Session 2" or keep only the order |
| Distinctive events (a named local incident, a rare diagnosis with a small community) | Leave the note out of the AI sample and review by hand |
Two things follow. First, keep a small standing rule that anyone who is unsure leaves the note out. Second, check the tool and plan before you paste. A business plan that does not train on your content is a better base than a free consumer account, but if your documentation falls under health-privacy law where you work, de-identification does not remove the duty to check your insurer's and regulator's position. The deeper confidentiality questions are covered in AI note-taking tools for therapists: a confidentiality checklist.
Checklist 4: score the sample with a fixed prompt
Use the same prompt every month so that results are comparable. Put it in a ChatGPT Project or a Claude Project, which keep standing instructions without retyping. Both work on business plans, and a Claude Free account can hold up to five projects. If two senior clinicians share the audit, note that ChatGPT can share a project on every plan, while Claude shares projects only on Team and Enterprise.
You are auditing one de-identified occupational therapy session note against a rubric. Do not rewrite the note. Do not comment on clinical reasoning.
For each rubric item, return: item number, PASS / PARTIAL / MISSING, and a one-line reason quoting the words in the note that justify the score (or "no evidence found").
Rubric:
1 Date, length, setting and attendees present
2 Goal named or referenced
3 Intervention described specifically
4 Client response or progress stated
5 Plan with a next step
6 Consent or safety line present where relevant
7 Not a copy of a standard phrase set (flag identical boilerplate sentences)
8 Plain, factual language; abbreviations explained
Finish with: total score out of 16 (PASS = 2, PARTIAL = 1, MISSING = 0) and a list of the three most useful fixes.
If the note is too short to judge an item, say so; do not guess.
[paste one note]An illustrative output for a note reading "Session 3. Worked on handwriting. Good participation. Continue plan." would score item 1 as PARTIAL (no length or setting), item 2 as MISSING (no goal), item 3 as MISSING ("worked on handwriting" names a topic, not an intervention), item 4 as PARTIAL ("good participation" is not progress), item 5 as MISSING ("continue plan" names no next step), item 6 as MISSING (no consent or safety line), item 7 as PARTIAL ("continue plan" is a stock phrase), and item 8 as PASS. Total 5 of 16, with fixes such as "name the goal addressed" and "state one observed change since session 2".
The quote-the-words instruction is the safeguard. A model that has to quote evidence cannot easily say PASS on an item that is not there. When you review, spot-check that every quoted phrase actually appears in the note. Do each note in a fresh chat, because scores drift when a model has just seen a stronger or weaker note.
One clinician's month of scores, item by item
Say a clinician in a five-person paediatric practice has five sampled notes. After scoring, the owner compiles the results in a spreadsheet, one row per item, one column per note.
| Rubric item | Note 1 | Note 2 | Note 3 | Note 4 | Note 5 | Month average (0-2) |
|---|---|---|---|---|---|---|
| 1 Context | 2 | 2 | 2 | 2 | 2 | 2.0 |
| 2 Goal linked | 2 | 1 | 0 | 1 | 0 | 0.8 |
| 3 Intervention | 2 | 2 | 1 | 2 | 1 | 1.6 |
| 4 Response | 1 | 1 | 1 | 0 | 1 | 0.8 |
| 5 Plan | 2 | 1 | 1 | 1 | 1 | 1.2 |
| 6 Consent/safety | 2 | 2 | 2 | 2 | 2 | 2.0 |
| 7 Originality | 1 | 1 | 0 | 1 | 0 | 0.6 |
| 8 Plain language | 2 | 2 | 2 | 2 | 2 | 2.0 |
The gap list writes itself: goals are not linked in three of five notes, response is rarely measurable, and two notes lean on identical boilerplate. The owner reads those two boilerplate notes beside the AI's quoted evidence, confirms the finding, and writes the feedback in plain terms: "In notes 3 and 5 the second paragraph is word-for-word the same as last month's. Add one line on what Client did differently this session." Goals and response get a short example of a good line from a de-identified, high-scoring note.
The score is for the conversation, not for a league table. Do not circulate comparisons between clinicians. Show each person their own row, their own quoted gaps and their own trend line.
Checklist 5: write the gap list per clinician
Ask the AI to turn the month's scored results into feedback, from the sheet rather than the notes themselves, so the notes never go in a second time.
Below are one clinician's audit scores for five notes (items 1-8, 0-2 each) and the fixes suggested for each note.
Write a gap list for that clinician: up to three recurring gaps, each with (a) what the pattern is, (b) why it matters for funders or colleagues reading the note, (c) one example of a stronger line, written generically.
Constructive, plain tone. Do not mention clinical competence, and do not compare with other clinicians.
[paste the scores and fixes]An illustrative output lists "Goal linking: missing in 3 of 5 notes", "Measurable response: 4 of 5 notes describe participation but no observed change", and "Boilerplate: 2 notes repeat the previous plan sentence", each with an example line such as "Today's session addressed Goal 2 (dressing independently); Client fastened three buttons with one prompt, compared with two prompts last session". You edit it before sending. The AI cannot know that this clinician is covering a vacancy and carrying a double caseload, and you can.
Hold each feedback conversation within two weeks of the audit, five to ten minutes, and agree one or two targets, not three or four. Record the agreed targets in the clinician's supervision file.
Checklist 6: re-audit the same gaps next month
An audit that does not return to the same gaps is a report. In the next cycle, draw a fresh random sample as usual, and add two notes chosen specifically from the clinician's weakest items from the last month. Run the same prompt, then compare item averages.
- Goal linking moves from 0.8 to 1.6: closed, keep monitoring at the standard sample size.
- Response moves from 0.8 to 1.0: a small shift, so give a worked example from a strong note and re-check next month.
- No movement: ask whether the template or practice software is part of the cause, for instance a note template that has no field for goals.
That last case is worth pausing on. Often a gap is a design fault, not a clinician fault. A template that hides the goal field behind a tab will produce "missing goal" in every note by every clinician. Fix the template and you fix the finding for the whole team. Related reading on a similar issue in physical therapy is in best AI to help with PT documentation: avoiding cloned notes.
Before and after: what a stronger note looks like to the rubric
Feedback lands better with a concrete rewrite. Here is the earlier illustrative note, scoring 5 of 16, followed by a strengthened version a clinician could plausibly write in the same time. Both are invented for illustration.
Before: "Session 3. Worked on handwriting. Good participation. Continue plan."
After: "Session 3, 45 minutes, clinic, parent present. Goal 2 (write own name legibly on lined paper): practised letter formation with a sand tray, then pencil grip with a triangular grip, then copied four letters on lines. Client formed three of four letters correctly with one verbal prompt, compared with two of four last session. Parent consented to a home practice sheet, which was given. Plan: add a first-name sentence next session; review grip after two more sessions."
Run the stronger version through the same prompt and an illustrative output scores items 1, 2, 3, 4, 5, 6 and 8 at 2 and item 7 at 1 or 2, depending on whether the plan sentence resembles the previous note. Total 15 or 16. The lesson for the clinician is the pattern, not the wording: a goal, an activity, one observed comparison and a next step. That fits on a four-line card that you can pin to the clinic wall or paste at the top of the note template.
Notice what the rewrite did not need: more words for the sake of it. It is only about twice as long, and every added phrase answers a rubric item. If clinicians say the audit makes notes longer, show them this pair. Longer is not the aim, and a rubric that rewards padding should be rewritten.
Practice-wide patterns, new starters and the template fix
After three months of sheets, stack the clinicians' item averages in one table and look down the columns, not across the rows. If item 4 (response) averages under 1.0 for the whole practice, the cause is almost certainly shared: the template, the training, or the time allowed between sessions. Treat that as a practice problem, discuss it at a team meeting without naming anyone, and change something structural, such as adding a "change since last session" field.
New starters need a different sample. For their first eight weeks, audit three notes every fortnight instead of five a month, and give the feedback within a week, while habits are still forming. Pair the gap list with a shadowing session where they read a high-scoring note from a colleague (de-identified, with permission). Locums and students can use the same rubric as a checklist before they submit, which turns the audit from a retrospective check into a drafting aid. Many errors then never reach the file.
One last use is a reply to an external request. When a funder or insurer queries documentation, the audit sheets are evidence that you review notes routinely, what you found, and what changed. Keep the sheets, with no client information on them, for as long as your records policy says you should keep quality records. That makes the monthly half-day pay for itself the first time you are asked.
Where an AI-scored note audit misleads you
The AI scores generously. Models tend to lean towards PASS. Counter it by asking for a quoted phrase, and spot-check ten per cent of the PASS items against the note. If you find a PASS with no real evidence, add "when in doubt score PARTIAL" to the prompt.
De-identification leaks. A note mentions the child's unusual hobby and a named local club. Your rule, "distinctive means leave it out", should catch it, and the monthly sheet should record how many notes you excluded and why. If exclusions rise, you have a de-identification training issue, not a tool issue.
Scores treated as truth. A clinician with a low total may be writing short, accurate notes in a setting where brief entries are the norm. The rubric measures presence of items, not quality of care. Say this to the team and mean it.
The audit becomes the note. If clinicians learn that quoted evidence earns a pass, they may begin writing to the rubric with stock phrases. Watch item 7 for this. The remedy is a rubric that rewards specifics, which is why the check says "a verb and an activity" and not "contains the word intervention".
Nobody owns it. A monthly job without a name and a diary slot lasts about three months. Put it in the senior clinician's calendar for the same week each month and decide who covers it in leave periods.
Time, cost and a decision on whether it is worth it
The AI cost is small. Thirty-five notes of 100 to 200 words, each sent with the 150-word prompt, come to roughly 9,000 to 12,000 words a month across 35 short chats, well within an everyday business plan's limits. ChatGPT Business Standard is $25 per user a month on monthly billing or $20 billed annually, from two users, and Claude Team Standard costs the same and also starts at two seats, so a small practice might simply use a plan it already holds for other admin. The real cost is the senior clinician's review time, and that is the investment that makes the output trustworthy.
A rough decision rule: if your practice has fewer than three clinicians, a quarterly audit of ten notes by hand may serve you as well. From four or five clinicians upwards, a monthly sample scored the same way each time becomes difficult to do consistently by hand, and that is where the fixed prompt earns its place. If you are comparing this with an AI scribe that writes notes for you, the questions are different, and AI scribe for occupational therapists: a buyer's checklist sets them out. For the wider set of jobs, how can AI help an occupational therapy practice: seven admin jobs gives the other six, and the AI hub collects the rest of the series.
The one-page monthly run sheet
Pin this beside the desk.
- Week 1: export the month's notes, draw five per clinician at random, record the sample sheet.
- Week 1: de-identify, leaving out anything distinctive.
- Week 2: score each note in a fresh chat with the fixed prompt; spot-check every quoted phrase and ten per cent of the PASS scores.
- Week 2: fill the spreadsheet; draft the gap list from scores only; edit it.
- Week 3: hold the feedback conversations and agree one or two targets.
- Week 4: delete the de-identified copies and chats; file the scores; add last month's weak items to next month's sample.
Run it for three months before changing anything. By then you will know whether the rubric needs a tighter item, whether a template needs fixing, and whether the half-day is worth it for your size of practice.
Questions OT practice owners ask about AI audits
How many notes should I audit each month?
A fixed small sample is more useful than an exhaustive one. Five notes per clinician, drawn at random across caseload types, gives you a trend you can compare month to month. Raise it for new starters or after a problem, and lower it for steady, high-scoring clinicians.
Can AI judge whether a note is clinically right?
No. It checks for presence, consistency and clarity against a rubric you wrote: is a goal stated, is the intervention linked to it, is there a plan. Whether the clinical reasoning is sound stays with a senior clinician.
Do clinicians need to know their notes are audited this way?
Yes. Tell the team the method, the rubric, who sees the results and what happens to them. Position it as a quality routine with feedback, not a hunt for faults, and keep the first month's scores out of any performance review.
What if a note cannot be de-identified?
Leave it out of the AI sample and review it by hand. Notes with rare circumstances, small-community details or free-text narrative about family members are the usual ones to keep out.
Further reads
- ChatGPT for Occupational Therapy Goals: Drafting Without Client Data — Goal drafting without client data, upstream of this audit.
- Best AI Note Taker for Therapists: Recording or Dictation Style? — Recording versus dictation, if you add a scribe later.
- AI in Occupational Therapy: Funding Letters and Parent Updates — Funding letters and parent updates for an OT practice.
- Telling Patients You Use AI Note-Taking: Consent Wording That Works — Consent wording when AI is involved in records.
- Best AI Software for Therapists in a Group Practice: Seat Costs — Seat costs when several clinicians share one tool.
- Best AI Tools for Occupational Therapists: School, Clinic, Home Visit — One AI pick per job for school-contract, paediatric clinic and home-visit occupational therapists, with privacy checks, cost bands and a worked example each.
- Infection Control Audit Action Plan: Turn Findings Into Fixes With AI — Paste anonymised audit findings into AI, get grouped fixes with owners and dates, then a re-audit sheet that proves each one closed.
- NDIS Occupational Therapy Report Template: A First Draft With AI — Report headings, an evidence table, a de-identified drafting method and a review checklist for an AI-drafted NDIS OT report you stay accountable for.
- AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.
Sources: OpenAI ChatGPT Business pricing page and Projects help article; Claude pricing and plan comparison page; general de-identification practice. Your own professional body and payer rules set the documentation standard to audit against.