A GP practice can summarise Friends and Family Test comments by removing names and clinical details, asking an AI tool to assign each comment one or two themes from a fixed list, counting the themes in a spreadsheet, and then reporting the counts plus three or four real quotes. A person approves every action.
The official guidance fixes the question and the response scale, requires at least one free-text question, and leaves the analysis of the words to each practice. The free-text box is where the useful information sits, and also where the privacy risk sits, because patients write names, conditions and dates. The method below handles both in an afternoon a month.
What the standard question gives you and what it leaves out
The mandatory question in the current Friends and Family Test guidance, in force since 1 April 2020, is "Overall, how was your experience of our service?". The answer scale is fixed too: very good, good, neither good nor poor, poor, very poor and don't know. Every practice must add at least one free-text question of its own choosing, and where it has no local preference the guidance recommends a pair: "Please can you tell us why you gave your answer?" and "Please tell us about anything that we could have done better". The overall-experience question replaced the older "how likely are you to recommend" wording, so check that your forms, posters and text-message templates no longer show the previous version.
That recommended pair is worth keeping if you have it, because it gives the AI two distinct jobs: the first answer explains the rating, the second is already a suggestion. Coding them as separate columns tells you which themes are reasons and which are requests.
The scale tells you how many patients were happy. It does not tell you why. The comment box does, and a practice that gets 200 responses a quarter might get 80 to 100 comments. Reading them is easy once and awkward every quarter; coding them consistently across quarters is the part that tends to slip. That is the job AI does well, as long as you design the process so it cannot go wrong in the ways that matter.
The Complete Excel Course with AI Integration
From formulas and pivot tables to Power Query, DAX, VBA and AI with Copilot, ChatGPT and Claude — hands-on from the first lesson.
By Sagnik Bhattacharya and Virginia Thorn, Primetime Emmy Award winner
Buy nowStep 1: Export responses and strip what must not leave the practice
Most collection tools (text message surveys, tablets in reception, paper cards typed in, online forms) give you a spreadsheet. Keep it in one place, with these columns: response ID, date, the scale answer, and one column per free-text question. Delete or never import anything that identifies the person: phone number, name, patient number, email.
Then redact the comments, because patients identify themselves and others in the text. Four kinds of detail to find and replace:
- Staff names: replace with "[clinician]" or "[receptionist]" and keep the role. "Dr [clinician] was kind" is enough.
- Other patients and family: "my husband" can stay; "my neighbour Mrs Smith" cannot.
- Conditions and treatments: a comment about "my cancer follow-up" reduces to "[health condition]" unless the condition itself is the theme you want to study, in which case keep the category, not the detail.
- Dates, places and anything unusual: "on Tuesday 9th at 4pm" becomes "[date]"; rare events become "[event]".
A spreadsheet find-and-replace handles staff names once you build a list. For the rest, a person reading each comment is faster and safer than any tool. At about 15 seconds a comment, 90 comments take around 22 minutes. The tutorial on anonymising before pasting into AI describes the replace-and-restore routine that keeps a copy of the original.
A before and after shows what to aim for. Before: "Dr Patel was lovely about my mum's dementia diagnosis but I waited 25 minutes after my 10.40 appointment and the receptionist Karen was rude on the phone on Monday." After: "Dr [clinician] was lovely about [family member's health condition] but I waited 25 minutes after my [time] appointment and the receptionist [name] was rude on the phone on [day]." The themes (clinician manner, waiting time, phone manner) survive; everything identifying has gone. The "25 minutes" figure stays because it is useful and not identifying.
Step 2: Pick the tool and settings before any comment goes in
Use a business or team plan of the chat assistant you prefer; both ChatGPT Business and Claude Team list at $20 a seat a month annually ($25 monthly) with a two-seat minimum, and business plans do not train on your content by default. A solo manager might use an individual paid plan with training switched off in privacy settings. Free plans are the wrong place for patient-derived text even after redaction, because redaction is never perfect.
If you work inside a larger platform, check whether it already offers comment analysis with a data agreement; that route is often better than a standalone chat tool. Where your law requires a processing agreement or a business associate agreement for any vendor handling patient-derived data, have it signed first. The answer on putting customer data into ChatGPT explains the plan differences in plain terms.
Step 3: Build a code frame before you ask AI to code anything
If you ask an AI to "find the themes", it will produce a different list every quarter, and you will not be able to compare. Instead decide your theme list yourself, from the last quarter or two of comments, and keep it fixed (add a new theme only when you see a pattern that does not fit).
A starter code frame for a GP practice, to adapt:
| Code | What it covers |
|---|---|
| ACCESS-APPT | Getting an appointment, waiting for the next free slot |
| ACCESS-PHONE | Phone queue, engaged lines, call-back |
| ACCESS-ONLINE | Online booking, apps, forms, repeat prescription requests |
| WAIT-ON-DAY | Waiting after arriving for an appointment |
| MANNER-CLINICAL | How a doctor or nurse communicated, listened or explained |
| MANNER-FRONT | How reception or admin staff communicated |
| PRESCRIPTIONS | Repeat prescription timing, errors, pharmacy hand-offs |
| FACILITIES | Building, parking, seating, toilets, signage |
| CONTINUITY | Seeing the same clinician, being known |
| OTHER | Anything that does not fit; list it for review |
Add a second tag for sentiment: positive, negative, mixed. A comment like the one above is "mixed", and it has three codes. Most comments carry one or two codes, so totals add up to more than the number of comments, which is normal.
Step 4: The coding prompt, and what the output should look like
You are coding anonymised patient feedback for a GP practice.
Use only these codes: ACCESS-APPT, ACCESS-PHONE, ACCESS-ONLINE, WAIT-ON-DAY,
MANNER-CLINICAL, MANNER-FRONT, PRESCRIPTIONS, FACILITIES, CONTINUITY, OTHER.
For each numbered comment return one row: ID | codes (one to three, comma
separated) | sentiment (positive, negative or mixed) | a quote of at most
12 words copied exactly from the comment that best supports the main code.
If a comment is empty or unreadable, write "no comment". Do not invent
details. Do not summarise across comments yet. Do not name individuals.
Comments:
[paste 30 comments at a time, each with its ID]Thirty comments per batch keeps accuracy up and lets you check each batch. An illustrative row of output: R-0412 | WAIT-ON-DAY, MANNER-CLINICAL | mixed | "waited 25 minutes after my [time] appointment". Always ask for the exact quote: it exposes a code that was not earned, and it gives you a real quote for the report.
Check a sample of the coding by hand
Take 10 of the 90 comments at random and code them yourself without looking at the AI's answer, then compare. If you disagree on more than two, tighten the descriptions in the code frame (usually two categories overlap) and re-run. This is the single step that makes the whole analysis defensible when a partner or the patient group asks how you arrived at the themes.
Step 5: Count with a spreadsheet, not the AI
Paste the AI's rows into a spreadsheet, split the codes into columns, and use a filter or pivot table to count. Chat models are poor at counting long lists, so any total in your report should come from the sheet. This also keeps a clean audit trail: the same rows give you the count and the quotes.
Here is a worked illustration with made-up numbers for one quarter. The practice received 212 responses. On the standard question: very good 118, good 61, neither good nor poor 14, poor 11, very poor 6, don't know 2. Positive (very good plus good) is 118 + 61 = 179 of 212, about 84 per cent. Poor or very poor is 17, about 8 per cent. Of the 212, 97 included a comment.
| Theme | Comments mentioning it | Of which negative or mixed |
|---|---|---|
| ACCESS-APPT | 31 | 27 |
| ACCESS-PHONE | 24 | 20 |
| MANNER-CLINICAL | 22 | 3 |
| WAIT-ON-DAY | 11 | 10 |
| PRESCRIPTIONS | 9 | 7 |
| ACCESS-ONLINE | 8 | 6 |
| MANNER-FRONT | 7 | 4 |
| FACILITIES, CONTINUITY, OTHER | 14 | 8 |
The reading: appointment access and phone access dominate the negatives (47 of the 51 negative or mixed comments sit in those two themes in this example, though one comment can count in both), while clinical manner is overwhelmingly positive. The action list writes itself: look at the phone system's busiest 30 minutes, and consider whether a call-back option or a different mix of on-the-day and booked slots would help. The numbers never decide the fix; they decide where to look first.
Step 6: The one-page report for partners and the patient group
Ask for the report only after the counts exist, and give the AI the counts, not the raw comments, so the totals come from you.
Write a one-page quarterly Friends and Family Test report for practice partners
and the patient participation group, from the figures and quotes below.
Use only the numbers and quotes supplied. Structure: headline (two
sentences with the percentage positive and number of responses), what patients
praised (with two quotes), what patients found difficult (with two quotes and
the number of comments), what we will look at next quarter (leave this blank
for me to complete), a note on limits (response count, comments are a
self-selected group). Plain English, no jargon, no praise or blame beyond the
data. 250 words maximum.
Figures and quotes:
[paste]Quotes need one more check before they travel. The guidance says a practice that wants to publish free-text comments must make sure the person who wrote them is happy for that, and suggests a tick box on the form. A redacted quote in the partners' copy is an internal matter; a quote on a poster, the website, a newsletter or a copy circulated beyond the practice should come only from a comment whose author ticked that box. If your form has no such box, add one now and paraphrase the theme until consented quotes build up.
The first draft will probably be fluent and over-confident. Typical fixes: "patients overwhelmingly praised" when the count is 22; "a significant increase" when you gave no comparison quarter; a quote slightly polished. Replace every adjective with the number, restore the quote exactly as it was written (after redaction), and write the "what we will look at" section yourself. If you have last quarter's figures, add them to the prompt so comparisons are real.
Step 7: A you-said-we-did poster that tells the truth
The guidance requires practices to publish their results locally, lets each practice choose the format, and recommends "you said, we did" as a useful one. A poster works when each "you said" is a real pattern and each "we did" is a real, dated action. Three or four rows beat ten. Where you have not acted yet, say "we are looking at" and give a date to report back, never a promise you cannot keep.
| You said | We did or will do |
|---|---|
| It is hard to get through on the phone first thing | From [date] we have added a second line between 8 and 9.30am; we will report the effect in [month] |
| I waited a long time after my appointment time | We are checking where delays happen on Mondays and will report back in [month] |
| I could not tell how to order a repeat prescription online | We have added a step-by-step guide at reception and on our website |
Give the AI your three actions and ask it to put each into patient-friendly words at reading age 12, at most 20 words per cell. Then check three things: the action is real, the date is right, and nothing implies a result you do not yet have. Have the patient group read the draft before printing; they will spot jargon in a minute.
Other jobs the same routine can do
- A running theme log. Keep each quarter's counts in one sheet so you can see whether ACCESS-PHONE is falling after the change you made.
- Thank-you wording for staff. Pull the positive comments about MANNER-CLINICAL and MANNER-FRONT (still redacted) and ask for a short note to the team quoting two of them. Share it at the team meeting, using the names you hold privately, never in the AI tool.
- Reply to a patient who left contact details and a complaint. That is not an FFT task: route it to your complaints procedure and do not run it through the coding process.
- A prompt to check the code frame. Give the AI 20 redacted comments tagged OTHER and ask it to propose, not apply, one or two new themes.
- A plain-English version for the website. A 100-word summary of the quarter, using the figures you have approved.
Related methods in other sectors show the same shape. Analysing customer feedback surveys with AI covers the general case, and summarising a school parent survey deals with hundreds of comments at once.
Handwritten cards, text replies and other messy inputs
Not every comment arrives as tidy typed text. Three awkward sources are common, and each needs a small adjustment.
Handwritten cards in a box at reception. Someone has to type them. Do the redaction while typing: replace names as you go, so the typed version is already clean. Do not photograph the cards and upload the images to an AI tool, because handwriting often carries a name or signature at the bottom.
Text-message replies. These are short ("Great service thanks", "Too long on hold") and often one word. They code well, but expect many OTHER or single-code rows. Do not stretch a three-word reply into a theme it does not support; the prompt's exact-quote rule stops that.
Comments in another language. AI can translate and code them in one pass, but ask it to give the original quote and an English gloss, and have a colleague who speaks the language check a sample, since tone and idiom are where translation slips. A comment you cannot verify goes under OTHER with a note.
Comparing two quarters without fooling yourself
The real value arrives in the second and third quarter, when you can see whether an action changed anything. Keep the comparison honest with a few rules.
Say last quarter the practice had 190 responses and 84 comments, of which 29 mentioned ACCESS-PHONE (about 35 per cent of comments). This quarter it has 212 responses and 97 comments, of which 24 mention ACCESS-PHONE (about 25 per cent). The share fell from 35 to 25 per cent, which is a ten-point change, and the raw count fell from 29 to 24. That is encouraging, but both figures come from self-selected comments, and the quarter changed in other ways too (perhaps a public holiday week). In the report, write: "Comments about the phone fell from 29 to 24, and from 35 to 25 per cent of comments. This is in the direction we hoped after adding the second line, but the numbers are small, so we will watch it next quarter." That sentence is accurate, modest and useful.
Three rules protect you. Compare percentages of comments as well as counts, because response numbers change. Never compare a theme's count if you changed the code frame in between. And do not claim an action caused a change; say what changed and what you did, and let readers draw their own link.
A monthly rhythm that fits around surgery hours
Quarterly reporting does not need a quarterly scramble. The monthly data return already asks for counts by response category and by collection method, not for the comments, so the coding below is extra work the return does not do for you. Spread it out and it costs perhaps 90 minutes a month.
- Weekly, 5 minutes: export new responses, delete identifying columns, and file them in the shared folder.
- Monthly, 40 minutes: redact and code the month's comments in two batches, check ten at random, and add them to the running sheet.
- Quarterly, 45 minutes: counts, the one-page report, the poster update, and a short agenda item for the partners' meeting or the patient group.
If your team wants another set of eyes on the survey itself rather than only its output, the sibling tutorial on writing a short patient satisfaction survey with AI covers question design and an action log.
Personal data handling: a short policy for the practice file
A paragraph in your data-protection documentation makes the process auditable. Draft it with your data-protection lead; the points to cover are:
- What is processed: anonymised free-text patient feedback.
- Which tool, on which plan, and what the agreement with the vendor says about training and retention.
- Who may run it and who reviews the output.
- How redaction is done, and that the original comments never go into the tool.
- Where outputs are stored and how long they are kept.
- What to do if a name or condition slips through: delete the chat, log the incident and tell the data-protection lead.
If a comment contains a complaint, a safeguarding concern or an allegation about a clinician, it stops being feedback. Take it out of the batch before coding and give it to the person who handles that process the same day.
Mistakes that make the summary misleading
- Mistaking volume for importance. Twenty comments about parking may matter less than three about a missed result. Add a short "serious concerns" line that a person writes after reading everything.
- Inconsistent codes between quarters. The fix is the fixed frame above.
- Treating a response count as the practice list. 212 responses from a list of several thousand is a sample of the people who chose to answer. Say so.
- Letting AI generalise. "Patients feel unheard" appears in the report though only two comments said anything like it. Always ask for the number of comments behind a claim.
Friends and Family Test questions practice managers ask about AI
Can I paste patient comments straight into ChatGPT?
Not unedited, and not into a free consumer plan. Comments often name staff, other patients or conditions. Redact names and details first, use a business plan whose terms exclude training, and have a processing agreement in place where your law requires one.
Should AI count the themes or should I?
Let AI propose the theme for each comment, then count the codes yourself in a spreadsheet. Chat assistants miscount long lists, so the totals in your report should come from a filter or a pivot table, not from the AI's own arithmetic.
How many comments do I need before themes mean anything?
A theme from two comments is an anecdote, not a pattern. Treat a theme as worth acting on when it appears in a handful of comments or in roughly one in ten, and name a small number plainly in the report rather than implying a trend.
Do I have to publish a you-said-we-did poster?
The guidance requires practices to publish their results locally but leaves the format to them, and it recommends "you said, we did" as a useful one. A poster plus a short website version covers most practices. Quote a patient's comment publicly only if they agreed to publication, for example by ticking a box on the form.
Further reads
- AI-Powered Chatbot Best Practices: A Weekly Review Routine for Clinics — A weekly review habit for anything patient-facing.
- AI Survey Analysis for HR Consultants: Coding Employee Comments Safely — Coding sensitive survey comments safely, from the HR side.
- Event Feedback Survey: Write It and Summarise the Answers With AI — Write a short survey and summarise the answers.
- Hotel Guest Feedback: Turn Post-Stay Surveys Into Fixes With AI — How feedback turns into a fix list with owners.
- Museums and AI: Turn Visitor Feedback Into Next Season's Plan — A themed summary for a small team with a tight budget.
- ChatGPT for Medical Notes: A Safe Drafting Workflow for Small Practices — A safe drafting workflow for small medical practices.
- Best AI for Paediatricians: Vaccine Recalls and School Forms — Build a vaccine recall list, letter templates and a school and camp form queue with AI, keeping parent data safe and symptom advice out.
- CQC and GP Practices: Build an Evidence Folder With AI — Organise a GP practice's CQC evidence by key question and quality statement, with an AI-built index, policy review log and mock-inspection summary.
- AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.
Sources: the official guidance Using the Friends and Family Test to improve patient experience (published September 2019, in force from 1 April 2020), including its general practice setting summary and the guidance page on monthly data submission; ChatGPT Business pricing page; Claude pricing page; figures in the examples are illustrative.