For a small law firm, neither tool wins in general, so test both on your own documents. Run the same clause summary and the same client letter in Claude and in ChatGPT, on a business plan, and score the results blind. Pick on accuracy and house style, then check every citation either tool gives you.
Published comparisons age within weeks and use someone else's documents. A same-task test takes a morning, costs at most a month's subscription, and answers the question for your firm's work rather than a reviewer's. You need four tasks, a marker who does not know which tool wrote what, a scoring sheet, and one settings check before any real client text goes near either tool.
Fix the rules of the test before you open either tool
A fair comparison needs the same input, the same instructions and the same marker. Write these down first.
- Input. Pick three documents: one commercial lease or services agreement of 15 to 30 pages, one short letter from the other side, and one set of your own anonymised file notes. Replace names, addresses, figures and dates with plausible substitutes, or use public templates.
- Prompt. One prompt per task, written once and pasted unchanged into both tools. Do not tune the prompt for one tool and not the other.
- Marker. Someone who did not run the prompts, ideally a fee earner who knows the documents. They see outputs labelled A and B, not tool names.
- Repeat. Run each prompt twice per tool. Variation between two runs of the same tool tells you how much to trust a single result.
Four prompts, two runs, two tools is 16 outputs. Allow about three hours in total, including marking. A firm of four fee earners can split it so that nobody spends more than an hour.
Settings and plans: sort confidentiality first
Do not test with real client text until you know where it goes. Figures below are list prices checked on 4 November 2026; confirm them on the vendor pages.
| Question | Claude | ChatGPT |
|---|---|---|
| Business plan | Team Standard: $20 a seat annually or $25 monthly, two to 150 seats | ChatGPT Business Standard seat: $20 a seat annually or $25 monthly, from two users (shown on OpenAI's business pricing page) |
| Training on your content | None by default on Team and Enterprise | None by default on Business and Enterprise |
| Consumer plans | Free, Pro ($17 annually, $20 monthly): training is opt-out, a setting you manage yourself | Free, Go ($8), Plus ($20): training is opt-out through the "Improve the model for everyone" setting |
| Projects with reference files | Yes, files up to 30 MB each; sharing needs Team or Enterprise | Yes, 40 files per project on Business (25 on Go and Plus, 5 on Free); sharing on every plan |
| Minimum seats | Two | Two |
A sole practitioner pays for two seats on either business plan, or uses an individual plan and tests with sample documents only. If you are deciding between a general assistant and a legal-specific product, ChatGPT or a legal AI tool: which should a small firm use covers that fork; this test only compares the two general tools.
Two further checks before you begin. First, find the account-level training or model-improvement switch on consumer plans and turn it off, and screenshot the setting for the test file. Second, read your professional body's current guidance on client confidentiality and AI, and your insurer's if they have one. No general guide can tell you what your rules require, and they differ by jurisdiction and over time.
The Complete Excel Course with AI Integration
From formulas and pivot tables to Power Query, DAX, VBA and AI with Copilot, ChatGPT and Claude — hands-on from the first lesson.
By Sagnik Bhattacharya and Virginia Thorn, Primetime Emmy Award winner
Buy nowTask one: summarise the clauses that matter
Summaries are where firms save the most time and where a confident mistake costs most. Use one prompt for both tools.
You are assisting a lawyer at a four-fee-earner firm. Read the
attached agreement. Produce:
1. A table of the ten clauses most likely to matter to the tenant:
clause number, heading, what it says in one sentence, and any
deadline or sum in it.
2. A list of anything unusual compared with a standard agreement of
this type, each with the clause number.
3. A list of missing provisions you would expect.
Quote the exact words for every deadline and every sum. If you are
unsure of a clause number, say so. Do not give legal advice.Marking this task is mechanical, which is its strength. Check each clause number, each sum and each deadline against the document. Count three things per output: errors in numbers or references, clauses missed that you would have listed, and statements about "standard" practice that you cannot vouch for.
An illustrative result and what a marker catches
Imagine the lease sets rent review at every five years, a break clause at month 36 on six months' notice, and a service-charge cap that does not appear in the clause the summary cites. A good output lists all three with the right numbers. A typical weaker output gets the break date right and the notice period wrong, or attributes the cap to clause 8 when it sits in a schedule. Neither error is exotic; both are exactly what slips past a tired reader. The exact-quote instruction in the prompt is there so the marker can compare words rather than trust a paraphrase.
Score each output out of ten: one point per correct clause row (up to six), two for correct treatment of the unusual items, two for sensible missing provisions, minus one for each wrong number or reference. A score below five means the tool is not ready for summaries in your firm without full re-reading.
Task two: draft a letter in your firm's voice
Letters test something different: tone, structure and whether the tool follows instructions. Use the short letter from the other side as the stimulus.
Draft a reply from our firm to the attached letter. Our client is the
tenant. Points to cover: (a) we do not accept the proposed date; (b)
we propose 14 days for the landlord to provide the service-charge
accounts; (c) all rights reserved. Tone: courteous, firm, no
adjectives. Length: under 250 words. Use "[client]" and "[landlord]"
for names. Start with our reference "[ref]". Do not invent facts.Marking criteria for letters: did it cover all three points; did it stay under 250 words; did it invent a fact, such as a date or a clause number you did not give; is the tone something you would send after a light edit, or something you would rewrite. Time the edit. A letter that needs two minutes of fixes is worth having; one that needs ten has saved nothing.
Differences often show in length discipline and in what each tool adds unprompted. One may add a paragraph about "without prejudice" negotiations that you never asked for; another may soften "we do not accept" into "we are unable to agree at present". Both change the meaning of your instruction. Note them on the sheet, because they predict the corrections you will make every week.
Task three: citations and authorities, the honest limit
Ask each tool one research question where you already know the right answer: "What is the general test for X, and which authorities support it? Give the name, court, year and a one-line holding for each." Then open every authority it names. Count: exists and correct, exists but misdescribed, does not exist. Neither tool should be trusted on authorities without that check, and a tool that invents a plausible case name sounds as sure of itself as one that does not. If you want a routine for the checking itself, the small-firm routine for stopping invented case law sets one out.
This task is a diagnostic, not a pass mark. A tool that says "I cannot verify a source for this" more often than it invents one is behaving better for a firm, even if its answers look thinner.
Task four: your own file notes into a client update
Take the anonymised file notes and ask for a plain-English status update to the client of 150 words. This tests a skill you will use daily: turning internal shorthand into something a client can read. Score for accuracy of facts, for what it leaves out that the client needs (next step, who does it, by when) and for anything that over-promises, such as an outcome or a date the notes do not support. Over-promising is the error to weigh heavily.
Score the 16 outputs and read the pattern
Put the marks into a sheet with one row per output.
| Task | Run | Tool (hidden until marked) | Errors | Misses | Minutes to fix | Would send? |
|---|---|---|---|---|---|---|
| Clause summary | 1 | A | 2 | 1 | 12 | No |
| Clause summary | 1 | B | 1 | 2 | 9 | After edit |
| Letter | 1 | A | 0 | 0 | 3 | Yes |
| Letter | 1 | B | 1 | 0 | 6 | After edit |
These rows are illustrative, not results. Look at three patterns. First, which tool has fewer errors per task, since errors cost more than misses. Second, how far two runs of the same tool disagree: a tool whose two summaries differ by four clauses is one you must always double-check. Third, minutes to fix, which is the number closest to money. A tool that saves ten minutes on every letter and costs five on every summary is a different decision from one that is even on both.
When the result is a tie, choose on practicalities: which plan has the admin features you need, which fits your document formats, which your staff prefer to use. A tool nobody opens saves nothing. If you decide on Claude, the project, citation and data-control set-up written for tax practices carries over almost line for line.
A letter output side by side, and what you would fix
Here is an illustrative pair of openings from the letter task, labelled only A and B, so you can see the kind of difference a marker records.
A: "Dear [landlord], Re: [ref]. Thank you for your letter. We do not
accept the proposed date. We ask that you provide the service-charge
accounts within 14 days of this letter. All our client's rights are
reserved."
B: "Dear [landlord], Re: [ref]. We write in response to your
unfortunately worded letter. Our client cannot possibly accept the
proposed date and expects the accounts within 14 days, failing which
proceedings may follow. All rights reserved."A follows the brief. B adds an adjective the prompt banned ("unfortunately"), an adverb that hardens the tone, and a threat of proceedings nobody authorised. The marker's fix for B is a rewrite of two sentences and a call to the client to confirm they never wanted to threaten anything. Notice the cost: a threat inserted by a tool is not a style problem, it is an instruction you did not give. Put "no threats, no deadlines other than those listed" into the prompt and test again; if a tool still adds one on the second run, count it as a repeated fault.
A cost and time sum for a four-fee-earner firm
Say each fee earner spends about three hours a week on first-draft letters and document summaries that a tool could start. Suppose the test shows that, after your own checking, the better tool saves about 25 per cent of that time. That is 45 minutes a week each, or three hours across the firm. Over 46 working weeks it comes to about 138 hours.
Against that, four seats on a business plan at the monthly list price of $25 cost $100 a month, $1,200 a year (at the annual rate of $20 a seat, $960). Even if you value fee-earner time at a modest $60 an hour, 138 hours is $8,280. These are illustrative figures, and the saving is only real if people check the output as the test taught them to. If checking eats half the saving, the sum still comes out ahead; if the tool saves nothing on your documents, the test has just saved you $960 to $1,200 a year and a few unpleasant months. The sum is also a reminder that the subscription price rarely decides the choice; error rates and fix time do.
An extra check: comparing two versions of a document
Once the four tasks are done, add a fifth that many firms use daily: paste an earlier and a later draft of a short agreement and ask for the differences as a table with clause number, old wording, new wording and who it favours. Plant three changes in the later draft yourself, one obvious, one buried in a definition and one in a schedule. Score the number of planted changes each tool finds, and the number of invented differences it reports. Tools tend to be good at the obvious change and variable on the buried one; the invented differences are the ones that waste a partner's time. A tool that finds all three and invents nothing across both runs has earned a place in your review routine, always alongside a redline from your document software.
Which tool for which job: a decision table
After scoring, translate the marks into a simple allocation. The example below is illustrative; yours should come from your own sheet.
| Job | If errors are low and fix time is short | If the tools tie | If both score badly |
|---|---|---|---|
| Clause summaries | Allowed for first pass, full read still required | Choose the tool with fewer invented references | Do not use; read the document |
| Client letters | Allowed for routine letters, fee earner signs | Choose the one nearer your house tone | Use templates only |
| Status updates | Allowed with the next-step line checked | Choose on speed | Do not use |
| Authorities | Treat as leads and open each source | Same | Do not use for research |
| Document comparison | Second check beside your redline tool | Same | Do not use |
The fourth column matters. A firm that can say "we tried it and it was not good enough for X" has made a better decision than one that adopted a tool because a colleague liked it. It also gives you something to retest when the tools change.
Reading the unfinished results honestly
If one tool wins the clause summaries and the other wins the letters, resist the urge to buy both for every seat. Count which task your firm does more often and how much each error costs. A summary error can sit unnoticed in a file note for weeks; a letter error is usually spotted before it leaves the building because someone reads it to sign it. That asymmetry often argues for choosing on the summary result and tolerating a slightly worse letter, which you will edit anyway. Write the reasoning on the half-page note so that the choice survives a change of staff.
Keeping a record of the test
Save the test pack in one folder: the four prompts as pasted, the sixteen outputs with the tool names restored, the marker's sheet, a screenshot of each tool's privacy and plan settings on the day, and your half-page decision. That folder is cheap to build while the work is fresh and awkward to reconstruct a year later. It is also the starting point for your next test: change the documents, keep the prompts and the scoring sheet, and you will see whether a newer model version has moved any number. Add the date and the model names shown in each tool's picker, because those labels change often and a result is only meaningful for the version that produced it.
Mistakes this test commonly shows up
- Tuning one prompt. If you improve the prompt after seeing one tool's output, rerun it in both, or the comparison is spoiled.
- Marking with the labels visible. People mark the tool they expect to win higher. Hide the names.
- Using public templates only. They are well represented in training material, so tools look better on them than on your odd, amended, scanned leases. Include one messy document.
- Testing the free tiers and buying the business tier. Models, limits and file handling differ by plan. Test on the plan you will buy, paying for a single month at the monthly price if you need to.
- Forgetting the time of day. Heavy use can change speed. Note whether a tool was slow, but do not mark quality on speed.
Decide, document and set a review date
Write a half-page note: which tool, which plan, which tasks it may be used for, which it may not, what must never be pasted, and who checks what. Date it, and set a review three months on. Tools change monthly, so a decision is a snapshot. The note also answers the question a client or insurer may ask later, and it gives a new starter something to follow on day one.
If you plan to roll out beyond the test, the sequence in an AI implementation plan for a small law firm sets out the first 90 days. And if you would rather have help choosing the three tasks and running the first test, that is what my AI implementation consultation covers; the test above works without it.
Questions small firms ask when choosing between the two
Which plan does a law firm need?
A business plan: Claude Team or ChatGPT Business. Both require at least two seats and neither trains on business content by default, while consumer plans leave the training choice with each user. A sole practitioner can still run the test on individual plans using sample documents only.
Can I put real client documents into the test?
Only once your engagement terms, your professional rules and the plan allow it. A good test uses old matters with the names and figures changed, or public documents such as a template lease, so that no real client is exposed while you compare tools.
Will one tool always win?
Unlikely. Results vary by task, by month and by how each prompt is written. Expect a split, for example one tool nearer your house style on letters and the other stricter on clause summaries, and decide which task matters more to your week.
Do I need a legal-specific tool instead?
Not always. Legal-specific products add practice-management links, templates and workflow, and charge for them. If your needs are summaries and drafts, a general assistant on a business plan can be enough. The test shows whether it is.
Further reads
- AI for Small Law Firms: What to Automate First — Which tasks to hand over first
- Per-Seat or Pay-As-You-Go? Legal AI Pricing for Small Firms — Per-seat versus pay-as-you-go costs
- AI Contract Review for Small Firms: What It Catches and Misses — What AI contract review catches and misses
- How Small Law Firms Use AI to Draft Letters and Routine Documents — Letter drafting routines for fee earners
- AI Software for Estate Planning Attorneys: Automation or Generative? — Rules-based document assembly against generative drafting for wills and trusts: how each fails, what each costs and the review step that suits each.
- ChatGPT Business for Law Firms: Seats, Privacy Settings and Cost — What ChatGPT Business costs a small law firm at three sizes, the privacy and sharing settings to fix before the first prompt, and what it cannot provide.
- Claude or ChatGPT for a Law Firm: Which One to Pay For — Team seat costs, long-document limits, privacy defaults and Claude for Legal plugins compared, with a weighted three-task trial a small firm can run.
- Free ChatGPT for Lawyers: Where the Free Plan Stops Being Safe — Free ChatGPT suits generic legal drafting, not client facts. What the Free plan keeps, how to anonymise a matter, and when a business plan becomes necessary.
- AI Legal Brief Writing: A Draft and Cite-Check Routine — A routine for small litigation firms: build an authority table, draft only from it, cite-check each source line by line, and meet court AI-disclosure rules.
- Claude for Legal: Which Plugins a Small Law Firm Should Turn On — What Claude for Legal adds over a normal Claude plan, which plugins suit a small firm, and how to trial one matter type safely.
- AI for Criminal Defence Lawyers: What Is Safe With Client Confidences — A firm-level client-data policy checklist for criminal defence lawyers: safe and unsafe AI tasks, plan settings, discovery handling and a first pilot.
- Can Immigration Lawyers Use ChatGPT? Evidence Letters and Redaction — A firm-owner policy for ChatGPT in immigration practice: which plan, what to redact, how support letters get drafted and who signs them off.
- Can Personal Injury Lawyers Put Client Records Into ChatGPT? — Client-data rules for a personal injury firm: Business plan versus Free, a redaction list, three safe uses of ChatGPT and a never-paste list.
- AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.
Sources: claude.com/pricing, chatgpt.com/pricing and OpenAI's business pricing page (checked 4 November 2026); OpenAI's help article on projects; Anthropic's help articles on projects and file uploads.