AI Tax Return Review vs Software Diagnostics: What Each One Catches

Diagnostics catch what a rule was written to catch: blank required fields, broken cross-references and known e-file rejection causes. An AI review reads the source folder as well, so it can spot a transposed figure, a missing statement or an odd swing against last year. It misses anything outside the folder, and it can flag correct items with confidence.

The difference is mostly about inputs. A diagnostic sees only what has been keyed into the return, so a wrong figure that looks plausible passes every rule. An AI review can read the documents behind the return, but nothing obliges it to say what it skipped. One sample return, built with known errors, shows both gaps in an afternoon and gives you numbers instead of vendor claims.

Follow me on Instagram@sagnikteaches

Ten planted errors and two decoys: the sample return

Start with the test, because a vendor page tells you what a product reads, not how often it is right. Take a closed return from a past year or invent one. Change every name, identifier and account number, and keep the folder that belongs with it: income statements, a rental ledger, receipts, last year's return and the client intake questionnaire. Plant the errors below and keep the answer key where whoever runs the tools can't see it.

Connect on LinkedInSagnik Bhattacharya

The sample household has two earners, a rental flat and some charitable giving, and every figure is invented. Ten problems are planted. Two correct but odd-looking items are added as decoys, because a tool that flags everything will catch everything and still be useless.

Subscribe on YouTube@sagnikteaches

Write the key as a lookup: for each ID, the form and line where the error sits and the document that proves it, with the decoys marked as correct. Scoring later is then a lookup instead of an argument. Twelve items will not give you a statistical catch rate, and they are not meant to. They show which kinds of error each layer can reach at all, which is the question that decides where each one sits in your review.

What a diagnostic can see, and the question it was never asked

A diagnostic is a rule. When the data keyed into the return meets a condition, the package shows a message. Intuit's Lacerte help article sorts them into fatal (they block printing and e-file validation and cannot be bypassed), critical, e-file and informational, and it says some e-file diagnostics exist only because they cause rejections even when the paper return is fine. UltraTax CS keeps its list current "every time you add or change data" and splits it into tabs: Critical, ELF Critical (resolve before electronic filing), Overrides, FYI, Watch and others.

Run the sample return through your package and log every message with its severity. Expect E1 and E2 to light up, because a blank required field and a malformed account number are the textbook rule-based catches. E3 to E6 are different. Each is a plausible number in a legitimate field, and the rule has no second number to compare it with, because the statement it came from was never loaded into the software. That is a structural gap, not a flaw in one package.

Two settings change the result. First, count the informational messages as well as the serious ones. If the sample return throws 14 FYI items and two that matter, the real cost of diagnostics is the reviewer's attention, and you want that figure before you compare it with an AI tool's flag list. Second, if your package has a prior-year comparison report or a variance setting, switch it on for this run. E7 is the one planted error a variance check can reach, and a fair test gives each layer its best setting.

Giving the AI review the same folder

Upload the return as a PDF, every source document, last year's return and the questionnaire, then ask for evidence rather than opinions. The prompt below is built to stop the two commonest failures: findings with no source behind them, and a tidy summary that hides what was never read.

You are reviewing a draft tax return against the documents behind it.
Files supplied: the draft return (PDF), every source document, last year's
return, and the client intake questionnaire.

Task: find differences between the return and the documents. Do not suggest
tax positions and do not rewrite the return.

For each finding give: the return line or field; what the return shows; what
the source shows (file name and page); the difference; your confidence (high,
medium or low); and the question I should ask.

Rules:
1. Report only differences you can point to in a named file and page. If you
   cannot cite it, leave it out.
2. Put year-on-year swings in a separate list headed "Queries, not errors".
3. If a rule or limit matters, name where the rule comes from. If you cannot,
   write "rule not verified".
4. End with two lists: "Documents I could not read or did not open" and
   "Checks I did not run". Never write "no issues found" without both lists.

Here is the kind of output that prompt can return on the sample folder. It is illustrative, written to show what to look for, and your tool will word it differently.

The two closing lists matter as much as the table. In this sample the tool reported that it could not read bank_letter.pdf (a scanned image with no text layer) and had not opened rental_ledger.xlsx. Those two sentences explain why E2 and E7 are missing from the findings, and without them the table would look like a complete review.

Three fixes follow from reading the output with the answer key beside it:

• The sixth row is fine, the fifth is not. The 70,400 matches the statement, so row 5 is a false alarm: the tool inferred an error from the size of the swing. Re-run with rule 2 enforced so swings land in the "Queries, not errors" list, and score a decoy as a false alarm only when it is called an error.
• Close the reading gaps. Convert the bank letter to text, export the ledger to a format the tool accepts, and run again. Compare the two runs before trusting either.
• Spot-check three citations. Open the named page for the first, third and last finding. A wrong page number on a correct finding is a sign the matching is loose, and loose matching is how a misread figure slips through later.

The prompt earns its length. A flag worded "Employment income may not match the supporting documents" sends a reviewer hunting through the folder. Rule 1 pushes the output towards "return 48,215; statement_1.pdf, page 1 shows 48,125", which takes under a minute to confirm or reject. Compare your tool's wording with those two versions on the first run, because the gap between them is most of the time a review saves.

Three review products and what their pages say they read

With the method fixed, the vendor pages are easier to read. This is what each says, as of November 2026, in the vendor's own words where they are short. None of the three pages gives a catch rate for errors, so the sample return is the only evidence you will have.

The table hides a point that tool roundups tend to blur. Ready to Review and Instead prepare the return and then work through the diagnostics on their own draft, while Agent Andrew reviews a return that a person prepared. Those are different jobs. Ask each vendor which one it is selling you, because reviewing your own preparer's work is the case the sample return models, and a product that checks its own output may share its blind spots.

If you would rather try a general assistant such as ChatGPT, Claude or Gemini, the same prompt works, but redact first. The walk-through on how to redact a return before you paste it covers what to strip, and it is worth reading before any real return leaves your systems.

A one-page tally: catches, misses and false alarms

Score each tool against the key. A catch is a flag that names the right line and gives a reason that matches the key. A false alarm is a decoy called an error, or any flag with nothing in the folder behind it. Noise is every message someone had to read and dismiss. The results below are invented to show the arithmetic, so don't read them as a verdict on any product.

On those invented results diagnostics caught 3 of 10 and the AI review 5 of 10, and the two lists do not overlap. Together they reach 8. The AI raised 8 flags in all: the six in the sample output above (5 right and the D1 false alarm) and 2 more with nothing behind them in the folder, so about 63% of what it raised was worth a reviewer's time. Diagnostics raised no false alarms on the decoys but produced 14 informational messages to dismiss. Run 2 added E9, which leaves E8 for a person who knows the difference between a repair and an improvement.

The pattern to look for in your own numbers is that overlap. When the layers catch different errors, one cannot stand in for the other, and the sensible question becomes order and cost, not which to buy. When they catch the same errors, you have a candidate for dropping one.

Six ways an AI review still misses

The sample return is built to expose these. Each comes with the sign that shows it in a real review and the habit that contains it.

The document never reached the folder

E10 is the pattern. If the client never sends the sale statement, no tool can compare against it, and the AI only catches the gap when another document, here the questionnaire, says a sale happened. The sign is two documents disagreeing. The habit is to put the questionnaire in the pack on every run and to chase the missing statement before anyone reviews.

The scan was misread, so the tie-out passed

A tie-out compares the return with what the tool read, not with what the paper says. If a skewed scan turns 8,260 into 8,280 and the return was keyed 8,280, the check marks it clear. The sign is a clean result on a poor scan. The habit is to compare three figures per return with the original file, and to ask providers for text-based statements where they offer them.

The tool applied last season's limit

D2 tests this. A model can state a threshold with complete confidence and be a season out of date, and then flag a correct entry. In the invented results above the AI passed this decoy, and yours may not. The sign is a rule quoted with no source. The habit is rule 3 in the prompt: any limit must name where it comes from, and you check it against your software's current guidance before acting. A flag that says "rule not verified" is doing its job.

The same return gave different flags on a second run

In the scoring above, E9 appeared only on run 2. Generative tools do not return identical answers every time, so one run is a sample, not a verdict. The sign is two flag lists that differ. The habit is to run twice on any return above your complexity threshold, review the union of both lists, and keep both outputs in the file.

A tidy summary sat on top of a thin check

"No issues found" reads like assurance. If the tool opened seven of nine documents and ran four checks, it is a statement about those seven and four. The sign is a conclusion with no lists beneath it. The habit is the two closing lists in the prompt, and a rule that a review without them goes back for a re-run.

A treatment call dressed up as a flag

E8 sits here. Even when a tool says "check whether this is a repair or an improvement", it has handed the decision back, and it should. Whether a roof replacement is deductible now or capitalised depends on facts and rules that belong to the preparer and the reviewer. The sign is a flag that ends in a question about treatment. The habit is a short list of treatment areas your reviewer reads personally, whatever either layer says.

A review order that uses both without doubling the work

The test tells you what to expect from each layer. Put them in this order:

1. The preparer clears every critical and e-file diagnostic and notes how many informational messages were dismissed.
2. Export the cleared return as a PDF, and build the pack: return, source documents, prior-year return and questionnaire. Convert anything the tool cannot read.
3. Run the AI review with the prompt above, and save the output with the date, the tool and the prompt version.
4. The reviewer treats each flag as a question: accept and fix, reject with a one-line reason, or send to the client. Nothing is silently ignored.
5. The reviewer then reads the areas the test showed no layer reaches: treatments, elections and any questionnaire answer without a matching entry.
6. A one-page sign-off sheet records the diagnostics cleared, whether the AI review ran, how each flag was dealt with and who signed.

The same order keeps the file defensible. If a client or colleague later asks how a figure was checked, the sheet shows which layer looked at it and who made the call. The longer extract, review and sign-off workflow shows how this step fits with intake and preparation, and tying the return back to source documents goes deeper on the tie-out itself.

Put a price on it before you commit. Suppose a reviewer costs the practice $90 an hour. If reading the AI flags takes 10 minutes and replaces 25 minutes of manual tie-out, you save 15 minutes, which is $22.50. Credit pricing makes the other side of the sum explicit: at the $30 a credit on Instead's page, a return that used two credits would add $60, so time saved alone would not cover it. The case then rests on errors caught, which is exactly what the sample return measures. Ask each vendor how many credits or seats a typical return uses, and run the sum with their answer.

Making the test repeatable each season

Keep the sample pack. Re-run it after every software update, every change of AI tool or model and every edit to the prompt, and add one new planted error each season so the key does not become something a vendor's demo has seen. Agree the pass marks before the run, for example at least 7 of 10 caught, no more than two false alarms, and no review accepted without its two closing lists. Targets set after the results tend to match whatever the results were.

Use more than one return, because the layers behave differently by return type. An employment-only return tests tie-out. A rental return tests prior-year comparison and classification. A business return tests everything the preparer judged by hand. Three packs, each with its own key, is a morning's work to build and gets reused for years.

Handle the data as you would any client file. Use closed returns with identifiers replaced, read the vendor's terms and security page, and add the tool to your written security plan. The template for adding your AI tools to a security plan without gaps is a good place to record which tools may see which kinds of document. Business tiers of the general assistants don't train on business content by default, but check the setting rather than the headline.

Questions practices ask before running the test

Can I test with a real client's return?

Use a closed return with names, identifiers and account numbers changed, or an invented one. A real return needs the client's agreement and a tool whose terms you have read. Business plans such as ChatGPT Business and Claude Team don't train on business content by default, but a specialist tax product has its own terms, so read its security page before anything real goes in.

Should the AI review run before or after the diagnostics?

After. Clear every critical and e-file diagnostic first, so the AI spends its flags on tie-out rather than blank fields, and so it reads the return as it will be filed. If clearing a diagnostic changes any figures, re-export the PDF and run the review again on the updated return.

How many sample returns make the test fair?

Three is a practical minimum: an employment-only return, one with rental or investment income, and one business return. Twelve planted items per return won't give a statistical catch rate, but they show which error types each layer can reach. Repeat the run whenever the software, the tool or the prompt changes.

Does a clean AI review mean the return is right?

No. A clean result means the tool found no differences in the files it read, using the checks it ran. Ask for the lists of unread documents and checks not run, and treat a review without them as incomplete. The preparer and the reviewer still carry the sign-off.

Further reads

• AI Tax Prep Software for CPAs: What to Test Before You Switch — What to test before switching tax prep software
• AI Software for Tax Professionals: Run a Pilot and Measure Cost per Return — Run a pilot and measure the cost of each return
• AI-Driven Tax Software: How to Test Blue J and ChatGPT on Your Own Cases — Test a research tool and a general assistant on your own cases
• Tax Preparation Automation: Source Documents to Software With AI — Cut keying errors at the source before review
• Tax Services AI for a Small Practice: Intake, Chasing and Review Notes — Stop missing statements before the return starts
• AI for Chartered Accountants: What to Tell Clients About Your AI Use — What to tell clients about the AI you use
• Tax Preparation AI Software: TaxDome, Canopy and ChatGPT Compared — TaxDome, Canopy and ChatGPT sit in different product categories. Compare their AI, their cost models and the consent step before return data goes in.
• Accountants AI Risk Checklist: Error Points, Each With a Named Check — An error-point map for accounting firms: six AI failure points, each with a named check, an owner and a sample log entry.
• TaxDome Automations: What to Automate First in a Small Practice — Which TaxDome automations to set up first in a small practice: onboarding, document requests and reminders, in order, with what to leave manual.
• Using AI in Financial Planning: Tax Return Data Into a Client Plan — A planner's source-to-plan workflow for tax-return figures, with a checked example and a client question log.
• Can AI Check a VAT Return Before You File? Coding Error Checks — Export your quarter's transactions, run VAT-specific checks for rate, zero-rated and reverse-charge coding errors, and sign off the return yourself.
• AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.

Sources: Thomson Reuters Ready to Review product page, TaxGPT home and pricing pages, Instead home page, Intuit Lacerte help article on common diagnostics questions and Thomson Reuters UltraTax CS help page on reviewing diagnostic messages, all read on 4 November 2026.