AI-Driven Tax Software: How to Test Blue J and ChatGPT on Your Own Cases

Coding Liquids tutorial cover featuring Sagnik Bhattacharya for AI-Driven Tax Software: How to Test Blue J and ChatGPT on Your Own Cases.
Coding Liquids tutorial cover featuring Sagnik Bhattacharya for AI-Driven Tax Software: How to Test Blue J and ChatGPT on Your Own Cases.

Blue J is worth $1,498 a year per user only if it saves you more hours of research than that figure costs and answers your own questions more accurately than ChatGPT does. Test both on ten closed cases you already know the answers to, score accuracy and citation quality, then compare cost per correct answer.

The price is not the whole decision. A general assistant at $20 a month is cheap and fast but cites from memory; a specialist tool is dear and cites from a curated library. Which wins depends on your mix of questions, and no review can tell you that. A ten-question test, built from your own closed files, can. The method below takes about half a day.

Follow me on Instagram@sagnikteaches

What you are comparing, with verified prices

Prices change, so check each vendor's page on the day you decide. As of November 2026, Blue J's pricing page lists two plans. The Individual plan is $1,498 per user per year, aimed at sole practitioners, with a seven-day free trial that needs no credit card; the FAQ describes these plans as annual subscriptions, and cancelling keeps access until the end of the paid period. It lists unlimited questions, drafting and file uploads, unlimited in-line citations and source lists, and SOC 2 Type II compliance. The Team plan, for firms with several tax professionals, adds dedicated support, guided onboarding and training, firm-wide usage insights, and single sign-on with custom access controls, with no published price (the page routes you to sales). The Individual plan's feature list also names the tax coverage it includes, and that list is narrower than every tax a practice might meet; a tool that does not cover your clients' tax area cannot win this test.

Connect on LinkedInSagnik Bhattacharya

For ChatGPT, the individual Plus plan is listed at $20 a month, and ChatGPT Business has a standard seat at $25 a month billed monthly or $20 a month billed annually, with a two-seat minimum. Claude's individual Pro plan is $20 a month, or $17 a month on the annual plan ($200 paid up front), and its Team plan's standard seat is $25 monthly or $20 a month billed annually, for teams of 2 to 150. Business plans do not train on business content by default; consumer plans have a switch you must set yourself.

Subscribe on YouTube@sagnikteaches
OptionList pricePer year per user (approx.)Citations
Blue J Individual$1,498 per user per year$1,498In-line, with source lists
ChatGPT Plus$20 a month$240Only if you supply sources or it searches the web; must be checked
ChatGPT Business, annual$20 per seat a month, 2 seats minimum$240 per seatSame
Claude Pro, annual$200 a year paid up front (about $17 a month)$200Same

The price gap is roughly six to seven and a half times. The question is whether the gap buys accuracy and time on your questions. Before you start, read TaxDome, Canopy and ChatGPT compared if you also need to choose practice-management software, since research tools and practice platforms are separate decisions.

The Complete Excel Course with AI Integration thumbnail

The Complete Excel Course with AI Integration

From formulas and pivot tables to Power Query, DAX, VBA and AI with Copilot, ChatGPT and Claude — hands-on from the first lesson.

By Sagnik Bhattacharya and Virginia Thorn, Primetime Emmy Award winner

Buy now

Step 1: build the ten-question test set from closed cases

The aim is questions where you already know the right answer and can show the source. Pull ten questions from files you have closed, so the answers are settled. Anonymise: change names, round figures, remove identifiers. Aim for a spread, because a tool may be strong on one type and weak on another.

#Type of questionIllustrative example
1Simple rule lookupA threshold or filing deadline you can state from memory
2Rule with a conditionDoes a deduction apply when a home office is used part-time by a sole trader?
3Calculation with a ruleDepreciation or allowance on an asset bought mid-year
4Rule that changed recentlyA limit or rate that moved in the last two years
5Edge caseA client with income in two tax areas
6Entity choiceSole trader or company for a plumbing firm earning a given profit
7Trap questionA fact pattern where the obvious answer is wrong
8DocumentationWhat records must be kept for a claim
9Client explanationExplain the answer to question 2 in plain language
10Question outside coverageSomething the tool should decline or flag as uncertain

Questions 4, 7 and 10 do the most work. Rules that changed recently expose stale knowledge; a trap question exposes pattern-matching; and an out-of-coverage question shows whether the tool admits ignorance or bluffs. For each question write down, before testing, the correct answer, the primary source (statute, guidance paragraph, form instruction) and your confidence in it.

Step 2: run the same questions on both tools, the same way

Consistency makes the comparison fair. Use the same wording on both tools. Start a fresh conversation for each question, so earlier answers do not colour later ones. For ChatGPT, use the model and plan you would actually use in practice, with web search on if you would normally use it. Use the identical prompt shell each time.

You are assisting a tax professional. Answer the question below
for the tax year stated. Give: (1) the answer in two sentences,
(2) the rule or provision relied on, with the exact reference,
(3) any condition that would change the answer, and (4) how
confident you are, and what I should check at source. If you are
not sure a rule exists, say so rather than guessing.

Tax year: [YEAR]
Facts (anonymised): [FACTS]
Question: [QUESTION]

Save every answer in a document, with the tool name and date. Blue J's own interface takes the question in its own way; ask it the same question in plain words, and record whether it shows in-line citations and a source list. Take screenshots only of your own test content, never of client data. Your trial is the cheap way in: use the seven-day free trial for Blue J so the whole test costs nothing, and book the half day inside those seven days.

Step 3: the citation check, answer by answer

This is the step that separates a research tool from a confident guess. For every answer from both tools, check each citation against the primary source.

  1. Does the cited provision exist? Open it. Invented references are the main risk with general assistants.
  2. Is it the right version for the tax year? A real provision quoted at an old rate or limit is still a wrong answer.
  3. Does it say what the tool claims? Read the paragraph; summaries often overstate.
  4. Is the conclusion the one your file reached? Mark correct, partly correct or wrong.

Use a four-line scoring code per answer: C for answer correct, S for citations all real and supporting, V for version and year right, and H for honest about uncertainty. Each is yes or no. Maximum 4 points per question, 40 per tool. A general routine for catching fabricated references is in how to stop AI inventing case law; the same discipline applies to tax rules.

A worked example: scoring and the break-even sum

An illustration for a sole practitioner. After the test, the scores come out like this (illustrative figures, not results from a real run).

MeasureBlue J (trial)ChatGPT, web search on
Answers correct (of 10)86
Citations real and supporting (of 10)95
Right year and version (of 10)85
Honest about uncertainty (of 10)86
Total (of 40)3322
Average time to a usable answer6 minutes9 minutes (checking took longer)

Now the money. Suppose the practitioner bills $150 an hour and does about 120 research questions a year of the sort tested. If the specialist tool saves three minutes per question compared with the general assistant, because fewer citations need rebuilding, that is 360 minutes a year, 6 hours, or $900 of billable time. Against the extra annual cost of about $1,258 ($1,498 minus $240), the tool does not pay for itself on time alone. If instead it saves 10 minutes per question, that is 20 hours, or $3,000, and it clearly does. And if the real gain is avoiding one wrong filing position that costs a client penalties and costs you a write-off of a day's work, the decision looks different again.

The break-even rule: divide the extra annual cost by your hourly rate. Here that is $1,258 divided by $150, about 8.4 hours. If the tool credibly saves you more than 8.4 hours a year over the cheaper option, it pays; if you doubt it, keep the cheaper tool and do the checking. Those figures are examples; use your own rate and volume.

Where each tool tends to win

When a specialist tool earns its fee

It tends to win where you ask many rule-based questions in its coverage area, need citations you can hand to a reviewer, and value a curated library over breadth. Practices with several preparers benefit when everyone gets the same sourced answers. The Team plan's firm-wide usage insights and access controls may matter if you have a compliance duty to show how research was done, but its price is not published, so you must ask.

When a general assistant is enough

General assistants are strong at the work around the rule: drafting the client email, restructuring a research memo, explaining a result in plain language, comparing two options in a table. If most of your AI use is that kind of work, a $20 plan covers it, and you verify any rule you rely on from the primary source. The risk grows when you ask it for the rule itself, so feed it the text of the provision and ask it to apply it, rather than to recall it. The difference between that and software checks is explained in AI tax return review against software diagnostics.

When neither is the right answer

Some questions should go to a primary source or a colleague: novel structures, disputed positions, anything where the cost of error is large. Question 10 in your test set tells you how each tool behaves at that boundary. A tool that flags uncertainty has given you something useful; one that produces a confident answer is dangerous for exactly those questions.

One question scored in full: a trap question

Numbers in a table hide how scoring feels, so here is one question worked through. Illustrative question 7 (a trap): "A client who runs a cleaning company bought a van in March and uses it 70% for the business and 30% privately. Can the full purchase price be claimed?" You know the answer from the closed file: no, only the business-use share qualifies, and the claim is subject to the rules in force for that tax year.

An illustrative general-assistant reply: "Yes, the van qualifies for the full first-year deduction, citing the relevant provision for vehicles used in a trade." You mark it C = no (wrong answer), S = no (the provision is real but concerns a different asset class), V = no, H = no, because there was no uncertainty flagged. Score 0 of 4.

An illustrative specialist-tool reply: "Only the business-use proportion qualifies; apportion at 70%. See the source list for the allowance rules and the mixed-use guidance." You open both sources, find the guidance paragraph says what the tool claims, and the year is right. C yes, S yes, V yes, H partly (it did not flag that the 70% business share needs a mileage record behind it). Score 3 of 4, with a note to ask the client for the mileage log.

The lesson is not that one tool is always right. The trap question catches a model that matches a familiar pattern ("van, allowance") rather than reading the facts, and your check is what catches it. If your own test shows the reverse, believe your test.

A question about a recent change

Question 4 is where general assistants most often fail, because their training stops at a date and rules move. Choose a limit or rate that changed within the last two years and that you know well. Run it in both tools with the tax year stated. An illustrative failure: the assistant states last year's limit as current, with a confident explanation. Even with web search on, it may find a commentary article rather than the official page. Score V as no.

A fix to try: paste the current official text into the prompt and ask the model to apply it. If the answer then comes out right, you have learned that this tool is safe for applying text you supply and unsafe for recalling rules, which is a perfectly workable rule for the team: "Paste the rule first." Write it in your procedure.

The out-of-coverage question

Question 10 asks about something the specialist tool does not cover, for example a niche tax or a different tax area from the one it supports. The right response from a research tool is to decline or say it is outside its library. In an illustrative run, the specialist tool says the question falls outside its coverage and suggests consulting primary sources, which scores H yes. The general assistant gives a fluent answer with a citation you cannot find, scoring H no. This is not a defect in the general assistant so much as its nature, and it is the reason you never use one as the final word.

Questions to put to the vendor before you commit

If the trial goes well, put your remaining doubts to the vendor in writing, and keep the answers with your test note. The plan terms quoted earlier already answer part of the terms row, so ask specifically whether any refund applies if you cancel early.

AreaQuestionWhat you need to hear
CoverageWhich tax areas and jurisdictions are covered, and which are not?A written list that matches your client base
CurrencyHow quickly are law changes added, and is there a changelog?A stated update routine with dates
DataIs anything I type or upload used to train models? How long is it kept?No training on customer data; a stated retention period
SecurityCan I see the SOC 2 report under NDA?Yes
TermsIs the plan annual only? Can I cancel mid-term or get a refund?The exact terms, in writing
Team pricingWhat does the Team plan cost for three users, and what does it add?A quote with a price per seat

The Team plan matters because many small practices have two or three preparers. If the per-seat price is higher than the Individual plan's, the break-even sum changes; rerun it with the quoted figure. A seven-day trial is short, so ask whether it can be extended while you finish the citation checks.

Protecting client data during the test and after

The test itself should carry no client data. After you adopt a tool, your rules matter more. A short written rule set works: no client names, tax identifiers or full account numbers in any AI tool unless the vendor's terms and a data-processing agreement cover that use; anonymise facts by default; paid business plans only, not free consumer accounts; and a named person who reviews tool settings quarterly. The consumer plans for ChatGPT and Claude have a settings switch for training on your chats, while business plans do not train on your content by default, but read the current terms and confirm yourself rather than relying on a summary.

Keep a log of which tool was used on which piece of client work. If a reviewer or a client later asks how a position was reached, you can show the question, the tool, the source check and your sign-off. That log is cheap, and it is also the evidence that your AI use is controlled.

A realistic mistake and how it showed up

A common one: an assistant gives a tidy answer with a plausible reference to a provision, and the practitioner cites it in a client memo. A reviewer later opens the provision, finds it covers a different situation, and the memo has to be withdrawn. The fix is built into this test: citation checks are not optional, and the time they take belongs in your cost sums. It also helps to write the rule down as a policy, as in the seven AI mistakes that put client trust at risk.

Step 4: decide, and write down what you decided

Use a simple rule set agreed before you see the scores. Illustrative: adopt the paid tool if it scores at least 30 out of 40, beats the general assistant by at least 8 points, never fabricates a citation in the ten questions, and passes your data-handling check. Otherwise stay with the general assistant plus mandatory source checking. Record the date, versions, scores and decision in a one-page note.

  • Re-run the test annually, or when a vendor announces a major model or library change.
  • Add any question that either tool got wrong to a "known failures" list your team reads.
  • Keep client data out of any tool whose terms you have not read; for a data-handling checklist see how to stop AI tools training on your business data.
  • Decide who can use which tool for client work, and write it in your procedures.

If you are weighing wider pilots, running a pilot and measuring cost per return shows how to extend a test like this across a busy season. Whichever way the test comes out, the numbers are yours, drawn from your cases, which is more convincing to a partner or a client than any vendor demo.

Questions tax practices ask about Blue J and ChatGPT

How much does Blue J cost for a small practice?

The Individual plan, aimed at sole practitioners, is listed at $1,498 per user per year, billed annually, with a seven-day free trial and no card needed. The Team plan for firms with several tax professionals is quoted on request, so there is no published figure. Check the vendor's pricing page for current terms before deciding.

Can I use ChatGPT for tax research instead?

You can, with limits. A general assistant is cheap, at $20 a month for an individual paid plan, but it may cite rules that do not exist or are out of date. If you use it, give it the source text and check every citation. The test described below shows how big the gap is on your own questions.

Should client data go into the test?

Use closed cases with names, numbers and identifiers removed or changed. Never put identifiable client data into a tool whose data terms you have not read. The test works with anonymised facts because the question is about the tax rule, not the person.

What if both tools get a question wrong?

Record it and treat it as the most useful result of the test. It tells you which kinds of question need a human or a primary source regardless of tool, and that goes into your written procedure for AI-assisted research.

Further reads

Sources: Blue J pricing page and FAQ (Individual $1,498 per user per year, annual subscription, 7-day free trial with no card, Team plan via sales); ChatGPT pricing and ChatGPT Business pricing pages; Claude pricing page; OpenAI Data Controls FAQ and Anthropic privacy articles on model training; all checked 4 November 2026. The test, scores and sums are illustrative.

Want an AI research test built around your own clients?

On a 1:1 call we can sketch the research questions your practice answers most often, decide which tool is worth testing on them, and write the checking step into your procedure.

Book a 1:1 call with me