AI Transcription for Lawyers: How to Test Accuracy on Poor-Quality Audio

Coding Liquids tutorial cover featuring Sagnik Bhattacharya for AI Transcription for Lawyers: How to Test Accuracy on Poor-Quality Audio.
Coding Liquids tutorial cover featuring Sagnik Bhattacharya for AI Transcription for Lawyers: How to Test Accuracy on Poor-Quality Audio.

AI transcription for lawyers has no reliable single accuracy percentage on poor-quality audio: noise, overlap and quiet speech change the result. Test it on the firm's own interviews and calls against a human-checked reference. Measure word errors, speaker attribution, critical facts and correction time. Treat the output as a draft until a reviewer checks every consequential passage against the recording.

A transcript can be 95% right by word count and still misstate “did not sign”, swap two speakers or lose a figure. The test therefore needs two scores: ordinary transcription error and case-relevant error. Keep the original recording, the machine output and the corrected version distinct so the team can explain exactly what was heard and what was inferred.

Follow me on Instagram@sagnikteaches

Select recordings that resemble the firm's difficult work

Choose a small test pack before comparing products. Include at least one quiet defence interview, one two-person recorded call with interruptions, one call with a weak connection and one cleaner control recording. Use recordings the firm is entitled to process and has approved for vendor testing; otherwise create realistic, consented test audio that contains no client confidences. A glossy vendor sample with close microphones cannot answer what happens to your own muffled calls.

Connect on LinkedInSagnik Bhattacharya

For each clip, note length, number of speakers, recording device, audible defects and intended use. “Telephone call, 12 minutes, one speaker on loudspeaker, traffic noise from minute 4” is useful. “Bad audio” is not. Put one or two target passages in each file: a denial, an address, a date, an amount, a description of sequence or a change of speaker. Those passages will receive stricter review than filler words. Keep the same test pack for every candidate service.

Subscribe on YouTube@sagnikteaches

Illustrative sample: a 90-second interview excerpt contains “I did not see the envelope before Tuesday”, but a passing vehicle masks “not”. A transcript reading “I did see” is a high-impact failure even if only one word is wrong. Mark the passage “inaudible or uncertain” if listening cannot resolve it. Never repair ambiguity by choosing the statement that fits a case theory. The audio, not the model's fluent sentence, is the source.

The Complete Excel Course with AI Integration thumbnail

The Complete Excel Course with AI Integration

From formulas and pivot tables to Power Query, DAX, VBA and AI with Copilot, ChatGPT and Claude — hands-on from the first lesson.

By Sagnik Bhattacharya and Virginia Thorn, Primetime Emmy Award winner

Buy now

Make a reference transcript before scoring vendors

Have a trained reviewer produce a reference from the original audio, with a second reviewer checking disputed passages. Decide whether you want strict verbatim speech, including false starts and repetitions, or an edited record for internal notes. Use one convention across the comparison. A machine should not be penalised for dropping “um” if the accepted reference deliberately removes it, yet it must not turn uncertainty into a clean assertion. Record “[inaudible 04:18]” or “[unclear: signed/assigned]” where listening cannot settle the words.

Mark speaker turns independently from the text. For a two-speaker call, use neutral labels until identity is verified: Speaker A and Speaker B. If the interviewer says a name, that alone does not prove which voice answered. The reference should contain a short annotation whenever two people speak at once. Those overlapping seconds are where an automatic diarisation system, meaning speaker separation, is most likely to produce confident but wrong labels.

Set an adjudication rule before anyone sees the machine text. If the first reference reviewer hears “fifteen” and the second hears “fifty”, each records the timestamp and the phrase they heard independently. They then replay the original at normal speed, and may listen to a clearly identified duplicate at a slower speed. If they still disagree, the reference reads “[unclear: fifteen/fifty]”. Exclude that one word from a simple right-or-wrong score and count it in a separate uncertainty column. Otherwise the test can reward a lucky guess as accuracy.

Sonix's own help article on fixing speaker labels says poor audio quality or overlapping voices can cause it to misidentify speakers, including splitting one person into multiple labels. Its article on multiple speakers says separate tracks can be uploaded, named and combined into one automatically labelled transcript, and that multiple tracks give more accurate transcriptions. That supports a practical test: if the recording system can provide separate channels, compare them with the mixed file. Do not assume the feature exists in the firm's recording equipment or that splitting an already mixed file restores lost speech. See AI dictation accuracy for the separate issue of a lawyer speaking directly into a device.

Run a blind, identical transcription pass

Upload the same original clips to each approved service, using the same language and speaker settings. Record the product, date, settings, file version, elapsed processing time and any manual edits before export. Do not give one product a cleaned recording and another the untouched original. Export the raw transcript before correcting it; otherwise the comparison becomes a test of the editor rather than the model. If a service allows a custom vocabulary, test once without it and once with a controlled list of genuine case terms.

A custom vocabulary list might contain a company name, a technical term and an uncommon surname from the approved test audio. Do not add the answers to the target passages, such as the disputed amount, before testing. That would leak the reference into the system. Use placeholders in an unapproved environment. Ask the vendor how uploaded audio, transcripts, edits and vocabulary lists are stored and removed; do not infer that deleting the visible transcript removes every derived file.

One illustrative side-by-side comparison might show Service A turning “fifteen” into “fifty”, while Service B gets the number right but attributes the answer to the interviewer. Both need correction. A plain percentage would obscure that difference. The review form should record the timestamp, exact source phrase, machine phrase, error category, effect on meaning and reviewer action. If the source is genuinely unclear, mark the reference as unclear rather than giving one system an unearned correct score.

Keep the test blind where practical. Give the reviewers files marked Candidate 1 and Candidate 2, with vendor names removed, and rotate the order of review. A preferred brand or a cleaner looking editor can influence a subjective score even when the words are identical. Ask reviewers to flag uncertainty before consulting the reference. Only after the score is locked should the team compare usability: timestamp navigation, export format, correction controls and ease of returning to the audio. Those features matter, but they should not distort the accuracy test.

Calculate word error rate without hiding material errors

Word error rate, or WER, counts substitutions, deletions and insertions against the reference, divided by the number of reference words. For an illustrative 400-word clip, suppose the machine substitutes 18 words, omits 10 and adds 4. The calculation is (18 + 10 + 4) / 400 = 8% WER. That is 92% word accuracy under this convention. Keep punctuation and formatting out of the calculation unless they change meaning; use the same normalisation rules for all candidates.

Score the whole clip and the hard passage separately. If the 400 words contain 40 in a noisy segment and 12 of the 32 errors occur there, the segment WER is 12 / 40 = 30%. That tells the firm more than the 8% headline. Count critical errors in a separate column: negation, number, name, speaker, date, sequence and omitted uncertainty. One wrong negation may be enough to make a transcript unsuitable for the intended task even when the total WER looks attractive.

Do not turn an illustrative pass threshold into a universal legal standard. A sensible internal trial rule could be “no unresolved critical errors in any passage used for advice or evidence, and all such passages checked against audio”. That is a process criterion, not a statistical claim. The firm may tolerate more routine errors in a searchable internal index than in a quotation intended for a witness statement. The responsible lawyer sets the use and review level.

For each clip, record both the number and the type of changes. Two candidates can each have 8% WER, yet one makes eight harmless filler-word insertions while the other changes a date and omits a denial. A weighted severity score can help internal discussion, but avoid pretending that a homemade weight is a scientific measure. A clearer decision sheet states “critical word errors: two; speaker errors: four; unresolved source words: one”. That keeps the actual failure visible to the lawyer who must decide whether the transcript can support the intended work.

Test speaker labels as their own task

Measure a speaker-turn error whenever words are assigned to the wrong voice, a real speaker is merged with another, or one person is split into several identities. Count the number of turns reviewed and the number needing correction. For an illustrative 32-turn call, four turns may be mislabelled even though the words are accurate. A search for “what the client said” then returns a statement made by the interviewer. That is not a minor formatting fault.

Listen around every interruption and every change in microphone distance. Check the first turn after a long silence, where a service may restart with the wrong speaker. If there are three voices, name them only after a human confirms identity. A convenient output label should never become an evidential attribution by default. Where the audio does not support identity, preserve the uncertainty in the final transcript and in any downstream summary.

Illustrative correction note: at 06:42 the model labels “I gave it back on Thursday” as Speaker A, while the audio indicates Speaker B. The reviewer changes the label, records the reason and flags any case note that already quoted the line. Merely fixing the transcript file is insufficient if a summary was generated from the earlier version. Summarising transcripts safely covers that downstream source-checking problem.

Measure the time needed to make the draft usable

Start a timer when the reviewer opens the raw output and stop when the transcript meets the intended use. Separate listening, speaker correction, fact checking, formatting and escalation to another reviewer. A transcript that processes in one minute but takes 50 minutes to correct may be poor value beside a human route. Compare against the firm's normal method on a similar clip. Include upload, security review and export time, not just the time in the editor.

In an illustrative 20-minute interview, the AI pass takes three minutes. First review takes 28 minutes; a second person checks six disputed passages for 12 minutes; filing takes five. The total human handling is 45 minutes. A clearer 20-minute call might take 14 human minutes. The firm should not average those two recordings into one promise of savings. Sort results by audio condition and intended use. A service can be useful for clean call search while unsuitable for poor-quality defence interviews.

Put direct costs beside the time figures without inventing a universal vendor price. If a candidate quotes a per-minute transcription charge, multiply the quoted rate by all uploaded minutes, including reruns. Add reviewer minutes at the firm's own internal costing rate and any required human transcription fee. An illustrative trial might upload 80 minutes, rerun 20, and need 150 minutes of staff review. The billable upload quantity is then 100 minutes, while the staffing quantity is 150; neither can be replaced with the headline 80-minute duration. Ask whether exports, storage or extra users are charged separately before deciding.

Keep a “stop listening” rule for unusable segments. If repeated playback and a second reviewer cannot hear a material word, mark it unclear and refer the issue to the lawyer. Do not spend unlimited time refining the model's guess. If the matter requires a certified or formal transcript, ask the appropriate qualified provider and check applicable rules. The machine draft may still help locate a passage, but that is a different deliverable.

Check retention, access and permission before live uploads

Interview audio can contain privileged or highly sensitive information. Before any live file enters a transcription service, the firm should review its client obligations, recording permissions, vendor agreement, access controls, storage location, deletion behaviour and whether the content is used to train models. Ask about both the audio and derived transcripts, including temporary files, exports, backups and shared links. A setting visible in the editor is only one part of the retention story.

Sonix's security page says anyone with the right role can delete a file at any time, that deletion wipes both the audio and the transcript from its platform, and that it never uses the audio, video or transcripts customers process to train its models, on any plan or trial. Those are vendor statements, so verify contract terms and the current policy for the plan under consideration before relying on them for a client matter. The firm still controls downloaded copies, email attachments and any summaries created outside the service. A vendor deletion button cannot erase those.

Write a short internal rule for the trial: approved file types; who may upload; whether redaction is required; which accounts can share; where the reference and corrected transcript live; and who authorises deletion. Test the controls on a dummy file. An illustrative failure is a reviewer using a personal account because the firm account has a queue. The transcript may then sit outside the firm's access and retention process. The remedy is an approved fallback, not a reminder to be careful.

Recording permission needs a separate check from software permission. The fact that an audio file exists does not establish that it may be sent to an outside processor or shared with every member of a matter team. The responsible lawyer should check the firm's rules and the obligations for that matter before upload. If permission is uncertain, use a synthetic recording for the product test and resolve the question before moving to live work. Keep the approval in the matter file so a later reviewer can see why the upload was allowed.

Keep the transcript linked to the recording

Export with timestamps and a version label. Store “raw AI draft”, “reviewed transcript” and “original audio” as distinct records, with an audit note naming the reviewer and date. If a passage is quoted in a letter or court document, the lawyer should listen to that passage in the original recording. A transcript is an aid to finding and reading speech, not a substitute for the recording where wording or tone matters.

Another illustrative issue: noise reduction makes a quiet “no” sound like a click. The cleaned audio produces a fluent transcript that omits it. The original file still contains an uncertain syllable. Keep both copies, state which was used for transcription and disclose the uncertainty to the person making the legal judgement. Never silently replace the original with a processed version. The chain from source to quote should remain understandable without specialist software.

Use a final acceptance sheet with five entries: intended use, critical passages checked, speaker labels checked, unresolved uncertainty listed and storage location confirmed. Have the responsible lawyer sign off on any external use. The two-week legal AI trial scorecard can help compare this work with a wider software pilot, but the audio-specific checks above should remain intact.

Choose the route by audio condition and consequence

After testing, define routes rather than declaring one product “accurate”. Clean internal calls may receive AI transcription and a light review. Muffled interviews may require full listening and a second check of disputed passages. Audio with unresolved material words may go directly to a qualified human transcriptionist or remain marked uncertain. The recorded call may require different handling from a defence interview even if both are 20 minutes long.

Re-run the pack when the product, model, recording hardware or team process changes. Keep a few known hard clips for regression checks, subject to the firm's retention policy. Track WER, speaker-turn errors, critical errors and review time together. If overall WER improves but the system still mishears negations in the noisy interview, the relevant risk has not improved. A successful legal transcription workflow makes uncertainty explicit and makes each important word easy to trace back to sound.

Questions before transcribing a live matter

Is a high overall accuracy score enough for legal audio?

No. A transcript may score well while changing one negation, number or speaker attribution that matters to the case. Score word errors and review a separate list of consequential statements against the audio. The intended use determines how much human checking is needed.

Can a lawyer improve a poor recording before transcription?

A cleaner copy may help, but preserve the original. Test any noise reduction on a duplicate because aggressive processing can hide quiet speech. Record which copy produced the transcript and return to the original whenever a disputed word matters.

When should the firm choose a human transcriptionist?

Use a qualified human route, with appropriate confidentiality arrangements, when the recording is too unclear to resolve critical passages, speaker identity is disputed, a verbatim evidential record is required or review time erases the benefit of the AI draft. The responsible lawyer still checks the final use.

Further reads

Sources: Sonix Help, article on fixing speaker labels; Sonix Help, How does Sonix handle multiple speakers?; Sonix security page. Checked November 2026.

Need a defensible audio trial?

On a 1:1 call we can choose representative recordings, define an error scorecard and fit human review into your firm's existing file process.

Book a 1:1 call with me