Can AI Read Bills of Lading and Fill Your TMS? How to Run Your Own Accuracy Test

Often yes on clean documents, but you should not trust it until you have tested it. Run 100 of your own bills of lading and air waybills through the tool, compare every field with what your staff keyed, and set a pass mark per field. Only validated, high-confidence results should reach the TMS.

Vendors quote accuracy figures measured on their own samples. Yours will differ, because your carriers, scan quality and handwritten amendments differ. The accuracy test is one you run yourself: two to three days of staff time that tells you which fields can be automated, which need a rule, and which will always need a person. The scores shown further down are an illustration of how to read results, not a measured outcome.

Follow me on Instagram@sagnikteaches

What you are asking the AI to read

Three document families matter to a forwarder, and each fails differently:

Connect on LinkedInSagnik Bhattacharya

• Ocean bills of lading. A master bill is issued by the carrier; a house bill by the forwarder to its customer. Layouts vary by carrier, and many have continuation pages for long container lists.
• Sea waybills and other non-negotiable forms. Similar content, often cleaner.
• Air waybills. The master air waybill is issued by the carrier or its agent and covers consolidated cargo. The house air waybill is issued by the forwarder and lists each shipment inside a consolidation. Mixing the two up is a classic error.

The TMS (transport management system) needs structured fields. Reading text off a page is only the first step; the real work is mapping messy layouts to the right field every time.

Subscribe on YouTube@sagnikteaches

The field map: what to extract and how to check each one

Write the map before you test anything. Each row is a field, its format and a rule that can catch a wrong value. Drafting this table is a good use of AI itself, but edit it with the person who keys the documents today.

Two checks worth coding on day one

Container numbers have a built-in check digit. The structure is three owner letters, a category letter (U for freight containers), six serial digits and one check digit. Each letter converts to a number (A is 10, B is 12, C is 13 and so on, skipping multiples of 11), each value is multiplied by two raised to its position, the products are summed and divided by 11, and the remainder is the check digit, with 10 counting as 0. The reference example CSQU3054383 passes: the products total 6,185, and 6,185 divided by 11 leaves 3. A misread digit almost always breaks the check, so it catches most OCR slips for free.

Air waybill numbers are 11 digits: a three-digit airline prefix, a seven-digit serial number and a check digit, which is the remainder when the serial is divided by 7. For serial 8114074 the remainder is 3, so the number ends 81140743. Both checks are a few lines of code or a spreadsheet formula, and neither needs AI.

A prompt for extraction, with a sample result and what to fix

For the test you can use a chat assistant with file upload, or a dedicated extraction tool. The prompt below works for either. It forces the model to say when it is unsure instead of guessing.

Extract the fields below from the attached shipping document. Return JSON only.
For each field return {"value": ..., "page": ..., "confidence": "high|medium|low"}.
If a field is not on the document, return null. Never infer a value that is not printed.
Keep consignee and notify party separate. Copy weights exactly with their printed unit.
List every container number with its seal number and package count.

Fields: document_type (master B/L, house B/L, sea waybill, MAWB, HAWB), document_number, shipper, consignee, notify_party, port_of_loading, port_of_discharge, vessel_or_flight, on_board_date, containers[], gross_weight, volume, freight_terms, incoterm.

An illustrative extract of what comes back for one document:

{
 "document_type": {"value": "house B/L", "page": 1, "confidence": "high"},
 "consignee": {"value": "Example Trading GmbH", "page": 1, "confidence": "high"},
 "notify_party": {"value": null, "page": 1, "confidence": "low"},
 "containers": [
   {"number": "CSQU3054383", "seal": "A1234567", "packages": 640},
   {"number": "CSQU3054384", "seal": "A1234568", "packages": 620}
 ],
 "gross_weight": {"value": "21,400", "page": 1, "confidence": "medium"}
}

Three things to notice. First, the notify party came back null with low confidence: the document says "SAME AS CONSIGNEE", which should be recorded as such, not left blank. That is a rule for the field map. Second, the weight carries no unit, so it cannot be used. Third, the second container number should be validated. Running the check digit on CSQU3054384 fails, so one digit was misread (or the document itself is wrong). Without the validation rule, this reached the TMS looking perfectly normal.

The 100-document accuracy test

Build the test set to look like your real workload, not your best documents. A split for a small forwarder (adjust to your mix):

• 40 clean digital PDFs straight from carriers
• 30 scanned documents of ordinary quality
• 15 poor ones: skewed, faint, faxed or photographed on a phone
• 10 multi-page documents with continuation sheets or long container lists
• 5 with handwritten amendments or stamps over the text

Include a spread of carriers, at least one air waybill batch if you handle air freight, and both master and house documents. Remove nothing because it looks hard. Those are the ones that will arrive on a busy Friday.

Build the answer key first

Two people key the 100 documents into a sheet independently. Where they disagree, a third settles it from the paper. This costs roughly two to three minutes a document each, so allow about eight hours of staff time between them. It is the most valuable part of the exercise: you now know the human error rate too, which sets a fair bar for the tool. Count every field where the two keyers disagreed; that count, not a vendor's figure, is the baseline the tool has to beat.

Score three things, not one

1. Field accuracy. The share of each field that matches the answer key exactly. Count formatting differences you can fix by rule (dates, units) separately from real misreads.
2. Document accuracy. The share of documents where every critical field is right. This is the number that predicts how much correction work is left.
3. Silent errors. Wrong values that pass validation with high confidence. A wrong value that failed a check is routed to a person and costs a few minutes. A silent error reaches the cargo, the invoice or a customs entry. Hunt for these deliberately.

An illustrative result

The figures below are made up to show how to read a score sheet; they are not from a real run. Each row adds up to 100 documents.

Reading this the way an owner should: container numbers, ports and freight terms are ready to automate with validation, because every error was flagged and none slipped through. Notify party needs a rule. Weight is the problem field: two silent errors in 100 documents is not acceptable when weight drives freight and safety. The fix is a unit rule (reject any weight with no printed unit) and a cross-check that the total equals the sum of the container weights, plus a plausibility cap per container type.

Awkward documents and how to treat each

The 15 poor documents and the five with handwriting are where a tool earns or loses trust. A few specific cases and what to do with each, so your rules are ready before the first live batch:

• Continuation pages. A bill listing 38 containers over three pages. Ask the tool to return a container count and compare it with the count printed on page one. A mismatch goes to the queue. Do not let it merge lists from two different bills in one PDF.
• Weights in mixed units. A document with the cargo weight in kilograms and the container tare in pounds. Extract the printed unit with every number and convert by a fixed rule in your system, never in the model's head. A kilogram figure near 2.2 times the previous shipment of the same cargo is a warning.
• Stamps over text. A "released" stamp covering the weight box. The tool will often read the stamp or guess. Treat any field with a stamp overlap as low confidence by rule if the tool reports it, or have the reviewer look at every document that carries a stamp.
• Handwritten corrections. A crossed-out consignee with a new name written beside it. Ask for both values and the page, and send the record to the queue. These are legitimate amendments and a person must decide which one stands.
• Several languages on one form. Boxes labelled in two languages are normally fine, but descriptions of goods in another language should be copied verbatim, with any translation in a separate field marked as a translation.
• Phone photos. Perspective and shadows cut accuracy sharply. A simple rule such as "ask the sender for a scan" avoids many exceptions. Add it to your onboarding note for agents.

Write each case on the test log with the document number. After a month, the same eight or ten patterns will account for most exceptions, and that list is your improvement plan.

The exceptions queue is half the system

You will not get to 100 percent, and you should not try. Design for a queue of documents that a person reviews, and make that queue pleasant to work in.

• Route by reason. Check digit failed, field missing, low confidence, customer not recognised, weight total mismatch, duplicate document number. Each reason tells the reviewer where to look.
• Show the page next to the value. The page reference in the prompt above lets the reviewer see the source in one click.
• One fix feeds the rules. When the same correction recurs, such as a carrier writing weights in a different layout, add a rule or a hint to the prompt.
• Sample the passes. Even documents that pass everything get a spot check, say 1 in 20, so silent errors are caught early. Raise or lower the rate as your data shows.
• Set a target time. A queue nobody owns becomes a backlog. Decide who clears it and by when.

In the illustrative scores above, the rules raised 34 flags across the seven fields. Some documents carry more than one flag, so at most 34 documents go to a person at the start; plan on about a third. That still leaves roughly two-thirds untouched, and the reviewers handle the flagged ones in a minute each instead of four, because the flag tells them which box to look at. As the rules improve, the queue shrinks. The wider principle of keeping the checking step light but real is in setting up human review for AI work without slowing down.

Putting results into the TMS safely

Treat the TMS as the system of record that the AI feeds, never as a notepad. A safe rollout in four steps:

1. Staging first. The extraction writes to a staging sheet or draft record, not the live shipment. Staff approve each record.
2. Shadow mode for four weeks. Staff key documents as usual and the tool runs beside them. Compare daily. You are measuring error rates on live traffic, not only on the test set.
3. Let clean documents through. Once a document type has run clean for four weeks, allow records that pass every rule and have high confidence to post automatically, with a daily sample check.
4. Retest regularly. Rerun a 20-document sample monthly, and rerun the full test whenever a new carrier, form version or tool version arrives.

Keep the original document attached to every record. If a customer disputes a weight six months later, the scan is the evidence, and it is also what a damage claim evidence pack will need.

A mix-up that passed every check

Here is a made-up but realistic case. A small forwarder handling house bills ran extraction on a batch from one overseas agent. Every container number validated, weights summed correctly and ports matched. A week later a customer complained that arrival notices were going to the wrong company.

The cause: this agent's template put "Notify party" in the box where most forms print "Consignee". The model followed the printed label, which was correct for the form, but the agent had filled the boxes the other way round. The data was internally consistent and wrong. No format rule could catch it. A customer-master check could: the "consignee" returned was a company the forwarder knew as a notify-party-only contact for another client.

The fix was two rules. A new agent's documents go to human review for the first 20 documents, and any consignee that the master lists only as a notify party triggers a flag. The lesson for your own test is to include documents from every agent and carrier you deal with, because layout quirks live there.

Choosing a tool for the test

You can start the experiment with a general chatbot that accepts files, which is enough to test the concept on 100 documents. For production you will likely want something that handles volume, keeps an audit trail and connects to the TMS, whether a dedicated document-extraction product or a workflow you build around an AI model's API. Do not take a tool's accuracy claims as your figures. Ask for a trial and run your 100 documents through it using the same scoring. For background on how modern AI extraction differs from the older template OCR, see AI document extraction vs traditional OCR, and for a simpler version of the same exercise on supplier bills, AI invoice processing.

A few buying questions that matter more than the headline accuracy figure:

• Does it return a confidence level and a page reference for every field?
• Can you add your own validation rules and customer master?
• Where do the documents go, how long are they kept, and are they used for training?
• Can it export to your TMS format, and does it keep the source document with each record?
• What happens when it fails: does it say so, or return a guess?

Time and cost of the whole exercise

For a small forwarder, the first pass looks like this (illustrative):

That is two to three working days of mixed staff time, spread across the team, plus a month of a paid chat plan or a vendor's free trial for the tool itself. The answer is worth far more than the cost: a figure for your own documents, the list of rules to write and a clear picture of what will always go to a person.

The same approach works on rate sheets, where the layouts are different but the test is identical; extracting carrier rates from PDFs covers that case, and tracking demurrage and detention free days shows what the extracted dates can feed once they are reliable.

If you would like to plan the test and the exceptions queue with someone, that is exactly what an AI implementation consultation covers; you can book one here.

Questions forwarders ask about document extraction

How accurate does extraction need to be?

Set the bar per field. For fields that move money or cargo, such as container number, weights, consignee and ports, aim for at least 99 percent after validation, with anything doubtful routed to a person. Lower-risk fields such as remarks can tolerate more errors.

Why test on my own documents?

Accuracy depends on your carriers, layouts, scan quality and languages. A vendor's average figure says little about your mix of clean PDFs, faxed copies and phone photos. A hundred of your own documents give you a figure you can plan around.

Should AI write directly into the TMS?

Not at first. Have it fill a draft or staging record that a person approves, and run in shadow mode for several weeks beside manual entry. Once error rates are known and validation is in place, let clean documents through and keep a sample check.

What is the most dangerous kind of error?

A wrong value that passes every validation rule, such as a swapped consignee and notify party, or weights in pounds read as kilograms. Count these separately in your test, because they are the ones nobody sees until the cargo or the invoice is wrong.

Is it safe to upload shipping documents to a chatbot?

They contain customer names, addresses and sometimes prices. Check your plan's data settings and any customer confidentiality terms first. Business plans and dedicated document tools generally offer stricter handling than free consumer chat; confirm in writing before uploading real documents.

Further reads

• Freight Forwarder Lead Generation: AI Finds Shippers, Drafts Emails — Use AI to find new shippers once your paperwork is under control.
• AI for Freight Business Analytics: Margin by Lane From a Spreadsheet — Turn clean shipment data into margin by lane.
• AI Freight Quoting: Turn Customer Emails Into Draft Quotes — Turn customer emails into draft freight quotes with checks.
• Carrier Vetting Checklist: AI Checks With a Human Sign-Off — A checklist with human sign-off for vetting carriers.
• Carrier Packet Onboarding: AI Collects and Checks the Documents — Another document-heavy job where AI collects and checks.
• How to Use AI in a Trucking Company: Load Paperwork Without Retyping — A step-by-step way to turn rate confirmation PDFs and broker emails into reviewed load records with ChatGPT or Claude and a two-tab sheet.
• AI for Freight Brokers: Triage Load Emails and Rate Confirmations — A broker's review queue for load emails and rate confirmations, with field-level checks before anyone signs.
• AI for Freight Business Sustainability: Emissions Figures for Tenders — A haulier's tender sheet separates measured fuel, allocation, factor source and the emissions figure a customer requested.
• Freight Invoice Audit: AI Checks Carrier Bills Against Quotes — Turn a carrier quote and invoice into two tables, let AI list every difference, sort the accessorial charges and send the dispute from a weekly routine.
• AI in the Trucking Industry: What a Small Fleet Owner Can Use Now — Dispatch messages, load paperwork entry and driver FAQs for a small fleet: what AI handles today, with a worked time sum and the pitfalls.
• AI Tools and AI Development: The Complete 2026 Guide — the AI hub, including every tutorial in the AI-for-business series.

Sources: the ISO 6346 container number structure and check-digit method (public reference material, worked example CSQU3054383); the air waybill number structure and the master and house waybill distinction (public reference material); ICC's Incoterms page (Incoterms 2020 is the current edition); UN/LOCODE structure. No accuracy test was run for this tutorial: every accuracy figure is illustrative.