Here is the number that changed how I think about AI accuracy: one in 870.
In an early test, a single AI read of a parts list got one field wrong in every 870. Most people would celebrate that. I nearly did.
Then I looked at what the one error actually was. It was a value read wrongly, with complete confidence, in a perfectly clean format. Nothing about it looked suspicious. There was no warning, no hesitation, no "I'm not sure about this one." It sat in the table looking exactly like the 869 correct values around it.
In a fabrication shop, one wrong field in a parts list is a wrong plate thickness, a wrong quantity of tubes or a wrong flange rating. It gets discovered when the material arrives, or worse, when the inspector does. So the question in front of us was never "how accurate is the model?" It was a harder one: "how will we know when it's wrong?"
Accuracy is a statistic. Trust is something else.
Across our five real drawings, the parts lists alone held 363 rows and 1,815 individual fields: part number, description, quantity, material, size and remarks. Three of those drawings had no real text at all, so every field had to be read from a picture.
A reader that's 99.9 percent accurate would still leave one or two silent errors in that set. And here's the uncomfortable part: the engineer wouldn't know which ones. An error rate tells you how many mistakes exist. It doesn't tell you where they are. Trust needs the second thing.
Two readers who never compare notes
The answer wasn't new. Accountants use it. Pilots use it. Pharmacists use it. Two independent checks, then compare.
So RevE reads every parts list twice, and we worked hard to make the two reads genuinely independent. The second read looks at the drawing at a different resolution. The rows are grouped differently, so a row that sat at the bottom of a batch the first time sits in the middle the second time. Neither read ever sees the other's answer. Then plain code compares them, field by field.
Where both reads agree, the value is trusted. Where they disagree, it goes to a person, with both versions side by side.
And where a drawing has real text, we don't "read" at all. Code simply copies it, and the AI is only asked about the handful of cells the text layer misses.
The result across all five drawings: 1,815 of 1,815 parts-list fields matched the checked reference. Two rows needed a person to look at the picture and decide.
The most interesting finding wasn't the score
It was the pattern of the mistakes.
On the notes and title blocks, both reads made errors. One misread a number badly. Both, at different times, "helpfully" corrected spelling mistakes that were actually printed on the drawing. METALIC became METALLIC. EUIPMENT became EQUIPMENT. They did this even though the instruction said to copy exactly.
That's a very human-looking mistake, and a very dangerous one. A fabrication drawing is a contract document, typos and all. A system that silently tidies it up is changing a legal record without telling anyone.
But across the entire set, the two reads never made the same mistake at the same place. Every error showed up as a disagreement. Every disagreement went to a person. Nothing slipped through quietly.
That's the real lesson. You don't need a perfect reader. You need two imperfect readers whose mistakes rarely line up, plus a cheap, exact comparison. In my steel-plant days we never trusted a single thermocouple for a critical temperature. We used two and alarmed on disagreement. It turns out AI deserves exactly the same respect.
Confidence has to be earned, not claimed
AI models will happily tell you how confident they are. We decided not to listen.
In RevE, confidence follows simple rules applied by code:
- A value read from a picture can never be marked 100 percent certain.
- Agreement between two independent reads is what raises confidence. The model's opinion of itself never does.
- A value is only as confident as the weakest line it came from.
- Anything low-confidence must carry a short note saying why.
This matters because the app shows confidence everywhere, as small green, amber and red tags. If those colours came from the model's own self-assessment, they'd be decoration. Because they come from agreement and rules, they mean something. An engineer can learn within a week to trust green and look at amber.
The day the machine audited the human
There was a second rule that sounded almost stubborn: every value is copied exactly as printed, with its printed unit, and code checks that the text really exists on the drawing. No converting, no rounding, no tidying.
Before we started, a careful engineer had built a reference catalogue of all the values on the five drawings by hand. We used it as our answer key. As RevE matured, it began disagreeing with the answer key, and when we went back to the drawings, RevE was usually right.
The human catalogue had copied some values from a neighbouring drawing. It had converted units for convenience and rounded a seismic factor. It had 31 part numbers that didn't match, a baffle count one short, and a tubesheet diameter taken from the wrong exchanger.
I want to be careful here, because this is not an "AI beats engineers" story. The person who built that catalogue was doing a different job, summarising quickly across drawings. RevE was doing a narrower job, slowly and literally. The lesson is that literal copying with verification is a discipline, and machines are very good at disciplines. The engineer, to his credit, was grateful.
A judge, not a typist
None of this removes the need for a person. The question is what that person should do.
The worst design asks the engineer to check everything. They won't, and you can't blame them. After twenty correct rows anyone starts skimming, and the twenty-first row is where the error lives. The second-worst design hides uncertainty and presents a clean table. That just moves the checking to the shop floor.
RevE's "To review" list contains only what has earned doubt: disagreements between reads, low-confidence values, balloons that don't tie cleanly to a parts-list row. On a large drawing that's typically 80 to 115 items, out of thousands of values. Each item shows a picture of the exact spot on the sheet. The engineer has three buttons: *correct as read*, *save the fix* or *not relevant*.
A few small design choices made a big difference:
- Decisions never overwrite what the AI read. They're stored as a separate layer, so you can always see what the machine read and what the person decided.
- Decisions go stale. If the drawing is read again and a value changes, the old decision reopens.
- Where possible, the answer is constrained. For a balloon, the engineer picks from a list of real part numbers, not a blank box.
- Everything is exportable. The full review record goes to Excel, which is exactly what a quality manager wants to see.
On a large drawing, settling all the doubtful items takes a focused half hour or so. Compare that with a full manual take-off, or with a silent error discovered weeks later.
Six rules I'd copy for any AI reading job
If you're using AI to read invoices, test certificates, inspection reports or contracts, this is the pattern I'd borrow:
1. Read twice, independently. Change the crop, the resolution or the grouping. Never let the second read see the first.
2. Compare in code, exactly.
3. Copy verbatim, then check it exists.
4. Earn confidence from agreement. Ignore the model's opinion of itself.
5. Send only disagreements to people, with the picture and simple choices.
6. Keep human decisions as a layer. Never overwrite the machine's output.
When people ask me how accurate RevE is, I now give two answers. The first is the number: 1,815 out of 1,815. The second matters more: “we know which values to doubt, we show them to a person with the evidence, and we keep a record of what they decided.”
Accuracy is a statistic. Trust is a process. Build the process.
Next, in Part 3: reading a drawing correctly is not the same as the drawing being correct. What happens when a sheet disagrees with itself?
#IndustrialAI #QualityAssurance #HumanInTheLoop #AIAccuracy #Manufacturing #EngineeringDrawings #TrustworthyAI #DocumentAI #Fabrication #SandurTech
Disclaimer
This series introduces SandurTech RevE, a product in development for AI-assisted engineering drawing analysis and generation. Examples reflect our current capabilities and ongoing validation. Outputs should be reviewed by qualified engineers and verified against approved drawings, standards, and project requirements before use.
