01The problem with asking a model
Ask a chatbot the same question twice and it may take two different approaches. That's fine for a first draft. It's a problem when the answer is a measurement, a deadline, or a number someone has to sign off.
02What we do instead
So we don't let the AI decide how the work gets done. We sit down with the people who do it and write down their steps, settings and rules. Each step becomes a tool that works the same way every time. The AI only picks the tool and fills in the details. If a request breaks the rules, it's stopped before it runs. In our October test, 22 of 30 rule-breaking requests were stopped before they ran; we are closing the 8 that got through.
A tool here is a program that does one step of the method the way the expert does it: the same inputs in, the same calculation, the same checks, the same result out. It doesn't improvise, and it doesn't get clever. The model decides which tool and which details; the tool does the work. When a better open model appears, we swap it in and your tools stay as they are.
03What you see when it runs
Every result shows where it came from: which data, which step, which rule. The report you get from us does the same.
04How we know it works
Each test uses one real input, the same plain-English request for every tool, and many runs, so a lucky answer can't pass as a result. Every run is on the proof page.
First test, May 2026
We took one real measurement from the lab and gave it to our tool and to three leading AI platforms, all on paid plans with every feature turned on. Each one got the same file and the same request, four times.
| Runs | Same answer | What happened | |
|---|---|---|---|
| Our tool | 4 | 4 of 4 | The same result in all four runs, 34 seconds each. |
| GPT-5.5 | 4 | 0 of 4 | Saw one peak in some runs and two in others. Its results moved by up to 54.8% from run to run. |
| Claude Sonnet 4.6 | 4 | 0 of 4 | Measured the wrong feature in most runs, and drew its boundaries differently each time. |
| Gemini 3.1 Pro | 0 of 4 usable | 0 of 4 | Got a unit conversion wrong every time, so none of its results were usable. |
Re-test, October 2026
The same kind of test on today's tools, with many more runs, and a second instrument. On today's tools the picture is the same as in May: the cloud models gave a different exponent almost every run; our tool gave one.
| Runs | Usable | Same answer | What happened | |
|---|---|---|---|---|
| Our tool aloneno language model in the loop | 100 | 100 | 100 of 100 | One value every time, 5 s per run. On a second researcher's file: 100 of 100 after a fix on 18 Oct (98 of 100 before it). |
| Omicron with our toolthrough the chat | 100 | 94 | 93 of 94 | The tool returned one value; the model wrote it to four decimals in 72 runs and rounded it in 21. 6 runs stopped before the fit; 1 run gave another value while our own fault test was hitting the same service. |
| Claude Opus 5.5agent with a shell, paid plan | 10 | 10 | 4 different values | 0.988–1.003; wrote and ran its own fitting code each time. |
| Claude Sonnet 5.5agent with a shell, paid plan | 30 | 30 | 19 different values | 0.97–1.03; wrote and ran its own fitting code each time. |
| GPT-6 Lunavendor's command-line tool, high reasoning | 30 | 30 | 20 different values | 0.69–0.93. |
| GPT-6 Lunavendor's command-line tool, vendor default | 30 | 29 | 20 different values | 0.73–1.21; one run gave no result. |
Second instrument: electron-microscope image, dominant period: our typed function gave the same value in both reference runs; GPT-6 Luna (default) gave 9 different values in 30 runs, 26 inside the accepted range and one answering 57 nm; Claude Sonnet 5.5 (agent) 11 different values, 28 inside.
The privacy test, October 2026
When the data may not leave, the model that stays is the one that can answer.
Many files can't be sent to an outside AI service as they are. A client's identity is covered by a lawyer's duty of secrecy, personal data by the GDPR, a supplier's prices by a trade-secret clause. So we used three invented files — a civil case, a lab measurement campaign, a supply contract — with 20 questions each. The outside tools saw the files with the protected parts hidden; Omicron, running on our machines, saw all of it. We compared against the model each vendor sells most of; on the legal file we also ran each vendor's top model, so nobody can say we picked a weak opponent. On the twelve questions that depend on the hidden parts, no outside tool answered a single one, and every one said so instead of guessing. Omicron answered 97 % of the questions on the legal file and all of them on the other two. The full table is on the proof page.
05What the tests measure — and what they don't
These tests measure consistency, and what each tool can answer with the data it is allowed to see. They do not show that our system is more accurate in general. The paper doesn't claim these platforms can't give consistent results; only that they didn't, on this task, in these runs. Model names and plans change often, and the dates are the only ones these results speak for.
Choosing the right approach is still the expert's job, so we build theirs in.