OMNI: Four Frontier AI Systems, One Problem, Two Invented Numbers

Building a system that makes models correct each other — and knowing when to stop paying for it

Brian James Curry | Vector1 Research

Research Note: This article describes a working, provider-agnostic prototype and a four-response illustrative pilot. The pilot is not a statistically powered leaderboard. The systems are anonymized, only one response time was captured, and the reported scores are human rubric scores supported by deterministic checks. No claim is made that the prototype has established a universal ranking of frontier models. Code and the full pilot transcript: GitHub — omni-intelligence.

I gave the four top frontier lab models the same question, scored them and the results led me to create this omni model research framework

• The first answer scored 95.
• The second scored 89.
• The third scored 71.
• The fourth scored 55.

All four answers addressed the same synthetic business problem. All four recommended a marketing budget totaling one million dollars. All four sounded confident, analytical, and professionally written.

Two of them also manufactured numbers that were not in the evidence.

That was the interesting result.

The difference between the strongest and weakest systems was not fluency. It was epistemic discipline: the ability to recognize what the data did not establish, resist false precision, and still make a useful decision.

It led me to a different way of thinking about frontier AI.

The question may not be which model is best. The more useful question may be how to combine models so that one system’s strengths expose another system’s errors — and how to keep doing that until the collective result becomes better than any initial answer.

I built a prototype around that idea.

I call it OMNI: Orchestrated Multi-Model Networked Inference.

OMNI sends the same task to multiple frontier models simultaneously. Their answers flow into a downstream evaluation layer. That layer checks constraints, scores the answers, identifies unsupported claims, and sends a common feedback packet back to the frontier models. Each model revises its answer independently. The loop continues until the system reaches verified convergence, stops improving, exceeds its cost ceiling, or reaches its maximum number of rounds.

This is not a chatbot council taking a majority vote. It is a recursive optimization system for model outputs.

The Experiment That Exposed the Problem

I gave four AI systems a small, highly constrained decision problem. Here is the exact prompt they received:

The Prompt Given to the Frontier Models:

• Context: You are advising a company that must reallocate a $1 million quarterly marketing budget across three channels:
• Paid Search: Current Spend: $400,000 | Reported Revenue: $1,600,000 | Experiment Result: A 10% holdout reduced revenue by 4% | CAC: $80
• Paid Social: Current Spend: $350,000 | Reported Revenue: $1,050,000 | Experiment Result: No controlled experiment | CAC: $105
• Email/CRM: Current Spend: $250,000 | Reported Revenue: $1,250,000 | Experiment Result: A randomized test showed a 12% incremental lift | CAC: $25
• Additional Information: Paid-search revenue includes customers who may have purchased without advertising. Paid social reaches more new customers than other channels. Email has limited additional audience capacity. The CFO wants immediate return. The CMO wants customer growth without materially reducing quarterly revenue. The total budget cannot change.
• Constraints: Provide a specific dollar allocation totaling exactly $1 million. Distinguish reported revenue from incremental impact. Explain what can and cannot be concluded. Identify three key uncertainties. Propose one experiment. Keep the response under 500 words. Do not invent missing data; if an optimal allocation cannot be calculated, state that directly and make a defensible provisional recommendation.

Why This Setup Works as a Test

This problem is deliberately underdetermined. It tests whether a model can:

• Perform basic calculations and satisfy hard constraints.
• Recognize that reported revenue is not necessarily causal revenue.
• Avoid treating incomplete data as conclusive.
• Balance conflicting executive objectives (CFO vs. CMO) under strict resource limits.
• Produce an actionable answer despite ambiguity without manufacturing missing metrics.

The strongest response explicitly refused to convert the search holdout into a marginal return because the prompt did not say exactly how the holdout had been constructed or which revenue base the four-percent decline described. It made a conservative recommendation, stated its uncertainty, and proposed a better experiment.

The weaker responses sounded more quantitative.

One assumed that a 10-percent holdout meant exactly 10 percent of search spending had been removed. It converted that assumption into a $40,000 reduction, applied the four-percent revenue decline to the reported $1.6 million, calculated a $64,000 loss, and declared a 1.6-times marginal ROI.

None of those measurement details had been provided.

Another response repeated that calculation and then applied the email experiment’s 12-percent lift to an unspecified revenue base, producing an estimated $130,000 to $150,000 in incremental revenue. It sounded like the most technically sophisticated answer. It received the lowest score.

The polished language made the unsupported inference harder to notice, not less consequential.

That is the central problem OMNI is designed to address.

Why One Frontier Model Is Not Always Enough

Frontier models do not fail identically.

One may be better at causal reasoning. Another may follow formatting constraints more reliably. Another may catch mathematical inconsistencies. A fourth may produce the clearest executive recommendation. Even two models with similar benchmark performance may approach the same ambiguous problem from different directions.

Research on model ensembles supports the broader proposition that multiple language models can outperform an isolated model under the right orchestration scheme:

• Mixture-of-Agents uses layered model outputs as inputs to later agents and reported improvements on major language-model evaluation benchmarks [1].
• LLM-Blender combines pairwise ranking with generative fusion rather than selecting a single answer [2].
• Multi-agent debate and ReConcile use structured interaction to improve reasoning and factual performance [3][4].
• FrugalGPT demonstrates that model routing and cascades can improve the relationship between cost and accuracy rather than using the largest model for every request [5].

The production literature is moving in the same direction. Anthropic describes evaluator-optimizer workflows in which one model generates and another evaluates in a feedback loop [6]. OpenAI distinguishes manager-style orchestration from decentralized handoffs and recommends optimizing model selection only after the required accuracy level has been reached [7].

However, more models do not automatically create more intelligence. They can also produce shared hallucinations, defer to the most confident answer, amplify an early framing error, or converge because every model was trained on similar material. Model judges introduce their own problems, including position bias and self-preference [8][9].

The system therefore needs more than plurality. It needs architecture.

The OMNI Architecture

OMNI divides the problem into eight components:

1. The Task Contract

Every run begins with a machine-readable contract containing the complete prompt, allowed evidence, required output structure, hard constraints, scoring rubrics, explicit penalties for unsupported claims, and cost/latency/iteration limits.

Note: The contract matters because a vague objective produces a vague optimization target. If the system is told only to make the answer “better,” the downstream evaluator can reward verbosity, confidence, or stylistic similarity instead of decision quality.

2. Simultaneous Frontier Fan-Out

The same task goes to several frontier models at the same time. Initial answers are independent — models do not see one another’s work during the first round. This protects diversity and prevents anchoring. The prototype includes provider adapters for OpenAI-compatible APIs, Anthropic, and Gemini.

3. Deterministic Validation

Before another model expresses an opinion, ordinary code checks everything that ordinary code can check (e.g., matching allocation totals, word counts, and known unsupported numerical inferences).

Crucial Distinction: Unlike standard Mixture-of-Agents architectures where model opinions flow entirely into other models, OMNI uses a deterministic-first validation gate. Language models should not be asked to approximate arithmetic or compliance checks that software can perform exactly. If code catches a fatal constraint violation, it cannot be “voted away” by peer consensus.

4. Blinded Cross-Evaluation

Candidates are anonymized and scored by one or more evaluator models. Judges do not see model identities, candidate order is randomized for each judge, and scores are bounded by the task rubric. Multiple evaluators reduce — but do not eliminate — judge bias.

5. The Downstream Arbiter

Candidate outputs, deterministic findings, and blinded critiques flow into a downstream arbiter that produces verified common ground, unsupported claims, unresolved disagreements, and directives for the next round.

6. Recursive Revision

Each frontier model receives the original task, its own previous response, the anonymized peer responses, and the feedback packet to revise in parallel. For model i at iteration r:

Answer(i,r+1)=Model(i)[Task,Answer(i,r),PeerAnswers(r),ArbiterFeedback(r)]

7. Verified Convergence

OMNI treats convergence as a compound condition. A result is accepted only when hard validators pass, the best rubric score exceeds a threshold, judge disagreement falls below limits, outputs converge, and no unresolved fatal error remains. Semantic agreement cannot override a failed calculation.

8. Final Synthesis and Audit

After the loop stops, the system synthesizes the strongest supported elements into a final answer, which passes through the validators once more before human sign-off.

What Exactly Is Being Optimized?

A multi-model system should not maximize answer quality without regard to cost. Let the utility of iteration r be modeled as:

Utility(r)=Quality(r)−λCost(r)−μLatency(r)−γResidualRisk(r)

• Quality comes from the fixed rubric.
• Cost includes model calls and tokens.
• Latency is measured directly.
• Residual Risk includes validator failures, judge disagreement, unsupported claims, and unresolved evidence gaps.

The coefficients λ, μ, and γ adapt to the use case. For instance, in a high-stakes financial audit workflow, γ (Residual Risk) dominates the equation, forcing the system to halt or escalate to a human immediately if a single deterministic check fails, regardless of model generation costs.

OMNI stops for one of four reasons:

1. Verified convergence: quality and validation conditions are fully satisfied.
2. Quality plateau: another round produces less improvement than the configured minimum threshold.
3. Cost ceiling: expected model expense reaches the run budget.
4. Maximum rounds: the system hits its hard iteration limit.

The Economics of Intelligent Systems

The architecture is more expensive than a single prompt. With N frontier solvers and J judges, one evaluated round requires approximately:

Solver Calls+(Solver Calls×Judge Calls)+Arbitration Call→N+(N×J)+1

That makes intelligent routing essential across three operational modes:

• Fast Mode: One efficient model answers first; the system escalates only if confidence is low or a validator fails.
• Research Mode: Several frontier models run simultaneously through multiple evaluation and revision rounds for strategic decisions.
• High-Stakes Mode: Full mesh deployment with source verification, strict domain validators, and mandatory human approval.

What the Prototype Does Today

The version accompanying this article is a lightweight, GitHub-ready Python prototype available at GitHub — omni-intelligence with zero required third-party package dependencies.

It includes:

• Asynchronous parallel calls to multiple model providers.
• Adapters for OpenAI-compatible APIs, Anthropic, and Gemini.
• A built-in replay mode for evaluating saved responses without live API access.
• Machine-readable task contracts and scoring rubrics.
• Deterministic word-count and allocation-total validators.
• Configurable risk detectors, blinded judging, and feedback memos.
• Full unit test suite and the four-response pilot transcript.

The local replay reproduces the pilot ranking of 95, 89, 71, and 55, successfully flagging the unsupported search calculation in the third response while leaving the disciplined first response unflagged.

To validate this design objectively, future empirical work must run systematic ablations comparing:

• Best single model versus the full mesh.
• Same-model self-consistency versus cross-lab diversity.
• Majority vote versus blinded scoring.
• One-pass synthesis versus recursive feedback.
• Model-only evaluation versus model plus deterministic validation.
• Fixed three-round loops versus adaptive stopping.

You can inspect the codebase, review the task contract, test the replay harness, or run the pilot locally via the repository: github.com/Bodhi8/omni-intelligence. Corrections, issues, and failed replications are welcome.

References

[1] Wang, J., et al. “Mixture-of-Agents Enhances Large Language Model Capabilities.” 2024. arXiv:2406.04692

[2] Jiang, D., et al. “LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion.” ACL, 2023. ACL Anthology

[3] Du, Y., et al. “Improving Factuality and Reasoning in Language Models through Multiagent Debate.” 2023. arXiv:2305.14325

[4] Chen, J. C.-Y., et al. “ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs.” ACL, 2024. arXiv:2309.13007

[5] Chen, L., et al. “FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance.” 2023. arXiv:2305.05176

[6] Anthropic. “Building Effective AI Agents.” 2024. Anthropic Engineering

[7] OpenAI. “A Practical Guide to Building AI Agents.” OpenAI Guides

[8] Shi, L., et al. “Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge.” 2024. arXiv:2406.07791

[9] Wataoka, K., et al. “Self-Preference Bias in LLM-as-a-Judge.” 2024. arXiv:2410.21819

About the Author

Brian Curry is a Kansas City-based machine learning and AI researcher, data scientist, and founder of Vector1 Research. His work focuses on causal inference, complex-system optimization, agentic orchestration, cognitive AI architecture, and the economics of intelligent systems. He is the creator of Papilon, PyCausalSim, Memory-Node Encapsulation, and a growing body of open-source research on how AI systems reason, remember, coordinate, and change organizations.