You keep hearing that the “harness,” not the model, determines how an AI system behaves. True, but it’s a term that gets used without much explanation of what it’s actually made of.
The short version: six components, each doing a distinct job. Get the right one doing the right work, and problems that look like they need a smarter model often just need better architecture.
1. The Model Itself
The model is the reasoning engine, not the guarantee of correctness. It interprets a request and generates a response drawing on patterns learned in training, including, for Scripture-related requests, basic interpretation like identifying which passage a question is about. What it’s not responsible for, in a well-built harness, is guaranteeing its output is exactly correct. It doesn’t believe anything, doesn’t care about getting a quotation right, and doesn’t decide to double-check itself. It runs whatever harness it’s given, which is why the other five components exist.
2. Guardrail Gates
Guardrails catch a request or response before it causes a problem, at the input or output stage. A well-known real example: several major AI providers now refuse to reproduce more than a set number of words of copyrighted text in one response, because unconstrained generation of licensed text is a real liability. Guardrails are binary, catching or not, which is both their strength and their limit. They’re a safety net, not a substitute for the underlying architecture.
3. System and User Instructions
Baseline instructions define general behavior; instructions layered on top define product-specific rules like tone and how to handle particular content; the user’s own request shapes the specific interaction, as long as it doesn’t contradict what’s above it. For Scripture specifically, this is where you’d specify that biblical text is a distinct category the model shouldn’t attempt to generate directly.
4. Queried Context Versus Tool Calls
RAG hands the model information before it starts generating; tool calls let the model decide, mid-response, whether to fetch information itself. That timing difference matters: a tool call depends on the model reliably choosing to use it, a decision point RAG doesn’t have. Testing has shown models sometimes skip an available tool call entirely, judging (wrongly) that answering from memory is more efficient.
5. Output Transforms
This is the component most people underestimate: a deterministic process that modifies the model’s output after it’s generated but before a user sees it. A simple example is auto-linking a plain URL. A more consequential one, directly relevant to Scripture: instructing the model to output only a placeholder reference, then swapping in the actual licensed text from a reliable source. Because this happens entirely outside the model’s generation process, whatever unpredictability exists in how the model writes simply doesn’t apply to that final piece of output.
How the Five Pieces Fit Together
Roughly in order: input guardrail, retrieved context, assembled instructions, model generation (with tool calls along the way), output guardrail, then output transforms, before anything reaches the user. Each stage is a place something can go right or wrong, which is why “the harness” isn’t one setting, it’s a full pipeline, and it’s only as strong as its weakest link. For Scripture quotation specifically, this is why moving the verse text out of generation entirely and into a deterministic output transform produces such a dramatic accuracy jump. The model didn’t get better. One component in the pipeline took over a job the model was never going to do reliably.
FAQ
What is an AI “harness” in simple terms? Everything built around a model that shapes its behavior: safety checks, instructions, data lookups, tool access, and output processing, distinct from the model’s own reasoning.
What’s the difference between RAG and tool calls? RAG injects information before generation; tool calls let the model choose to fetch information during generation, a decision point RAG doesn’t require.
What is an output transform? A deterministic process that modifies a response after generation, before a user sees it, like swapping a placeholder Scripture reference for the real, licensed text.
Why don’t guardrails alone solve reliability problems? They catch problems after the fact; they don’t improve how reliably the model generates correct content in the first place.
Does a better model reduce the need for a well-built harness? Not meaningfully for verbatim reproduction tasks. Current-generation models are broadly capable enough that the harness around them, not the model, decides reliability.
Conclusion
“The harness matters more than the model” is a claim about six specific components: model, guardrails, instructions, retrieved context, tool calls, output transforms. Read the full study for a worked example, where one component swap took Scripture quotation accuracy from 32.6% to 100%.