Skip to main content
The Davenporter
Essays Builds Now About
Theme
Palette
Jason Davenport

I write about agents doing real engineering work: source control, memory, telemetry, and the human side of AI.

Subscribe

Elsewhere

  • About me
  • Workbench notes
  • Substack
  • Medium
  • LinkedIn
  • GitHub
  • X
  • RSS
  1. Home
  2. Essays
  3. Coding harnesses are cheap to swap now. Model behavior isn't.

Coding harnesses are cheap to swap now. Model behavior isn't.

Nov 19, 2026 · 5 min read

Model APIs have absorbed most behaviors, and models themselves have made the cost to switch effectively zero. Where the harness is now free, understanding and influencing model behavior remains the sweet spot for engineering value. Instead of trying to own the harness, own the policies, code, memories, and evaluation.

Two identical modular shipping containers opening to reveal two very different mechanical clockwork creatures working inside.

Teams that are still debating frameworks and custom loops are missing the point: you can swap in and out for close to zero cost now.

In coding agents especially, Anthropic, Google, and OpenAI all moved the core loop straight into their API's (Claude Managed Agents, the Gemini Interactions API, and OpenAI's Agents API). Open specs (AGENTS.md, SKILL.md, and MCP) standardized the outer contract across harnesses like Antigravity, Claude Code, and Codex. Even if you throw out your pipeline and rebuild on another SDK, a coding agent can rewrite that plumbing in an afternoon.

The catch is model behavior.

Most of what teams called a coding 'harness' is now dead code inside the model API.

Why the model API's ate the early coding harness

Early text API's were stateless. Tokens in, tokens out. Everything that turned a model into a coding agent lived in your own glue code (chat history arrays, hand rolled edit and verify loops, regex diff parsers, JSON tool retries, manual prefix caching, repo search wrappers, and generic while tool_call: loops).

It was not a moat.

Every major lab absorbed that layer straight into the API. Native thinking state, implicit caching, structured outputs, built in code and file search, server side state, and remote execution sandboxes all moved server side.

OpenAI gave us the cleanest tell for why this happened. They shut down the legacy Assistants API and launched the Agents API two weeks later, turning their internal Codex harness into a cloud endpoint.

Why kill Assistants and ship Agents right after? Assistants was an early guess built before labs ran real coding agents at scale. The surviving agent API's (Interactions API remote agents, OpenAI's Agents API, and Claude Managed Agents) were pulled straight out of battle tested coding harnesses.

Early Coding Harness LayerModel API EquivalentStatus in User Code
ReAct / edit verify prompt loopsNative thinking + preserved thought stateDead code
Regex diff parsers & tool call retriesConstrained decoding / Structured OutputsDead code
Chat history & manual prefix cache slicingServer side state + implicit cachingMostly absorbed
Boilerplate repo search & schema glueBuilt in search + Remote MCPMostly absorbed
Generic cloud while tool_call: sandboxesHosted coding sandboxes in the APIAbsorbed for cloud runs
Policies, code, memories & evaluationNot in the model APIWhat you still own

Where the platform is glue and loops, assume that the model API's have already consumed or will consume that behavior. What stays yours is owning the policies, the code, the memories, and the evaluation. The rest is interchangeable.

Diagram separating interchangeable coding harness plumbing from the four layers you still own: policies, code, memories, and evaluation.

Plug and play harnesses, different model behavior

It feels like Kubernetes or containers, but there's one large difference. Roughly 10 years ago today, the CNCF and Kubernetes was launched. This year, it's the Agentic AI Foundation. Standards are improving the MCP and plugins ecosystem, and making the 'harness' plug and play. But unlike containers where the underlying hardware also acts similarly, models act very differently (think of it almost like ARM vs. Intel, except they both accept the same input and will then give you different outputs).

A recent set of examples from my own machine hits this point.

First, same rules, opposite tool habits. In my audit of Gemini 4 Argon and Claude Opus 5.5 in Antigravity, switching models took zero code changes under identical AGENTS.md rules and the same ast-graph plugin. Argon read the tool schemas and queried the graph across most sessions. Opus skipped the schemas completely and stayed in bash running grep, except in FreightFloor, where a one line script let it call the graph from the shell.

Second, rules rot across model generations. I hit the flip side of that exact split with my UI skills (full scorecard in Notes). Old UI rules I wrote for smaller Flash models were still sitting in my repos. Opus skipped the skill menu and built clean layouts from its own defaults. Gemini read every matching skill step by step and faithfully built cluttered brown dashboards covered in static pills. Deleting 95% of my agent config fixed the UI builds right away.

Third, even after that cleanup, rebuilding a 10 ft projector UI showed the split is still live. Gemini opened a built in web guidance skill on step 1 while Opus didn't open it at all. It built desktop dropdowns until I reminded it a projector isn't a desktop browser. Swap models, or even upgrade inside one vendor, and the agent's coding habits still shift.

What's actually left to own in a coding harness

I wrote that 100% of developers should be building things to improve the workflow instead of hand patching broken output.

Owning the workflow doesn't mean maintaining commodity orchestration glue. Writing a custom edit loop or tool retry wrapper now is like writing a custom container scheduler a decade ago.

The reality: the parts of a coding harness worth owning are the boundaries a model API can't enforce from inside its own weights, plus the feedback loops that catch behavioral drift across swaps.

In my own repos, four layers still earn their keep:

  • Policies outside the prompt: OS command sandboxes, fail closed approval rules for shell and file writes, and real spend or token caps. On FreightFloor, a stale spend counter and crawler traffic pushed launch week past \$120 before we checked the bill. Guardrails only count when they don't rely on the model behaving.
  • Code and dual path tools: git worktrees for parallel branches, private repo boundaries, and shipping key dev tools as both MCP and CLI so models with different habits can reach them.
  • Memories and repo context: keeping your coding rules, skills, and project context in plain open files (AGENTS.md and SKILL.md) and pruning stale instructions before they poison newer models.
  • Evaluation and verification hooks: stop checks that block completion when tests fail, edit hooks that keep local code indexes fresh, and session log audits every time you swap or upgrade a coding model.

This is where engineering time still adds value.

Take a look at what your team calls a 'coding harness' or 'agent framework' this week. Cut the glue. Delete the retry loops, cache plumbing, and custom tool wrappers the model API's already absorbed. Keep your repo workflows in plain open files (AGENTS.md, SKILL.md, and simple CLI / MCP sidecars) so switching harnesses stays cheap.

Then put the time you get back into policies, code, memories, and reading your coding session logs every time you swap a model (usually you'll find a surprise).

What custom coding harness glue have you deleted, and where did a model swap still bite you? Let me know in the comments!

← All essays Browse all 27 essays Older essay → FreightFloor two weeks in: the bill, the wobble, and the cab view

Get new posts by email

I send new essays by email through Substack. Subscribe there and they'll land in your inbox.

Subscribe on Substack ↗
About · Notes · Subscribe on Substack · RSS · Privacy ·
Ask

Social card and structured data

The share image, feeds and Schema.org JSON-LD that search engines and AI tools read for this page.

Social card

Standardized modular containers on clean tracks contrasting uniform outer plumbing with distinct agent behaviors inside.
Card URL: /og/agent-harnesses-cheap-to-swap-behavior-isnt.svg

Machine feeds

/llms.txt /llms-full.txt /sitemap.xml /api/agent/spec

Structured data

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Person",
      "@id": "https://davenporter.com/#jason-davenport",
      "name": "Jason Davenport",
      "alternateName": "@davenportjw",
      "jobTitle": "AI & Distributed Systems Architect",
      "description": "Exploring agent-first source control (cosm), trajectory-to-lineage observability, Spanner-backed agent memory consolidation (Electric Sheep), hybrid local/cloud LLM routing, and real-time multiplayer browser simulations on Google Cloud Run.",
      "url": "https://davenporter.com",
      "sameAs": [
        "https://davenporter.substack.com",
        "https://www.linkedin.com/in/davenportjw",
        "https://github.com/davenportjw",
        "https://x.com/davenportjw",
        "https://medium.com/@davenportjw"
      ]
    },
    {
      "@type": "WebSite",
      "@id": "https://davenporter.com/#website",
      "url": "https://davenporter.com",
      "name": "The Davenporter",
      "description": "Learnings about software, technology, agentic systems, and occasionally home renovation.",
      "publisher": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "inLanguage": "en-US"
    },
    {
      "@type": "ImageObject",
      "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#primaryimage",
      "url": "https://davenporter.com/static/images/essays/agent-harnesses-cheap-to-swap-behavior-isnt/og-harnesses-vs-behavior-1200x630.png",
      "contentUrl": "https://davenporter.com/static/images/essays/agent-harnesses-cheap-to-swap-behavior-isnt/og-harnesses-vs-behavior-1200x630.png",
      "width": 1200,
      "height": 630,
      "caption": "Standardized modular containers on clean tracks contrasting uniform outer plumbing with distinct agent behaviors inside.",
      "representativeOfPage": true,
      "creator": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "license": "https://creativecommons.org/licenses/by/4.0/",
      "acquireLicensePage": "https://davenporter.com/writings"
    },
    {
      "@type": "WebPage",
      "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#webpage",
      "url": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt",
      "name": "Coding harnesses are cheap to swap now. Model behavior isn't. | The Davenporter",
      "description": "Model APIs have absorbed most behaviors, and models themselves have made the cost to switch effectively zero. Where the harness is now free, understanding and influencing model behavior remains the sweet spot for engineering value. Instead of trying to own the harness, own the policies, code, memories, and evaluation.",
      "isPartOf": {
        "@id": "https://davenporter.com/#website"
      },
      "about": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "primaryImageOfPage": {
        "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#primaryimage"
      },
      "inLanguage": "en-US"
    },
    {
      "@type": "FAQPage",
      "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#faq",
      "mainEntity": [
        {
          "@type": "Question",
          "name": "What is \u201cCoding harnesses are cheap to swap now. Model behavior isn't.\u201d about?",
          "acceptedAnswer": {
            "@type": "Answer",
            "text": "Model APIs have absorbed most behaviors, and models themselves have made the cost to switch effectively zero. Where the harness is now free, understanding and influencing model behavior remains the sweet spot for engineering value. Instead of trying to own the harness, own the policies, code, memories, and evaluation."
          }
        }
      ]
    },
    {
      "@type": "BlogPosting",
      "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#article",
      "mainEntityOfPage": {
        "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#webpage",
        "url": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt"
      },
      "headline": "Coding harnesses are cheap to swap now. Model behavior isn't.",
      "alternativeHeadline": "Model APIs and open specs drove switching costs for coding harnesses near zero, right up until you swap the model.",
      "description": "Model APIs have absorbed most behaviors, and models themselves have made the cost to switch effectively zero. Where the harness is now free, understanding and influencing model behavior remains the sweet spot for engineering value. Instead of trying to own the harness, own the policies, code, memories, and evaluation.",
      "datePublished": "2026-11-19T16:25:00Z",
      "dateModified": "2026-11-19T16:25:00Z",
      "wordCount": 1110,
      "keywords": "Agent Engineering, Developer Workflow, Developer Tools, Engineering Leadership",
      "author": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "publisher": {
        "@id": "https://davenporter.com/#jason-davenport"
      },
      "image": {
        "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#primaryimage"
      },
      "url": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt"
    },
    {
      "@type": "BreadcrumbList",
      "@id": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt#breadcrumb",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://davenporter.com/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "Essays",
          "item": "https://davenporter.com/writings"
        },
        {
          "@type": "ListItem",
          "position": 3,
          "name": "Coding harnesses are cheap to swap now. Model behavior isn't.",
          "item": "https://davenporter.com/writings/agent-harnesses-cheap-to-swap-behavior-isnt"
        }
      ]
    }
  ]
}

Ask The Davenporter

Ask anything about the essays and builds on this site.