Ornith 9B vs 35B: Cost, Context, and Speed

I paused at the routing sheet before I paused at the model. That is usually where the real decision starts. Ornith 9B vs 35B looks like a model-size comparison, but for a local coding model it quickly turns into something less neat: what can run close to the developer, what needs shared serving, and what breaks when three agents hit the same box at once.

The official release frames Ornith-1.0 as an open-source agentic coding model family, with 9B Dense and 35B MoE variants among the lineup. I’m treating the official Ornith-1.0 release note as the high-level source, and the individual model cards as the publish-day check. This conclusion has an expiration date. Models update fast.

The short version: start with 9B when the work is local, iterative, and cost-sensitive. Move to 35B when the agent is doing repo-scale reasoning, tool-heavy edits, or work where one bad shortcut costs more than the extra serving budget.

Ornith 9B vs 35B: The Core Tradeoff

The tradeoff is not “small model bad, large model** good.**” That framing wastes time.

The 9B variant is the practical first stop when the question is: can I keep this close to the machine, cheap enough to run often, and responsive enough that the agent does not interrupt my flow? The Ornith-1.0-9B model card documents the coding-agent use case, OpenAI-compatible serving examples, and local runtime paths. That matters because a local LLM deployment is only useful if it fits the way developers already call models.

The 35B variant is for a different kind of pain. The Ornith-1.0-35B points to heavier serving recipes, long-context settings, reasoning parsing, and tool-call parsing. That is not just “more parameters.” It is more operational surface.

So that’s where the bottleneck was.

Local testing and edge use cases

I would start with 9B for local trials, prompt harness work, coding CLI experiments, offline checks, and small automation loops.

Not because it will always win on answer quality. It won’t. The point is that a smaller local coding model lets you test the shape of the workflow before you spend time hardening infrastructure around it.

This is where Ornith Ollama becomes useful. The model card shows an Ollama path through Hugging Face GGUF, and the Ornith-1.0-9B-GGUF page is the page I would re-check before publishing any exact command or quantized file assumption. GGUF makes the local path easier to try, but it does not make quality loss disappear. Quantized builds need their own eval.

For edge-style work, I care about three things:

Constraint	Why 9B is usually the first test
Local responsiveness	Smaller serving footprint is easier to keep interactive
Privacy-sensitive drafts	Local execution can reduce data movement
Frequent experiments	Cheap repeated runs matter more than peak score

Good enough. That is the most honest assessment I can give.

Higher-quality coding agents and heavier deployment cost

35B earns attention when the agent is no longer doing isolated snippets.

The moment the task becomes “read this repo, infer the pattern, edit three files, run tests, recover from failure,” I stop asking only about latency. I start asking whether the model can keep the task scaffold intact.

The 35B GGUF path also exists, and the Ornith-1.0-35B-GGUF page is the source I would check for local quantized availability. Still, a 35B local run is not the same operational category as a 9B local run. The bigger model asks more from memory, scheduling, and patience.

If the team expects multiple coding agents to run at once, 35B usually belongs behind shared inference. Not always. But often enough that I would plan for it early.

Compare by Production Constraint

I use a boring routing table here. Boring is good. It prevents the team from turning benchmark screenshots into deployment policy.

Production constraint	Start with 9B when…	Move toward 35B when…
Hardware budget	The model must run on local or modest shared hardware	Dedicated serving budget already exists
Latency target	Interactive response matters more than deep reasoning	A slower but better agent is acceptable
Concurrency	One developer or one background worker is using it	Several agents share the model
Task risk	Mistakes are cheap to catch	Mistakes trigger costly review or rollback
Repo size	The task touches narrow files	The task needs repo-scale reasoning

This table is not a ranking. It is a routing rule.

Hardware budget, latency target, and concurrency

9B is the model I would use to answer: “Can this workflow exist?”

35B is the model I would use to answer: “Can this workflow hold up when it matters?”

That distinction keeps teams from overbuilding too early. A local 9B trial can validate the agent loop: prompt format, tool calls, code review gates, failure handling, and logging. If those pieces are wrong, a better model only hides the mess for a week.

Concurrency changes the math. One developer running a model locally is a different system from six agents hitting the same endpoint during CI. Once concurrency appears, serving becomes a queueing problem. Then the model decision is tied to batch behavior, request shape, timeout policy, and fallback.

Speed is not the goal. Not breaking flow is.

Context length needs and repo-scale tasks

The public Ornith cards show long-context recipes, including 262144-token settings in serving examples and benchmark footnotes that reference 128K, 256K, and larger task windows. I would write that as “supports long-context serving paths shown by the model card,” not “every deployment gets perfect 256K context.”

That distinction matters.

A 256K context window is useful when the agent needs to keep architecture notes, test output, stack traces, and multiple source files in view. It is less useful when the task is a two-function edit and the bottleneck is instruction following.

For repo-scale tasks, context length is only one part of the eval. I would also check:

Whether the model preserves file boundaries.
Whether it follows tool-call schema.
Whether it stops after the requested edit.
Whether it can recover after a failing test.
Whether review diffs become smaller or larger.

A long window that produces messy patches is not a win. It is just a larger mess.

Decision Framework for Model Routing

The cleanest route is staged.

Run 9B first for workflow shape. Promote specific task classes to 35B only after the eval shows a repeatable quality gap. Keep rollback rules separate from benchmark scores.

This is where many teams get sloppy. They see a stronger benchmark, switch the default model, and then discover their internal harness was tuned around different behavior. The demo looked better. The review queue got worse.

When to start with 9B

Start with 9B when the work is close to the developer and repetition is the main cost.

I would put these tasks there first:

Local code explanation.
Small refactors.
Test generation drafts.
Shell-command planning with human approval.
Offline repo Q&A.
Agent harness development.
Prompt and tool-schema iteration.

The win is not only lower cost. It is shorter feedback loops. If the model can run where the work happens, developers test more often. More tests expose the real failure modes earlier.

9B is also the better default when the team has not built eval ownership yet. No model should become central infrastructure before someone owns the failure log.

When 35B earns the extra serving cost

35B starts earning its keep when the task requires longer reasoning chains, better recovery, or more stable tool use.

I would test 35B on:

Multi-file bug fixes.
Repo migration planning.
Large-context code review.
Agentic coding with tool calls.
Tasks where the model must infer conventions from existing code.
Work where a bad patch burns reviewer time.

The word “earns” matters. I would not switch because the model is larger. I would switch because the eval shows fewer bad diffs, fewer retries, or fewer human corrections on the task class that matters.

One fewer switch. Sounds small. Adds up fast.

FAQ

When should a team graduate from a local trial to shared inference?

Graduate when the model stops being a personal experiment and starts becoming a dependency.

The signal is not only usage volume. It is coordination. If multiple developers need the same endpoint, the same logs, the same model version, and the same rollback path, local trials have done their job. Shared inference gives the team one place to manage versions, access, observability, and policy.

I would make the move before the model enters CI or customer-facing workflows. Waiting until after that is how “just a local test” becomes production infrastructure with no owner. Fun sentence. Bad week.

Who should own eval results when model variants change?

The owner should be the team that pays the review cost.

If platform owns serving but engineering owns code quality, both need a seat. Platform can track uptime, latency, concurrency, and model versions. Engineering should own task-level quality: accepted diffs, failed tests, review comments, rollback events, and cases where the agent looked confident but missed the point.

Do not let eval live only in a spreadsheet maintained by the person most excited about the model. That spreadsheet will age badly.

What rollback signal should override a better benchmark?

A production rollback signal should beat a public benchmark when it maps directly to your workflow.

The strongest signals are boring: rising review rejection rate, more failed CI runs, more manual patch cleanup, tool-call schema errors, repeated context truncation, or a spike in “agent touched unrelated files” incidents. Any one of those can justify rolling back even if the new model looks stronger on a leaderboard.

Benchmarks tell me where to look. Internal failure logs tell me what to route.

For Ornith 9B vs 35B, that is the decision I would keep: 9B for local iteration and cheap workflow discovery; 35B for heavier coding agents when eval proves the extra serving cost reduces real review pain. This is where my data ends. Run it against your own repo. That will tell you more than a ranking ever will.

Previous posts: