Introducing Polyglot: One Coding Agent, Any Model

The moment it broke

I have a soft spot for running models locally. No API keys, no per-token meter running in the background, nothing leaving my machine. So when I wanted a terminal coding agent - the kind that reads files, proposes edits, runs commands - my first instinct was to point an existing one at a local open-weight model through Ollama.

It fell apart on the first tool call.

Not with a dramatic crash. The model clearly wanted to call a tool. It said so, it even produced something that looked like a tool call. But the JSON had a trailing comma. Or it used single quotes. Or it wrapped the arguments in an extra {"name": ..., "arguments": ...} envelope because that’s the shape it saw most often in training. The agent framework, which was built to trust a provider’s native function-calling API, took one look at the malformed payload and gave up. The loop stalled. I got a half-answer and a spinner.

I tried a few different CLIs. Same story, different model. The tools that worked beautifully with Claude or GPT were brittle the moment the model on the other end wasn’t one of the two or three that everyone tunes against.

Why this happens

Almost every mainstream agent CLI leans on a provider’s native function-calling API. You hand the provider a JSON schema for each tool, and the provider promises to return tool calls that conform to it exactly. For the frontier hosted models, that promise mostly holds - they’ve been trained and reinforced specifically to honor it.

Open-weight models are a different situation. Qwen, DeepSeek, GLM, Llama and friends are extremely capable at the actual reasoning, but they haven’t all been drilled on one company’s exact tool-call envelope. So they improvise. They emit tool calls that are 95% right and 5% off - and a parser built to expect 100% conformance treats 95% and 0% the same way.

The usual response is to add per-model adapters: special-case this model’s quirk, patch that one’s formatting. That path never ends, because there’s always another model and another quirk.

The reframe

Polyglot starts from a different assumption: a tool call is text the model produced, and text can be parsed and repaired.

Tools aren’t registered through a provider’s function-calling API. They’re described to the model directly in the system prompt, in a simple taught format. When the model streams its response back, a fault-tolerant parser pulls the tool calls out of the raw text and repairs them on the way through:

• Trailing commas, single quotes, unquoted keys - normalized.
• A near-miss tool name (read_files when the tool is read_file) - matched to the closest real tool.
• The model defaulted to OpenAI-style {"name": ..., "arguments": ...} instead of the envelope it was taught - unwrapped and accepted anyway.
• The call is split across streaming chunks - reassembled before parsing.

The model doesn’t have to be perfect. It has to be close enough that the intent is recoverable, and most of the time it is. And when a model slides past that point - producing calls that can’t be salvaged even after repair - the loop stops cleanly and tells you, instead of spinning in a silent retry loop.

One engine under every provider

The part I care about most is that this isn’t a “local model mode” bolted onto the side. The same parser, the same repair pipeline, the same tool executor and permission gate run underneath every provider. Whether you’re on Claude or on a 7B model running on your laptop, the agent loop is byte-for-byte the same code. Behavior doesn’t silently diverge between the “well-behaved” models and the “flaky” ones - there’s just one path, and it’s built to tolerate mess.

The /model switcher. A local Qwen and hosted Claude sit one keystroke apart, and the loop underneath either is identical.

Under the hood the project is split into a UI-agnostic core engine and a thin terminal frontend built with Ink. The tool-call grammar, the streaming parser, and the repair pipeline all live in the core. A future editor extension or desktop app would be a new frontend against that same engine, not a rewrite.

What you actually get

npm install -g @usepolyglot/cli

Point it at a local model:

export POLYGLOT_PROVIDER=openai-compatible
export POLYGLOT_MODEL=qwen2.5-coder
export POLYGLOT_BASE_URL=http://localhost:11434/v1   # Ollama, vLLM, LM Studio, ...
polyglot

Or at Claude, with the same commands and the same behavior:

export POLYGLOT_PROVIDER=anthropic
export POLYGLOT_MODEL=claude-sonnet-4-5
export ANTHROPIC_API_KEY=sk-ant-...
polyglot

A full run against a local 14B model: read the file, propose the change, show the exact diff, apply it on approval - then /status confirms nothing left the machine.

Beyond the core loop, it has the pieces you’d expect from a real coding agent:

• Runs fully offline against any OpenAI-compatible server - no keys, no token costs once the model is pulled.
• MCP support - connect multiple Model Context Protocol servers and expose their tools to the agent.
• Permission modes - manual approval, auto, or plan-only, so the agent can’t run a command or edit a file without the access you’ve granted.
• Scriptable - a non-interactive -p mode for piping Polyglot into other tools.
• Web search as an optional built-in tool, plus real token accounting and session persistence with context compaction for long runs.

Full documentation - first-run walkthrough, the settings and environment reference, the permission model, MCP setup, and the model/server compatibility matrix - is at usepolyglot.dev/docs.

Where it is, and what would help

This is early. It works, I use it daily, but it’s a young project and the most valuable contributions right now are very concrete:

• Fixture tests from real messy model output. If you catch a model emitting a tool call that Polyglot mishandles, that raw text is gold - it becomes a regression test that makes the parser stronger for everyone.
• Windows testing. Most of the development has been on Linux and macOS.
• More OpenAI-compatible provider quirks. Every server has its own small deviations; documenting and handling them is ongoing work.

The CONTRIBUTING guide has the dev setup and the details.

The code is released under the Functional Source License - the source is fully visible, you can read it, modify it, self-host it, and use it internally however you like; the only restriction is shipping a competing commercial product built on it, and each version converts to Apache 2.0 two years after release.

Try it

npm install -g @usepolyglot/cli

👉 Repo: github.com/giuseppe-sirigu/polyglot 👉 Docs: usepolyglot.dev/docs

If you’ve been frustrated trying to get a local model to drive a coding agent, I’d genuinely like to hear how it goes for you - and what breaks.