Local AI models for coding in 2026: Qwen3.8 vs Claude and Codex

HUB LLC · 16 September 2026 · Coding agents

A development team in 2026 can run a capable coding model on its own hardware, pay for a frontier model by the token, or hand a repository to a cloud coding agent. Which model is best matters less than which model should handle which part of the work, and what running one yourself actually involves.

This guide compares Qwen3.8, the current open-weight Qwen generation, with Anthropic’s Claude and OpenAI’s Codex. It covers where each one fits, what hardware a local model needs, and how to combine local and cloud models.

Why local AI models matter in 2026

Developers now choose between chat assistants, coding agents, API models called from their own tools, open-weight models on their own hardware, and combinations of these. Teams look at local inference for concrete reasons: control over source code and data, a model version that never changes unless they change it, offline or restricted environments, high-volume automation without per-request billing, and freedom to experiment with prompts, quantizations and fine-tunes.

Running a model locally does not make it cheaper, faster, smarter, more secure or easier by default. You buy and maintain the hardware, choose and update models, run the inference server and own every failure. Those costs decide whether local AI makes sense for a particular workload.

What Qwen is, and which version is current

Qwen is the family of large language models from the Qwen team at Alibaba Group. Many releases publish downloadable weights on Hugging Face and ModelScope, which makes Qwen one of the most common choices for local AI.

The current generation is Qwen3.8 (Qwen writes it without a space; “Owen 3.8” is a misspelling). Its open weights were published in August 2026 in two very different sizes:

• Qwen3.8-27B: a dense 27-billion-parameter model with a vision encoder, under the Apache 2.0 license. It has 262,144 tokens of native context, extensible to about one million, and reasons in a thinking mode that is on by default and can be switched off per request.
• Qwen3.8-2.4T-A95B: a text-only mixture-of-experts (MoE) model with 2.4 trillion total parameters, 95 billion of them active per token, released under its own Qwen3.8-Max license.

Older generations are still useful locally. Qwen3.5 (February 2026) ranges from 0.8B to 397B-A17B. Qwen3.6 (April 2026) added 27B dense and 35B-A3B MoE models. For coding agents, Qwen3-Coder-Next is an Apache 2.0 MoE model with 80B total and 3B active parameters.

Precision and quantization

Precision is the number of bits stored for each weight:

• BF16/FP16: 2 bytes per parameter. Qwen3.8-27B’s BF16 files are about 56 GB.
• FP8 or 8-bit integer (INT8, GGUF Q8_0): roughly half of that. Qwen publishes an official FP8 checkpoint of Qwen3.8-27B.
• 4-bit (GGUF Q4_K_M, NVFP4, 4-bit MLX): roughly a quarter, about 18 GB for Qwen3.8-27B.

Quantization makes local deployment practical: models fit into less memory and generate faster because less data moves for each token. The trade-off is some loss of quality that grows as the bit count drops. 8-bit is usually close to the original, 4-bit is the common compromise, and 1–3-bit versions need careful testing on your own tasks.

Local models, Claude and Codex are different tools

A downloaded model is an engine. Claude Code and Codex are complete agents built around frontier models, with tools to read repositories, edit files, run commands and manage work.

Local Qwen or other open models give you local processing, control over data and model versions, offline use, fine-tuning and high-volume jobs without API billing. The limits: hardware cost, memory ceilings, deployment and model management, agent tooling you assemble yourself, and weaker results than frontier models on many complex coding and reasoning tasks.

Claude: Anthropic’s lineup includes Claude Fable 5.1, for demanding reasoning and long-horizon agentic work, and Claude Opus 5, for complex agentic coding and enterprise work. Both have a one-million-token context window. Claude Code reads a codebase, edits files, runs commands and works with git from the terminal, IDEs, a desktop app or the browser. It is strong at understanding unfamiliar code, architecture reasoning, multi-file refactoring and documentation. See our Claude Fable 5.1 guide.

OpenAI Codex is available as a CLI, IDE extension, desktop app and cloud agent. It implements features, fixes failures, reviews code, runs tests and handles parallel tasks. Its most capable model is GPT-6 Astra, with a 1,050,000-token context window and selectable reasoning effort. “Codex Astra” is not an official model name; it means using GPT-6 Astra inside Codex. See our GPT-6 Astra article.

The Claude Code and Codex CLIs run on the developer’s machine, but inference happens in the provider’s cloud, so prompts and code excerpts leave the machine. Codex can also use a local provider (Ollama or LM Studio) with its --oss flag, one practical route to a hybrid setup.

Where local models can be the better choice

Local models suit work that is high-volume, repeatable and well-defined, where the expected output is clear and a wrong answer is cheap to detect:

• classifying source files by module, responsibility or risk;
• summarizing and grouping application logs;
• drafting documentation and code comments for review;
• translating UI strings, product content and documentation;
• extracting structured data from HTML, XML, JSON or documents;
• retrieval-augmented generation (RAG) over internal knowledge, as in the internal AI assistants we build;
• reviewing sensitive code without an external inference API;
• pipeline jobs such as commit classification, first-pass bug triage and test drafts.

Where Claude or Codex are still more appropriate

Cloud frontier models earn their cost on ambiguous requirements, large refactors and migrations, architecture decisions, unfamiliar repositories, complex debugging, multi-step autonomous coding across many files, and security analysis. What decides it is the cost of a wrong answer: a subtle checkout bug in production costs far more than any inference bill. When mistakes are expensive, use the strongest reasoning available and review the result against a production readiness checklist.

Task quality decides what a local model can do

Smaller models recover poorly from vague requests. The same model that fails an open-ended instruction can be reliable on a narrow one:

Poor:
Improve this website.

Better:
Inspect the PHP files responsible for product-page metadata. Identify where
canonical URLs are generated. Do not modify files. Return the relevant files
and explain the current logic.

Even better:
Inspect app/code/Vendor/Seo/ and determine why canonical URLs contain query
parameters. Do not change database data. Do not modify vendor/ files.
Return: the root cause, the affected files, a proposed fix
and a test procedure.

The last version has a narrow scope, explicit files, constraints, a defined output and implied acceptance criteria. Instead of “build this feature”, split the work into steps: inspect the architecture, locate the relevant files, explain the current implementation, propose changes, implement one component, run tests, review the diff. A well-prompted local model can often handle most of those steps; the implementation may need a cloud agent. Our free developer prompt generator builds this structure.

Local AI hardware requirements

Memory needs depend on parameter count, quantization, context length, runtime and concurrent users. The weights plus the KV cache must fit in GPU memory, or unified memory on a Mac, for good speed. Unsloth’s rule of thumb is that RAM plus VRAM should roughly equal the quantized model size; below that, inference slows sharply.

The KV cache is easy to overlook. From Qwen3.8-27B’s published configuration we calculate about 64 KiB of 16-bit cache per token: roughly 2 GiB at 32K tokens and 16 GiB at the full 262K. Every concurrent conversation needs its own cache.

Sizes are Ollama and Unsloth download sizes. Leave headroom for the KV cache, runtime and operating system. MoE models must fit completely in memory, but only their active parameters are read per token, so a 35B-A3B model generates text much faster than a dense 27B model on the same machine.

Apple Silicon and Mac Studio

Apple Silicon Macs share unified memory between CPU and GPU, so models can use most of the machine’s memory. Apple’s MLX framework is built for this design, llama.cpp treats Apple Silicon as a first-class target, and Ollama runs on MLX. Current Apple configurations:

• 32 GB (the M6 Mac mini maximum): small models, or a 27B model at 4-bit with short context. Ollama’s MLX announcement asks for more than 32 GB.
• 48–64 GB (Mac mini with M5 Pro; Mac Studio with M5 Max from 36 GB): Qwen3.8-27B at 4- or 8-bit, or Qwen3.6-35B-A3B.
• 96–128 GB (M5 Max MacBook Pro and Mac Studio up to 128 GB; M5 Ultra from 96 GB): Qwen3-Coder-Next at 4-bit, or several models loaded side by side.
• 256–512 GB (Mac Studio with M5 Ultra): the largest local models; Qwen3.8-2.4T fits only as its 1-bit version at 512 GB.

A model that fits may still be slow. Generation speed depends mostly on memory bandwidth: 153 GB/s on M5 and M6 chips, up to 614 GB/s on the top M5 Max and 1.2 TB/s on the M5 Ultra. Prompt processing depends on GPU compute and grows with context, so a large pasted codebase means a long wait for the first token. Concurrent users multiply cache memory and share throughput.

NVIDIA GPU systems

With discrete GPUs, VRAM is usually the limit. llama.cpp can split a model between GPU and system RAM, but throughput falls substantially.

• GeForce RTX 5090: 32 GB GDDR7 at 1,792 GB/s. Fits Qwen3.8-27B at 4-bit for one user, not an 80B model.
• RTX PRO 6000 Blackwell: 96 GB, enough for Qwen3-Coder-Next at 4-bit or a 27B model at 8-bit for several developers.
• DGX Spark: 128 GB unified memory; NVIDIA says it runs inference on models up to 200 billion parameters.
• Multi-GPU servers with vLLM or SGLang, which Qwen recommends, for many concurrent requests.

Beyond the GPU, plan system RAM, fast storage, power, cooling, noise and the number of requests that must run at once.

CPU-only inference and storage

llama.cpp uses AVX, AVX2, AVX-512 and AMX on x86 processors, so CPU-only inference works for small quantized models and for MoE models with few active parameters. Larger models run slowly but remain useful for batch jobs, overnight processing, background classification and experiments.

Storage fills faster than expected. Medium models take 18–40 GB each, large ones 50–85 GB, and teams keep several models, quantizations and formats (GGUF, MLX, safetensors) alongside embeddings, vector databases, datasets and containers. Plan on 1–2 TB or more of NVMe storage rather than 500 GB.

A local AI setup for a small development company

Developer
   ↓
AI router
   ↓
Local model  OR  cloud model
   ↓
Development tools

The router applies simple rules: simple, repetitive and sensitive tasks go to the local model; complex implementation goes to Codex; complex analysis goes to Claude or another high-capability model. This is AI model routing: choosing a model by task complexity, data sensitivity and cost of error. Ollama and llama.cpp’s server both expose OpenAI-compatible endpoints, so a router can be a small internal service.

Hybrid AI development

Often the best architecture uses both. A local model classifies the issue, collects relevant files, summarizes logs and writes a structured task. Codex or Claude handles the difficult implementation with a smaller, focused context. The local model then drafts documentation and updates the knowledge base. The cloud model sees less irrelevant context, which lowers token usage and usually improves answers. We build this pattern into client automation pipelines.

Security and privacy: local is not automatically private

Local inference gives more control over where data is processed, but privacy depends on the whole system:

• an inference server reachable from the network is an open API unless protected;
• runtimes, IDE extensions and agents may log or send telemetry;
• some tools mix local and cloud: Ollama offers cloud models that run on its servers;
• agents keep local copies: Claude Code stores plaintext session transcripts under ~/.claude/projects/ for 30 days by default;
• RAG databases, caches, backups and containers need the same access control as the source data;
• plugins may call third-party APIs, and downloaded model files should come from trusted publishers.

For cloud models, read the terms for your account type. Anthropic, for example, documents 30-day standard retention for commercial users, with zero data retention available for qualified organizations.

Cost: when local infrastructure pays off

Local AI monthly cost = hardware amortization + electricity + maintenance + engineering time
Cloud AI monthly cost = requests/tokens + subscriptions + related infrastructure

Cloud prices are per token; current list prices include $5/$25 per million input/output tokens for Claude Opus 5 and $10/$50 for Claude Fable 5.1 and GPT-6 Astra. With low or irregular usage, the cloud is usually simpler because nothing sits idle. With high, predictable volume, such as nightly classification of every log or product, local infrastructure becomes attractive if a local model meets the quality bar. Compare the cost of each correctly completed task, not each request.

Tips for working with local models

• Use quantized models and start smaller: test whether 9B or 27B already solves the task.
• Give files, not repositories. Relevant excerpts beat maximum context for speed and accuracy.
• Use structured prompts:

TASK:
CONTEXT:
FILES:
CONSTRAINTS:
OUTPUT:
ACCEPTANCE CRITERIA:

• Use RAG for internal knowledge, cache repeated results, and route easy work locally and hard work to the cloud.
• Benchmark your own workloads. Public benchmarks say little about Magento modules or PHP migrations. Keep a small internal test set of real tasks and rerun it whenever you change a model or quantization.

Example: a Magento store with 100,000 products

Local model tasks: product classification and attribute suggestions for review, translation checks, draft meta descriptions, classifying entries in exception.log and system.log, support ticket categorization, module documentation and code explanation for new developers.

Cloud agent tasks: complex extension development, checkout bugs involving quotes, totals and payment integrations, Magento upgrades and Composer conflicts, architecture changes such as queue-based imports, and security patches.

The first group is high-volume and easy to verify; the second is rare, risky and cross-cutting. More in our Magento AI optimization guide and AI for eCommerce services.

Example: an AI coding pipeline

Developer request
→ local model writes a structured specification
→ local model finds relevant files in the indexed repository
→ complexity classifier
   LOW  → local model
   HIGH → Codex / Claude
→ automated tests
→ developer review
→ local model generates documentation

The classifier can use simple signals: expected number of changed files, whether payments, authentication or data migrations are involved, and whether the specification is complete. Human review remains required for production changes; tests catch regressions but do not confirm business intent. This review-first approach is described on our AI engineering page.

Comparison: local Qwen, Claude and Codex

Conclusion: model routing, not model loyalty

Local AI is now useful for professional software development. Open-weight models such as Qwen3.8-27B and Qwen3-Coder-Next run on one well-equipped workstation and handle a meaningful share of everyday engineering work. They do not need to replace cloud models to be worth it.

Use the smallest, most controllable model that reliably solves each task. Local models suit repetitive, private, high-volume and well-defined work. Claude, Codex and other capable cloud systems suit work where complexity, reasoning depth and reliability justify the cost. For workstation planning, see our AI hardware guide for coding.

The future of AI development is model routing, not model loyalty.

Sources and further reading

Specifications, sizes and prices were checked in September 2026 and change often.

• Qwen: Qwen3.8 repository; model cards for Qwen3.8-27B, Qwen3.8-2.4T-A95B, Qwen3-Coder-Next
• Ollama: Qwen3.8 tags, MLX on Apple Silicon, cloud models; Unsloth: run Qwen3.8 locally
• Runtimes: llama.cpp, Apple MLX
• Anthropic: models overview, Claude Code, Claude Code data usage
• OpenAI: GPT-6 Astra, Codex models, Codex CLI reference
• Hardware: Apple Mac Studio, Mac mini, MacBook Pro; NVIDIA RTX 5090, RTX PRO 6000, DGX Spark