Setting up Qwen3 235B A22B on your hardware, agent stack live, production-grade in 14 days

Qwen3 235B A22B is the open-weight model that closes the gap with closed frontier labs. Released April 28, 2025 by the Qwen team at Alibaba, it ships under Apache 2.0 with no commercial caveats.

  • 14-day production cutover
  • Coding + research agent stack included
  • 235B total, 22B active per token (MoE) Parameters
  • 128K native, 262K on Instruct-2507 variant Context window
  • 14 d Production cutover

Qwen3 235B A22B at a glance

Specs that matter when picking the model and the hardware.

Parameters

235B total, 22B active per token

Architecture

MoE, 128 experts, 8 active per token

Context window

128K native, 262K on Instruct-2507

License

Apache 2.0

Release

April 28, 2025 (Instruct-2507 in July 2025)

Languages

119 languages and dialects

Modes

Hybrid Thinking and Non-Thinking

Min RAM (Q4_K_M)

143GB

Min RAM (IQ4_XS for 128GB Macs)

112GB

Tokens/sec on Mac Studio M3 Ultra 256GB Q4

15-22

Tokens/sec on 4x H200 FP8 (vLLM)

120-180 batched

Hardware that runs Qwen3 235B A22B in production

Three tiers, real SKUs, real throughput numbers.

  1. Entry tier: Mac Studio M3 Ultra 256GB or 512GB ($9,499-14,999)

    Throughput: 15-22 tokens/sec. Best for: Single-user power workflows, code generation, deep reasoning on private docs.

  2. Mid tier: 2x RTX Pro 6000 Blackwell 96GB ($20-28K) or 4x RTX 5090

    Throughput: 40-70 tokens/sec. Best for: 10-30 user firms, vLLM batched serving, MoE expert offload.

  3. Enterprise tier: 4x H200 SXM 141GB or 8x H100 80GB FP8 ($150-300K)

    Throughput: 120-300 tokens/sec. Best for: 100+ concurrent users, multilingual ops, agentic workflows.

Quantization picks for this model

GGUF, AWQ, FP8, INT4.

Recommended quantization

IQ4_XS for 128GB Macs (close to Q4_K_M quality, fits with 40K context), Q4_K_M for 256GB+ Macs, FP8 for H100 and H200, AWQ INT4 for Blackwell consumer cards

Tradeoffs

MoE models tolerate quantization slightly better than dense models because expert routing dilutes per-weight error. IQ4_XS loses around 3 percent on MMLU-Pro versus FP16. FP8 is the production sweet spot on Hopper and Blackwell, near-lossless and 2x throughput versus FP16.

GGUF IQ4_XS

File size: 112GB.

GGUF Q4_K_M

File size: 143GB.

AWQ INT4

File size: 118GB.

FP8

File size: 235GB.

Coding agents we wire to Qwen3 235B A22B

Native tool calling with the strongest reliability of any open model in 2026.

Qwen-Agent

Native option from Alibaba: pip install -U qwen-agent[gui,rag,code_interpreter,mcp]. Encapsulates tool-calling templates and parsers internally. Pass model_server as the vLLM /v1 URL. Bundles MCP, code interpreter, RAG, and Chrome extension.

Cline

OpenAI Compatible provider. Tool calling on Qwen3 is the most stable in the open model class, rarely hallucinating calls or dropping parameters. Plan and Act mode both work. The Thinking-2507 variant adds reasoning steps for deeper plans, the Instruct-2507 is faster for routine work.

Aider

Aider Polyglot leaderboard puts Qwen3 235B A22B at 65.3 percent, near the top of self-hostable open models. Configure --openai-api-base to vLLM endpoint. Edit-format mode works cleanly across Python, TypeScript, Go, Rust, and Java.

Continue.dev

Add to config.yaml with provider: openai and apiBase to your vLLM. Declare capabilities: [tool_use]. Agent mode auto-detects native tool calling. The 256K context on the Thinking variant unlocks Continue's full repo-context mode.

OpenHands

Strong fit. A 235B RFT checkpoint of the Instruct-2507 base hit 61.7 percent Pass@1 (74.3 percent Pass@5 at 500 turns) on SWE-bench Verified using OpenHands trajectories per Nebius, beating the 30B coding specialist (50.3 percent) at half the parameters of Qwen3-Coder-480B. Set LLM_BASE_URL to vLLM. Best autonomous-loop performance of any model in this batch.

Roo Code

Cline fork. Multi-mode (Code, Architect, Ask, Debug). The Architect mode + Thinking-2507 variant is a strong combination because Qwen3's reasoning trace surfaces architectural tradeoffs cleanly. Same OpenAI-compatible config.

Claude Code (with custom endpoint)

Claude Code accepts OPENAI_BASE_URL override per the unofficial guides. Qwen3 235B A22B is the open model that comes closest to Claude Sonnet on Claude Code's own SWE-bench eval. Watch tool-format compatibility, some Claude Code internal tools assume Anthropic-flavored function calls.

Coding agent use cases

Where this model + agent stack ships real work.

Research agents we deploy on Qwen3 235B A22B

256K context (Thinking-2507 variant) makes Qwen3 235B A22B one of the strongest open models for deep research.

Qwen-Agent

Built-in RAG, code interpreter, MCP. Single pip install gives a working agent stack. Best research starting point if the buyer is committed to the Qwen line.

GPT Researcher

Point OPENAI_BASE_URL at vLLM. Qwen3's instruction following (Qwen 3.5 hits IFEval 92.6) means GPT Researcher's planning prompts are followed precisely. Reports run 3 to 5 minutes for 5 to 10 sources.

AnythingLLM

Generic OpenAI provider works. Workspace RAG plus @agent mode covers 80 percent of buyer use cases. Qwen3's tool reliability means @agent rarely fails on multi-tool sequences.

Open WebUI

Default chat UI. Qwen3's hybrid thinking toggle exposes nicely via Open WebUI's per-conversation settings (chat_template_kwargs.enable_thinking). Buyers can flip reasoning on for hard questions, off for fast chat.

LangGraph

Production research orchestration. Qwen3's reliable tool calling is the differentiator: LangGraph state transitions trigger correctly without retry logic. Pair with LangSmith tracing.

CrewAI

Role-based research crews. Qwen3 235B A22B's strong instruction following means CrewAI's role and goal text is honored without prompt-engineering overhead.

LlamaIndex

Best retrieval pairing. Use LlamaIndex query engines as tools inside a Qwen-Agent or LangGraph workflow. The 256K context lets you keep a much larger retrieved chunk window than typical RAG.

DSPy

Prompt optimization. DSPy's compile step is unusually effective on Qwen3 because the model follows structured few-shot patterns precisely. Use DSPy to optimize the question-decomposition prompt, then deploy via LangGraph.

Research agent use cases

Where this model + research agent stack delivers reports your team actually uses.

Frameworks we ship for production agent workflows

Pick the framework by workflow shape.

Qwen-Agent

Native Qwen agent framework with MCP, RAG, code interpreter, and Gradio UI.

LangGraph

Production stateful orchestration. Best paired with Qwen3's reliable tool calling.

CrewAI

Role-based crews. Fastest to demo with Qwen3's instruction following.

AutoGen / AG2

GroupChat conversations. Useful for Qwen3 hybrid mode debate patterns.

DSPy

Prompt compilation. Strong with Qwen3's structured-format adherence.

Pydantic AI

Type-safe agents. Qwen3's structured output reliability is a clean fit.

LlamaIndex

Retrieval layer. Pair with the 256K Qwen3 context for huge retrieved chunks.

Who picks Qwen3 235B A22B

Three buyer profiles where this model is the right answer.

  1. International law firm, 80 attorneys, 5 offices

    Task: Cross-border contract review in English, French, German, Mandarin, plus discovery. Outcome: 119-language coverage replaces three separate translation tools. Apache 2.0 means redistribution to client portals is clean.

  2. Software consultancy, 40 engineers

    Task: Code generation, code review, agentic refactoring across multiple languages and frameworks. Outcome: 70.7 LiveCodeBench v5 and 2,056 CodeForces beats every dense open model. Hybrid Thinking mode for hard bugs, Non-Thinking for autocomplete.

  3. Pharma research org, 12 scientists

    Task: Literature synthesis, hypothesis generation, GPQA-class STEM Q&A on internal documents. Outcome: SuperGPQA leadership over DeepSeek-V3 and Llama-4 on expert STEM. 256K context handles full clinical trial dossiers.

What clients said after launch

Feedback from the people who were in the room when decisions were made.

I used to spend every Sunday reconciling contractor invoices. Now they match automatically. I got my weekends back.
Karen, Property ManagerDenver
Three drivers called out on the same morning. The system reassigned every route before I finished my coffee. That used to take two hours of phone calls.
James, Fleet ManagerDallas
They did not ask us to trust a rewrite plan. They proved one critical workflow first, then scaled from evidence.
Fatima, CTOSydney

Qwen3 235B A22B vs the closest rivals

Where this model wins, where it loses.

vs DeepSeek V3.2

DeepSeek V3.2 (671B MoE, 37B active) wins on math and code at the absolute top end. Qwen3 235B wins on memory footprint (143GB Q4 versus 376GB Q4) and on hardware accessibility for any firm without an 8x H100 budget.

vs Llama 3.3 70B

Qwen3 235B wins on reasoning, coding, multilingual, and benchmarks across the board. Llama 3.3 70B wins on serving simplicity, hardware floor, and ecosystem maturity. Pick Llama for 8 to 15 person firms, Qwen3 235B for 30+ where quality is the constraint.

vs Mistral Large 3 (675B MoE)

Mistral Large 3 wins on Apache 2.0 plus first-class multimodal vision. Qwen3 235B wins on hardware footprint (143GB versus 370GB at Q4) and on the size-of-firm that can self-host without an enterprise GPU node.

How Pineido sets up agents on Qwen3 235B A22B

Day 4 vLLM (or SGLang) serves the model with tool calling and reasoning parser.

Day 4

vLLM serves Qwen3-235B-A22B-Instruct-2507 with tensor-parallel-size 8, max-model-len 131072 (or 256K on Thinking-2507), --enable-reasoning --reasoning-parser deepseek_r1 for the Thinking variant. FP8 variant fits on 8xH100.

Day 6

Editor agents (Aider, Cline, Continue.dev, Roo Code) wired to /v1. OpenAI-compatible config across all. Tool calling is the most stable of any open model so almost no retry logic needed.

Day 8

Open WebUI plus AnythingLLM live. Hybrid thinking toggle exposed per conversation via chat_template_kwargs.enable_thinking. Workspaces isolate sensitive doc sets.

Day 9

Qwen-Agent installed (pip install -U qwen-agent[gui,rag,code_interpreter,mcp]). MCP servers wired for filesystem, time, fetch, and your internal APIs.

Day 10

LangGraph workflows for production research. Three workflows shipped: deep-research, code-review, customer-intake. LangSmith tracing on.

Day 12

OpenHands repo-autonomy demoed against three real tasks. Expected SWE-bench-class pass rate around 60 percent given the 235B Instruct base.

Day 14

Evidence pack with tool-call success rates, throughput numbers, accuracy spot-checks, runbook, and agent inventory delivered.

How we deploy Qwen3 235B A22B in 14 days

Fixed fee.

Day 1

Confirm hardware tier. 128GB Mac is borderline (IQ4_XS only). 256GB+ Mac or multi-GPU is the real floor. Bench Thinking versus Non-Thinking on actual workload.

Day 4

Serve via vLLM with FP8 (GPU) or MLX with Q4 (Mac). Wire OpenAI-compatible API. Configure thinking-mode toggle by route or by user role.

Day 9

RAG with multilingual BGE-M3 embeddings, Qdrant. SSO, audit logging, RBAC, expert-routing telemetry for cost monitoring.

Day 14

Pen-test cleared. Apache 2.0 NOTICE filed. Cutover. No attribution string required in product UI.

235B total, 22B active per token (MoE) Parameters

Book the 30-minute scoping call. We tell you if Qwen3 235B A22B plus our agent stack is the right pick before you sign.

Fixed fee.

Day 1 you see the exact model, the exact hardware, the exact agent stack, the exact cost math, and the exact compliance posture.

You leave with

  1. Day 0 (this week): 30-minute scoping call. Hardware shortlist, model pick, agent stack proposal, cost projection in your inbox by Friday
  2. Day 1: Confirm hardware tier. 128GB Mac is borderline (IQ4_XS only). 256GB+ Mac or multi-GPU is the real floor. Bench Thinking versus Non-Thinking on actual workload.

The questions that decide if we are right for you

Is Qwen3 235B truly free to use commercially?

Yes. Apache 2.0, no MAU cap, no attribution-in-product requirement, no paid tier. You can fine-tune, redistribute, ship in commercial products, and embed in client deliverables without contacting Alibaba. The only requirement is preserving the Apache 2.0 NOTICE file in source distributions, which is a one-line legal task.

What hardware actually runs Qwen3 235B at usable speed?

Mac Studio M3 Ultra 256GB at Q4_K_M is the cheapest viable path, 15 to 22 tokens per second decode for a single user. 128GB Mac requires IQ4_XS and runs out of room past 40K context. For multi-user production, target 2x RTX Pro 6000 Blackwell 96GB or a 4x H200 node with FP8.

What is Thinking versus Non-Thinking mode and when do I use each?

Thinking mode runs an internal chain-of-thought before answering, slower but stronger on math, code, and multi-step reasoning. Non-Thinking is direct response, faster, suited for chat, drafting, and tool calls. The 2507 Instruct variant is Non-Thinking by default. Route hard problems to Thinking and commodity tasks to Non-Thinking and your tokens-per-dollar improves 3 to 5x.

Should we deploy Qwen3 235B or DeepSeek V3.2?

DeepSeek V3.2 wins on hardest reasoning and IMO-class math. Qwen3 235B wins on hardware floor (143GB versus 376GB at Q4), multilingual breadth, and serving simplicity. For any firm without an 8x H100 cluster sitting idle, Qwen3 235B is the practical pick.

Why is Qwen3 235B A22B's tool calling considered the most reliable open option?

Qwen3's post-training pipeline included RL across 20+ general-domain tasks specifically to strengthen agent capabilities, instruction following, and format following. You get the lowest tool-call hallucination rate of any open model in the 2026 benchmarks, and the chat template natively encodes the OpenAI-compatible tool format plus MCP. In Pineido sprints we see noticeably fewer retry-loop bugs on Qwen3 versus Llama or Mistral.

Should we use Qwen3-235B-A22B-Instruct-2507 or Thinking-2507?

Both. Pineido deploys the Instruct-2507 as the default endpoint because it is faster and skips the thinking tokens. We deploy Thinking-2507 on a second port for hard problems. The two models share the architecture so vLLM serves them side by side. Cline, Roo Code, and Open WebUI can route per task. The Thinking variant scores ~7 to 12 points higher on hard reasoning benchmarks but is roughly 3x slower per response.

ByFounder and Imaginator

Pineido, based in Strumica, North Macedonia, is a software company that builds websites, online shops, web and mobile apps, business automation and AI for companies at home and abroad. Work ships in 30-day sprints at a fixed price, and the client owns the code.

Published Updated