Setting up Phi-4 on your hardware, agent stack live, production-grade in 14 days

Phi-4 is the model you pick when you need real reasoning on a single GPU. 14.7 billion parameters, MIT license, dense decoder-only architecture, and a training recipe that leans hard on synthetic data with verifiable answers.

  • 14-day production cutover
  • Coding + research agent stack included
  • 14B dense Parameters
  • 16K tokens Context window
  • 14 d Production cutover

Phi-4 at a glance

Specs that matter when picking the model and the hardware.

Parameters

14B (dense)

Context window

16K tokens

Architecture

Decoder-only Transformer

Training tokens

~10T (synthetic-heavy mix)

License

MIT (full commercial use)

FP16 VRAM

~28GB

Q4_K_M VRAM

~9GB

Reasoning variants

Phi-4-reasoning, Phi-4-reasoning-plus

Release date

2024-12-12

Available on

Hugging Face, Azure AI Foundry, Ollama, llama.cpp

Hardware that runs Phi-4 in production

Three tiers, real SKUs, real throughput numbers.

Entry tier: Mac Mini M4 24GB unified memory, RTX 3090 24GB, RTX 4070 Ti 16GB at Q4

Throughput: 10 to 30 tok/s on Mac Mini M4, 40 to 70 tok/s on RTX 3090 tokens/sec. Best for: Solo developers, local agents, offline reasoning workflows.

Mid tier: RTX 4090 24GB or single A6000 48GB at Q5 or BF16

Throughput: 60 to 120 tok/s, BF16 at full quality tokens/sec. Best for: Small team self-hosted, internal RAG with reasoning, code review bots.

Enterprise tier: 1x H100 80GB or 2x L40S for batched serving with vLLM

Throughput: 200 to 600 tok/s batched tokens/sec. Best for: Production multi-tenant API for math, structured extraction, or agent backends.

Quantization picks for this model

GGUF, AWQ, FP8, INT4.

Recommended quantization

Q4_K_M for local. BF16 if you have 28GB+ VRAM. Phi-4 is unusually quantization-tolerant because the synthetic training data is well-distributed.

Tradeoffs

Q4_K_M loses roughly 1 to 2 points on MATH and GPQA versus BF16, which is the smallest gap in its size class. Q3 starts to break reasoning chains. Q5_K_M is the safe middle if you have 12GB+.

BF16

File size: undefined.

Q8_0

File size: undefined.

Q5_K_M

File size: undefined.

Q4_K_M

File size: undefined.

Coding agents we wire to Phi-4

Phi-4 14B is a small dense model that punches above weight on math and structured reasoning, hits a wall on multi-file repo work.

Aider

Works with whole edit format on small repos. Aider Polyglot around 28 to 35 percent on Phi-4. The 16K context budget forces aider --map-tokens 4096 and aggressive file pinning. Multi-file refactors fail predictably once the diff history overflows the window.

Continue.dev

Solid for inline edits and chat over the open file. Configure as the chat model, not the autocomplete model, the 14B param count makes first-token latency too slow for keystroke-level suggestions. Best paired with a smaller model (DeepSeek Coder V2 Lite or Qwen2.5 Coder 7B) for autocomplete.

Cline

Hits the 16K wall fast. Cline emits a system prompt of 6K to 9K tokens on first turn alone. By turn 3, you are out of context. Workable only on tiny single-file tasks or with Cline's prompt-trimming experimental mode.

Cursor (BYOK via local proxy)

Phi-4 via local llama.cpp behind an OpenAI-compatible proxy works for chat. Cursor's context-stuffing aggressive in 2026 saturates the 16K window on any non-trivial file. Niche fit only.

Claude Code (BYOK route)

Anthropic-compatible bridge plus Phi-4 is theoretically possible. Practically, Claude Code's tool-use protocol assumes Anthropic-class tool-calling reliability. Phi-4 community tool variants drop tool calls 5 to 12 percent of the time. Skip.

smolagents

HuggingFace's tiny agent framework is the right fit. CodeAgent mode where the LLM emits Python directly suits Phi-4 reasoning strengths. We use smolagents plus Phi-4 for structured-extraction agents in client pipelines.

Coding agent use cases

Where this model + agent stack ships real work.

Research agents we deploy on Phi-4

16K context is the headline constraint.

GPT Researcher

Workable with smaller scope. Cap source URLs at 5 to 8, summarize each before final synthesis. Reasoning quality on synthesis is competitive with much larger models on STEM topics.

STORM (Stanford)

Phi-4-reasoning's outline-driven structure helps here. Multi-perspective question generation phase works well. Article synthesis phase needs aggressive chunking due to 16K window.

Open Deep Research (HuggingFace)

smolagents-based, naturally short-context. Best small-model fit for Phi-4. We have shipped this for client one-off market reports under 20 sources.

Pydantic AI ResearchAgent pattern

Typed schemas force discipline. Phi-4 emits clean structured output, Pydantic V2 validation is fast. Good fit for narrow extraction-then-synthesis pipelines, not broad open-ended research.

LlamaIndex sub-question query engine

Decomposes complex queries into sub-questions, runs them sequentially against retrieved context. Each sub-question fits in 16K, final synthesis stays in budget. The right architectural pattern for Phi-4 research.

Research agent use cases

Where this model + research agent stack delivers reports your team actually uses.

Frameworks we ship for production agent workflows

Pick the framework by workflow shape.

  1. LangGraph

  2. CrewAI

  3. AutoGen

  4. DSPy

  5. Pydantic AI

  6. smolagents

Who picks Phi-4

Three buyer profiles where this model is the right answer.

Local agent developer

Task: Multi-step planning and tool use on a laptop or Mac Mini. Outcome: Phi-4 reasoning runs offline at usable speed, handles JSON and structured output reliably, and stays on the device

Math and STEM tutor app

Task: Solving olympiad-level problems with step-by-step reasoning. Outcome: Phi-4-reasoning-plus lands around 82.5 percent on AIME 2025, beating DeepSeek-R1-Distill-70B at one-fifth the size (treat any single AIME run as noisy, Microsoft averages 50-plus generations)

Structured extraction pipelines

Task: Pulling fields from invoices, contracts, medical notes into JSON. Outcome: Synthetic-data training makes Phi-4 unusually good at strict schemas and refusal-on-uncertainty, with cheap inference cost per document

Phi-4 vs the closest rivals

Where this model wins, where it loses.

  1. vs Llama 3.1 8B

    Phi-4 is roughly 84 percent larger (14.7B vs 8B) but wins by 15 to 25 points on MATH, GPQA, and HumanEval. Llama 3.1 8B wins on multilingual coverage, 128K context, and long-tail factual recall.

  2. vs Mistral Small 22B

    Mistral Small wins on creative writing, multilingual, and 32K context. Phi-4 wins decisively on math, structured output, and reasoning. Phi-4 is also MIT versus Mistral's research license.

  3. vs Qwen2.5 14B

    Qwen2.5 14B has 128K context (vs 16K) and stronger Chinese and code coverage. Phi-4 beats it on Western math and STEM benchmarks. Pick by language and context needs.

How Pineido sets up agents on Phi-4 14B

Two-week sprint.

Day 1

Variant selection. Phi-4 base for extraction agents, Phi-4-reasoning-plus for STEM research (32K context), Phi-4-mini for native tool calling. Tool-calling Modelfile picked (zac/phi4-tools or custom).

Day 3

vLLM plus Outlines deployed on a 24GB GPU (RTX 4090 or A6000) at FP16. guided_json and guided_regex tested against client schemas. Hit-rate target 99 percent or escalate to FP16 from any quantization.

Day 6

Pydantic V2 schemas defined for every tool input and output. Pydantic AI agent runtime configured with strict result_type validation. Failure modes mapped (model emits invalid JSON, retry with regenerate or fall back).

Day 9

Context-budget enforcement. Tiktoken-based pre-flight token count on every prompt. LangGraph node-level summarization or smolagents short-memory mode locked in. Hard fail if prompt exceeds 14K tokens (2K safety margin).

Day 11

Eval harness on client task suite. Schema-conformance rate, end-to-end task pass rate, average context utilization. Production SLA locked.

Day 14

Production cutover. Small footprint means single-machine deployment is realistic. Audit log, prompt log, and Pydantic validation errors all flow to one place.

What clients said after launch

Feedback from the people who were in the room when decisions were made.

I used to spend every Sunday reconciling contractor invoices. Now they match automatically. I got my weekends back.
Karen, Property ManagerDenver
We didn't rush. We just stopped wasting time on things that didn't matter. The risk was managed, not ignored.
Kwame, Digital DirectorSingapore
This wasn't theoretical. They got into the weeds with us and didn't leave until the code was live.
Marcus, Engineering ManagerGermany

How we deploy Phi-4 in 14 days

Fixed fee.

  1. Day 1

    Pull Phi-4 with Ollama: ollama pull phi4. Validate baseline tok/s on your hardware and run your reasoning eval set

  2. Day 2

    Wire it to your application via the OpenAI-compatible endpoint at localhost:11434. Test JSON mode and tool-calling round trips

  3. Day 3 to 5

    If quality is short, evaluate Phi-4-reasoning-plus for math-heavy tasks. Otherwise tune sampling (temperature 0 to 0.3 for reasoning) and add a system prompt that enforces step-by-step output

  4. Week 2

    Move to vLLM or TGI for production. Batch requests, enable prefix caching, set up Prometheus metrics, and benchmark cost per million tokens against your previous OpenAI bill

What people ask us first

Is Phi-4 really MIT licensed?

Yes. Microsoft released both Phi-4 and the reasoning variants under MIT in early 2025. No commercial restrictions, no MAU caps, no acceptable use clauses beyond MIT's standard terms. Use it in products, fine-tune it, redistribute it.

How does Phi-4 beat models 30x its size?

Synthetic data with verifiable answers. Microsoft generated training problems where the correct reasoning path is known, then used Pivotal Token Search and DPO to teach Phi-4 to follow that path. It is narrower than a general 405B, but on the topics it studied, it is competitive.

Is 16K context enough?

For most reasoning and structured-output tasks, yes. For long-document RAG you will need to chunk aggressively or pair Phi-4 with a 128K model like Llama 3.1 8B for retrieval. The Phi-4-reasoning variants extend reasoning depth, not context length.

Does it run well on a MacBook?

On a 24GB Mac Mini M4 or M4 Pro MacBook, Phi-4 at Q4_K_M runs at 10 to 25 tokens per second through Ollama or LM Studio. On 16GB Macs you can run Q4 but expect memory pressure with any meaningful context. 24GB is the comfortable floor.

Does Microsoft Phi-4 14B support tool calling natively?

The base microsoft/phi-4 release does not ship with a native tool-calling chat template. Community Modelfile variants (zac/phi4-tools, jacob-ebey/phi4-tools) bridge the gap with a custom prompt template plus parsing layer. Phi-4-mini, the smaller sibling, has official Microsoft tool-calling support. For Phi-4 14B production tool use in 2026, plan to use a community variant or build the template yourself. Reliability sits at 88 to 95 percent depending on quantization.

Can Phi-4 14B work with Aider on a real codebase?

Only on small repos. Phi-4 base has a 16K context window. Aider's repo-map alone burns 8K to 12K tokens on a medium codebase, leaving 4K to 8K for files, diffs, and conversation. The fail mode is silent quality degradation: Aider truncates older context and the model loses track of earlier decisions. Workarounds: aider --map-tokens 4096, pin specific files with /add, switch to Phi-4-reasoning (32K) for repos over 30 files.

14B dense Parameters

Book the 30-minute scoping call. We tell you if Phi-4 plus our agent stack is the right pick before you sign.

Fixed fee.

Day 1 you see the exact model, the exact hardware, the exact agent stack, the exact cost math, and the exact compliance posture.

You leave with

  1. Day 0 (this week): 30-minute scoping call. Hardware shortlist, model pick, agent stack proposal, cost projection in your inbox by Friday
  2. Day 1: Pull Phi-4 with Ollama: ollama pull phi4. Validate baseline tok/s on your hardware and run your reasoning eval set

ByFounder and Imaginator

Pineido, based in Strumica, North Macedonia, is a software company that builds websites, online shops, web and mobile apps, business automation and AI for companies at home and abroad. Work ships in 30-day sprints at a fixed price, and the client owns the code.

Published Updated