Entry tier: Mac Mini M4 24GB unified memory, RTX 3090 24GB, RTX 4070 Ti 16GB at Q4
Throughput: 10 to 30 tok/s on Mac Mini M4, 40 to 70 tok/s on RTX 3090 tokens/sec. Best for: Solo developers, local agents, offline reasoning workflows.
Phi-4 is the model you pick when you need real reasoning on a single GPU. 14.7 billion parameters, MIT license, dense decoder-only architecture, and a training recipe that leans hard on synthetic data with verifiable answers.
Specs that matter when picking the model and the hardware.
14B (dense)
16K tokens
Decoder-only Transformer
~10T (synthetic-heavy mix)
MIT (full commercial use)
~28GB
~9GB
Phi-4-reasoning, Phi-4-reasoning-plus
2024-12-12
Hugging Face, Azure AI Foundry, Ollama, llama.cpp
Three tiers, real SKUs, real throughput numbers.
Throughput: 10 to 30 tok/s on Mac Mini M4, 40 to 70 tok/s on RTX 3090 tokens/sec. Best for: Solo developers, local agents, offline reasoning workflows.
Throughput: 60 to 120 tok/s, BF16 at full quality tokens/sec. Best for: Small team self-hosted, internal RAG with reasoning, code review bots.
Throughput: 200 to 600 tok/s batched tokens/sec. Best for: Production multi-tenant API for math, structured extraction, or agent backends.
GGUF, AWQ, FP8, INT4.
Q4_K_M for local. BF16 if you have 28GB+ VRAM. Phi-4 is unusually quantization-tolerant because the synthetic training data is well-distributed.
Q4_K_M loses roughly 1 to 2 points on MATH and GPQA versus BF16, which is the smallest gap in its size class. Q3 starts to break reasoning chains. Q5_K_M is the safe middle if you have 12GB+.
File size: undefined.
File size: undefined.
File size: undefined.
File size: undefined.
Phi-4 14B is a small dense model that punches above weight on math and structured reasoning, hits a wall on multi-file repo work.
Works with whole edit format on small repos. Aider Polyglot around 28 to 35 percent on Phi-4. The 16K context budget forces aider --map-tokens 4096 and aggressive file pinning. Multi-file refactors fail predictably once the diff history overflows the window.
Solid for inline edits and chat over the open file. Configure as the chat model, not the autocomplete model, the 14B param count makes first-token latency too slow for keystroke-level suggestions. Best paired with a smaller model (DeepSeek Coder V2 Lite or Qwen2.5 Coder 7B) for autocomplete.
Hits the 16K wall fast. Cline emits a system prompt of 6K to 9K tokens on first turn alone. By turn 3, you are out of context. Workable only on tiny single-file tasks or with Cline's prompt-trimming experimental mode.
Phi-4 via local llama.cpp behind an OpenAI-compatible proxy works for chat. Cursor's context-stuffing aggressive in 2026 saturates the 16K window on any non-trivial file. Niche fit only.
Anthropic-compatible bridge plus Phi-4 is theoretically possible. Practically, Claude Code's tool-use protocol assumes Anthropic-class tool-calling reliability. Phi-4 community tool variants drop tool calls 5 to 12 percent of the time. Skip.
HuggingFace's tiny agent framework is the right fit. CodeAgent mode where the LLM emits Python directly suits Phi-4 reasoning strengths. We use smolagents plus Phi-4 for structured-extraction agents in client pipelines.
Where this model + agent stack ships real work.
16K context is the headline constraint.
Workable with smaller scope. Cap source URLs at 5 to 8, summarize each before final synthesis. Reasoning quality on synthesis is competitive with much larger models on STEM topics.
Phi-4-reasoning's outline-driven structure helps here. Multi-perspective question generation phase works well. Article synthesis phase needs aggressive chunking due to 16K window.
smolagents-based, naturally short-context. Best small-model fit for Phi-4. We have shipped this for client one-off market reports under 20 sources.
Typed schemas force discipline. Phi-4 emits clean structured output, Pydantic V2 validation is fast. Good fit for narrow extraction-then-synthesis pipelines, not broad open-ended research.
Decomposes complex queries into sub-questions, runs them sequentially against retrieved context. Each sub-question fits in 16K, final synthesis stays in budget. The right architectural pattern for Phi-4 research.
Where this model + research agent stack delivers reports your team actually uses.
Pick the framework by workflow shape.
Three buyer profiles where this model is the right answer.
Task: Multi-step planning and tool use on a laptop or Mac Mini. Outcome: Phi-4 reasoning runs offline at usable speed, handles JSON and structured output reliably, and stays on the device
Task: Solving olympiad-level problems with step-by-step reasoning. Outcome: Phi-4-reasoning-plus lands around 82.5 percent on AIME 2025, beating DeepSeek-R1-Distill-70B at one-fifth the size (treat any single AIME run as noisy, Microsoft averages 50-plus generations)
Task: Pulling fields from invoices, contracts, medical notes into JSON. Outcome: Synthetic-data training makes Phi-4 unusually good at strict schemas and refusal-on-uncertainty, with cheap inference cost per document
Where this model wins, where it loses.
Phi-4 is roughly 84 percent larger (14.7B vs 8B) but wins by 15 to 25 points on MATH, GPQA, and HumanEval. Llama 3.1 8B wins on multilingual coverage, 128K context, and long-tail factual recall.
Mistral Small wins on creative writing, multilingual, and 32K context. Phi-4 wins decisively on math, structured output, and reasoning. Phi-4 is also MIT versus Mistral's research license.
Qwen2.5 14B has 128K context (vs 16K) and stronger Chinese and code coverage. Phi-4 beats it on Western math and STEM benchmarks. Pick by language and context needs.
Two-week sprint.
Variant selection. Phi-4 base for extraction agents, Phi-4-reasoning-plus for STEM research (32K context), Phi-4-mini for native tool calling. Tool-calling Modelfile picked (zac/phi4-tools or custom).
vLLM plus Outlines deployed on a 24GB GPU (RTX 4090 or A6000) at FP16. guided_json and guided_regex tested against client schemas. Hit-rate target 99 percent or escalate to FP16 from any quantization.
Pydantic V2 schemas defined for every tool input and output. Pydantic AI agent runtime configured with strict result_type validation. Failure modes mapped (model emits invalid JSON, retry with regenerate or fall back).
Context-budget enforcement. Tiktoken-based pre-flight token count on every prompt. LangGraph node-level summarization or smolagents short-memory mode locked in. Hard fail if prompt exceeds 14K tokens (2K safety margin).
Eval harness on client task suite. Schema-conformance rate, end-to-end task pass rate, average context utilization. Production SLA locked.
Production cutover. Small footprint means single-machine deployment is realistic. Audit log, prompt log, and Pydantic validation errors all flow to one place.
Feedback from the people who were in the room when decisions were made.
I used to spend every Sunday reconciling contractor invoices. Now they match automatically. I got my weekends back.
We didn't rush. We just stopped wasting time on things that didn't matter. The risk was managed, not ignored.
This wasn't theoretical. They got into the weeds with us and didn't leave until the code was live.
Fixed fee.
Pull Phi-4 with Ollama: ollama pull phi4. Validate baseline tok/s on your hardware and run your reasoning eval set
Wire it to your application via the OpenAI-compatible endpoint at localhost:11434. Test JSON mode and tool-calling round trips
If quality is short, evaluate Phi-4-reasoning-plus for math-heavy tasks. Otherwise tune sampling (temperature 0 to 0.3 for reasoning) and add a system prompt that enforces step-by-step output
Move to vLLM or TGI for production. Batch requests, enable prefix caching, set up Prometheus metrics, and benchmark cost per million tokens against your previous OpenAI bill
Yes. Microsoft released both Phi-4 and the reasoning variants under MIT in early 2025. No commercial restrictions, no MAU caps, no acceptable use clauses beyond MIT's standard terms. Use it in products, fine-tune it, redistribute it.
Synthetic data with verifiable answers. Microsoft generated training problems where the correct reasoning path is known, then used Pivotal Token Search and DPO to teach Phi-4 to follow that path. It is narrower than a general 405B, but on the topics it studied, it is competitive.
For most reasoning and structured-output tasks, yes. For long-document RAG you will need to chunk aggressively or pair Phi-4 with a 128K model like Llama 3.1 8B for retrieval. The Phi-4-reasoning variants extend reasoning depth, not context length.
On a 24GB Mac Mini M4 or M4 Pro MacBook, Phi-4 at Q4_K_M runs at 10 to 25 tokens per second through Ollama or LM Studio. On 16GB Macs you can run Q4 but expect memory pressure with any meaningful context. 24GB is the comfortable floor.
The base microsoft/phi-4 release does not ship with a native tool-calling chat template. Community Modelfile variants (zac/phi4-tools, jacob-ebey/phi4-tools) bridge the gap with a custom prompt template plus parsing layer. Phi-4-mini, the smaller sibling, has official Microsoft tool-calling support. For Phi-4 14B production tool use in 2026, plan to use a community variant or build the template yourself. Reliability sits at 88 to 95 percent depending on quantization.
Only on small repos. Phi-4 base has a 16K context window. Aider's repo-map alone burns 8K to 12K tokens on a medium codebase, leaving 4K to 8K for files, diffs, and conversation. The fail mode is silent quality degradation: Aider truncates older context and the model loses track of earlier decisions. Workarounds: aider --map-tokens 4096, pin specific files with /add, switch to Phi-4-reasoning (32K) for repos over 30 files.
14B dense Parameters
Fixed fee.
Day 1 you see the exact model, the exact hardware, the exact agent stack, the exact cost math, and the exact compliance posture.
ByKostadin StambolievFounder and Imaginator
Pineido, based in Strumica, North Macedonia, is a software company that builds websites, online shops, web and mobile apps, business automation and AI for companies at home and abroad. Work ships in 30-day sprints at a fixed price, and the client owns the code.
Published Updated