What temperature, top-p, top-k and the other model settings do

These settings decide how a model picks each next word. For most work you leave them alone, and change one at a time when an answer is too flat, too random or keeps repeating itself.

  • llama.cpp and Ollama names
  • Pineido Studio defaults
  • 8 Settings explained on this page
  • 1 Setting to change at a time

How a model picks the next word

Every setting on this page is a knob on one step, called sampling.

One token at a time

A model writes one token at a time. A token is a word or a piece of a word. At every step the model scores each token it knows. The runner turns those scores into odds, cuts away the unlikely candidates, and draws one token at random from what is left. That draw is sampling.

Wording, not knowledge

Sampling doesn't change what the model knows. It changes which of the model's own candidate words get picked. A low setting won't make a model get facts right that it doesn't know; it only makes the wording steadier.

The filters stack

Top-k, top-p and min-p each cut the list of candidates. A token has to pass every filter that is switched on. It is simpler to steer with one of them and leave the others loose.

Temperature, top-k, top-p and min-p

These four decide how safe or adventurous each pick is.

Temperature

Temperature reshapes the odds before the draw. Below 1, the favourites get stronger and the long shots almost vanish. Above 1, the odds flatten and unlikely words get a real chance. At 0, llama.cpp takes the top token every time. What it changes: low gives steady wording that can turn flat; high gives more variety, more slips, and long answers that drift. When to change it: lower it for facts, summaries and code, raise it for ideas, names and stories, in small steps.

  • llama.cpp: --temp. Ollama: temperature
  • In Pineido Studio: Creativity, "chosen for each message from what you ask"

Top-k (word variety)

Top-k keeps only the k most likely tokens and drops the rest. With top-k 40, the model picks from its 40 favourites, however sure or unsure it is. In llama.cpp, 0 means no limit. What it changes: a small k keeps the wording safe and can make it repetitive; a large k lets in rarer words. When to change it: lower it when a model throws in odd words, raise it a little when every answer sounds the same. Because k is a fixed count, it is a blunt tool next to top-p and min-p.

  • llama.cpp: --top-k. Ollama: top_k
  • In Pineido Studio: Word variety, "top k - candidate words considered". Default 40, a slider from 0 to 100

Top-p (nucleus sampling)

Top-p keeps the smallest group of favourites whose odds add up to p. With top-p 0.9, the model picks from the words that together hold 90% of the odds. When the model is sure, that can be two or three words; when it isn't, hundreds. A top-p of 1 turns the filter off. What it changes: a lower top-p trims the long tail, so the answer stays closer to the obvious wording. When to change it: most people leave it near 0.9 and move the temperature instead. Lower it if a high temperature starts producing nonsense words.

  • llama.cpp: --top-p. Ollama: top_p
  • In Pineido Studio: no slider. Set with the temperature for each kind of message: 0.9, or 0.95 for creative writing

Min-p (minimum probability)

Min-p drops every token whose odds are below a share of the favourite's. With min-p 0.05 and a favourite at 60%, anything under 3% is cut. The cut moves with the favourite: when the model is sure, min-p prunes hard; when it is torn, it leaves the options in. What it changes: it removes junk candidates without flattening answers where several words are equally good, so it pairs well with a higher temperature. When to change it: raise it a little if lively answers wander into strange words, lower it if answers feel clipped.

  • llama.cpp: --min-p. Ollama: min_p
  • In Pineido Studio: Min probability, "min p - drop words below this share of the top". Default 0.03, a slider from 0 to 0.5

Repeat penalty, max tokens, seed and thinking effort

These four control loops, length, repeatable answers and how long a reasoning model thinks.

Repeat penalty (repetition guard)

Repeat penalty makes tokens that already appear in the recent text less likely. A value of 1 does nothing; higher values push harder. llama.cpp and Ollama look back over the last 64 tokens by default. What it changes: it breaks loops where a model repeats a phrase or a list item. Set too high, the model avoids words it needs, like a name it already used or a variable in code. When to change it: raise it one step when a model loops; lower it for code if the model starts renaming things.

  • llama.cpp: --repeat-penalty. Ollama: repeat_penalty
  • In Pineido Studio: Repetition guard, "repeat penalty - higher = less repeating". Default 1.18, a slider from 1 to 2

Max tokens (response length)

Max tokens caps how long a reply can get. The reply ends when the model finishes or hits the cap, whichever comes first, so a low cap cuts answers off mid-sentence. -1 means no cap. What it changes: only the limit. It doesn't make the model write shorter; it stops it. When to change it: set a cap in scripts and batch jobs, so a stuck model can't run on. For chat, leave it unlimited and ask for a short answer in the prompt.

  • llama.cpp: -n or --n-predict. Ollama: num_predict
  • In Pineido Studio: Response length, "max tokens, -1 = unlimited". Default -1, a number box in steps of 64

Seed

The random draw starts from a seed number. The same seed, model, settings and prompt give the same answer; a random seed gives a different one each time. A different runner version or different hardware can still change the result. What it changes: repeatability, nothing else. No seed is better than another. When to change it: fix the seed when you compare settings, so the only thing that changes is the setting you moved.

  • llama.cpp: --seed. Ollama: seed
  • In Pineido Studio: Seed, "same seed = repeatable answers, blank = random". Blank by default

Thinking effort

Reasoning models write out a chain of thought before the answer. Thinking effort sets how much. More thinking costs time before the first word of the answer and can help on multi-step problems; less answers sooner and is plenty for simple requests. Models without a reasoning mode ignore it. When to change it: turn it down for quick questions and chat, up for maths, planning and tricky code.

  • In Pineido Studio: Thinking effort, on reasoning models only. Default "Auto (recommended)"
  • The other choices: "Off - fastest", "Low", "Medium" and "High - deepest". gpt-oss offers low, medium and high

Which settings to touch, and which to leave alone

Change one setting at a time, write down what you changed, and go back to the defaults if you get lost.

The answer repeats itself

Raise the repeat penalty one step.

The wording is too random

Lower the temperature, or raise min-p a little.

Every answer sounds the same

The opposite: raise the temperature or lower min-p, in small steps.

The answer stops mid-sentence

Raise max tokens, or set it to unlimited.

You want to compare two settings

Fix the seed first, then change one setting.

The answer is wrong on the facts

Sampling won't fix that. Give the model the source text, or try a larger model.

Do this in Pineido Studio

The response settings are in Settings > Assistant > Responses, with a "Reset to defaults" button that puts every one back.

Creativity is picked for each message

You don't set the temperature in Studio. Creativity reads "chosen for each message from what you ask - steady for facts and code, livelier for chat and stories". Studio sets temperature and top-p together by the kind of message: code 0.2 and 0.9, report 0.3 and 0.9, plain answer 0.4 and 0.9 (also used when the kind is unclear), chat 0.7 and 0.9, creative writing 0.8 and 0.95. Tool calls use 0.1 and 0.9.

The sliders and their defaults

  • Word variety (top k): 40, from 0 to 100
  • Min probability (min p): 0.03, from 0 to 0.5
  • Repetition guard (repeat penalty): 1.18, from 1 to 2

Length, seed and memory

  • Response length: -1, which means unlimited
  • Seed: blank, which means random
  • Memory (context window): Auto by default

The same screen in the manual

The manual walks through Settings > Assistant > Responses, setting by setting.

You only reach for the sliders sometimes

Because Pineido Studio picks temperature and top-p for each message, you only touch these sliders when an answer is too samey, too random or repeating. No models are included: you pick one and download it.

The context window, and where to start

The context window costs memory rather than changing word choice, so it has its own page.

The context window

The context window is how much of the conversation the model can see at once. A larger window costs memory and speed.

Answers, without the sales call

Does temperature 0 always give the same answer?

In llama.cpp, temperature 0 takes the top token every time, so the same model, prompt and setup give the same answer. A different runner version or different hardware can still change it.

Should I use top-k, top-p and min-p together?

You can: a token has to pass every filter that is on. It is simpler to steer with one of them and leave the others loose.

What temperature should I use for code?

A low one. For code messages Pineido Studio uses 0.2.

Will a lower temperature make the answers more accurate?

No. It makes the wording steadier, not the facts more right. If the model doesn't know something, give it the source text or try a larger model.

Why can't I set the temperature in Pineido Studio?

Studio chooses it for each message from what you ask: steady for facts and code, livelier for chat and stories. The sliders it does show are word variety, min probability and the repetition guard.

8 Settings explained on this page

Want a local model set up for your work?

Tell us what the model should do and which computers you have.

For one computer, Pineido Studio does this on a USB stick.

You leave with

  1. Tell us the work and the computers
  2. We test models on your own prompts

ByFounder and Imaginator

Pineido, based in Strumica, North Macedonia, is a software company that builds websites, online shops, web and mobile apps, business automation and AI for companies at home and abroad. Work ships in 30-day sprints at a fixed price, and the client owns the code.

Published