Temperature vs Top-P in Amazon Bedrock for AIF-C01

Temperature changes how spread out the model’s next-word probabilities are. Top-p cuts the list of next-word candidates down to the most likely ones before the model picks. Lower temperature makes answers steadier and more repeatable. Higher temperature makes them more varied. Lower top-p removes unlikely words from the running. Neither one changes what the model knows, and neither one changes the price of a call.

That is the whole idea. The rest of this post shows it with numbers, so the exam wording stops being confusing.

What does a model actually do when it picks the next word?

A model does not write a sentence in one go. It builds the answer one token at a time. A token is a small piece of text, often a word or part of a word (see tokens, embeddings and the model lifecycle for more).

At each step the model scores every possible next token and turns the scores into probabilities. Then it samples from them. AWS describes it this way in its Bedrock guide: for any given sequence, the model determines a probability distribution of options for the next token, and it samples from that distribution to generate each token.

Temperature and top-p are two ways to change that distribution before the sample is drawn. As of October 2026, the Bedrock documentation lists them under “randomness and diversity”, next to top-k.

What does temperature change?

Temperature changes the shape of the probability curve. A lower value steepens it, so the most likely token wins more often. A higher value flattens it, so unlikely tokens get more of a chance. AWS puts it as: lower temperature leads to more deterministic responses, higher temperature to more random ones.

Here is a small example. It uses the same three candidates the AWS documentation uses for the prompt I hear the hoof beats of ":

The last two columns come from a common way of applying temperature: divide each token’s raw score by the temperature, then renormalise. For the numbers here that is the same as raising each probability to the power 1 divided by the temperature, then scaling so they add up to 1. Treat the exact figures as an illustration. Real models do this on raw scores, and the Bedrock docs only describe the effect in words.

Read the table the way the exam wants you to read it. At 0.5, “horses” goes from 70% to about 91%. At 2, “unicorns” goes from 10% to about 20%. The candidates are the same. Only their odds moved.

What does top-p change?

Top-p does not reshape the curve. It cuts the list. The model sorts the candidates from most to least likely, keeps the smallest group whose probabilities add up to the top-p value, and ignores the rest.

With the same three tokens:

This matches the AWS example: at 0.7 the model considers only “horses”, and at 0.9 it considers “horses” and “zebras”. The size of the kept group changes from token to token. When the model is very sure, the group may hold one token. When it is unsure, it may hold dozens. That is the main difference from top-k, covered below.

What is the difference between temperature and top-p?

Use this when a question asks you to tell them apart:

A short way to remember it: temperature changes the odds, top-p changes who is allowed to play.

And what about top-k?

Top-k is the third setting in the same family. It keeps a fixed number of the most likely tokens. Top-k of 2 on the example keeps “horses” and “zebras”. Top-p keeps a share of the probability instead of a fixed count, so its group size moves with the model’s confidence. AWS notes that top-k, top-p and temperature are commonly found across models, but not every model supports every one, and the default values and ranges depend on the model.

Which setting should you reach for?

Start from the job, not the dial.

• Extracting fields from an invoice, classifying a support ticket, answering from a policy document: use a low temperature. You want the same answer each time.
• Brainstorming product names, drafting marketing copy, writing varied examples: use a higher temperature. You want variety.
• Output that sometimes goes off the rails with odd words: lower top-p to cut the rare tokens, instead of making everything bland.

A practical habit is to change one setting at a time. Both settings narrow or widen the same pool, so moving two at once makes it hard to know which one caused a change. Check the model’s own page in the Bedrock documentation for its defaults and allowed ranges before you tune.

How do you set them in Bedrock?

With the Converse API, the same settings go in one place for every model that supports them: the inferenceConfig block. It holds temperature, topP, maxTokens and stopSequences. A call with a steady setup looks like this:

import boto3

client = boto3.client("bedrock-runtime")

response = client.converse(
    modelId="YOUR_MODEL_ID",
    messages=[{"role": "user", "content": [{"text": "List three AWS storage services."}]}],
    inferenceConfig={
        "temperature": 0.2,
        "topP": 0.9,
        "maxTokens": 300,
    },
)
print(response["output"]["message"]["content"][0]["text"])
Copy

Replace YOUR_MODEL_ID with a model you have access to in your Region. Settings that are not part of the shared block can go in additionalModelRequestFields, because some models have their own extras.

Notice maxTokens. It belongs to a different group, which AWS calls length. It caps the response size and stops long answers. It does not make the output more or less random. Stop sequences are in the same group: they end the answer when the model writes a chosen piece of text. If you want to see what a call costs, pricing one Amazon Bedrock prompt from the token counts it returns walks through it. Temperature and top-p alone do not change the token price. A longer answer does.

How does the AIF-C01 exam ask about this?

The exam is multiple choice and asks about ideas, not code. Expect a short scenario and four answers. The usual shapes:

1. “A company wants consistent, repeatable answers from a model. What should it change?” Lower the temperature.
2. “A team wants more creative, varied text.” Raise the temperature.
3. “Which parameter limits the model to the most probable share of tokens?” Top-p.
4. “Which parameter limits how long the response is?” Maximum tokens (response length), not temperature.

Two traps are worth knowing:

• Temperature is not accuracy. A low temperature makes the answer steadier. It does not make it true. A model can repeat a wrong answer every time. For made-up answers you need grounding, such as the approach in measuring whether a RAG answer is actually grounded, not a lower dial.
• These are inference settings, not training. You change them on each call. They do not teach the model anything, and they are not fine-tuning. If a question says “without retraining”, this is the family of answers it has in mind.

A good habit with any parameter question: say in your head what the setting touches. Odds, candidate list, length, or the model itself. Only one of those four matches most questions.

Quick self-check

Answer these without looking back:

1. You halve the temperature. Does the set of possible next tokens change? (No. Only the odds do.)
2. You set top-p to 0.5 and the top token already has probability 0.6. How many tokens are kept? (One.)
3. A response keeps getting cut off mid-sentence. Which setting do you check? (Maximum tokens.)
4. A bank wants the same answer every time to a policy question. Which direction do you move temperature? (Down.)

Next step

Chapter 8 of the AWS AI Practitioner course on this site is the page to open next: AIF-C01 chapter 8. After that, try the AWS knowledge quiz to see whether the wording of these questions still trips you, and the AIF-C01 practice exam when you want the full format. The AWS AI Practitioner overview lists every chapter in order.

For the source of the definitions above, read AWS’s own page, Influence response generation with inference parameters.