temperature is a numerical parameter that regulates the degree of randomness in a language model's responses. Technically, it adjusts the probability distribution over the possible next words: low values make the model almost always choose the most probable options, while high values distribute the probability more evenly, allowing for less predictable outputs.
It usually ranges from 0 to 2, and its choice depends on the task:
- Low values (0–0.3): deterministic and precise responses, ideal for data extraction, code, or factual summaries.
- Medium values (0.5–0.8): balance between coherence and variety, useful for general assistance.
- High values (1–2): greater creativity and diversity, suitable for literary writing or brainstorming.
In practice, it is worth remembering that a high temperature does not improve the model's "knowledge," only its variability: it can increase both originality and the risk of incoherent or invented responses. For reproducibility, it is usually set to 0 or a very close value.