deployment
Top-P Sampling
A text generation strategy that samples from the smallest set of tokens whose cumulative probability exceeds a threshold P. Also called nucleus sampling, top-P dynamically adjusts the number of candidate tokens based on the probability distribution.
In practice
With top-P of 0.9, the model considers only the most likely tokens that together account for 90% of the probability mass.