What does “normal random sampling” mean in LLM text generation?

I am working on research involving the numerical behavior of LLM token sampling, and I need to reproduce the ordinary stochastic next-token sampling process as faithfully as possible. At each decoding step, I obtain the model logits and convert them to probabilities: logits = model(...).logits[:, -1, :] probs = torch.softmax(logits, dim=-1) I can then sample the next token in at least two ways. Using PyTorch: next_token = torch.multinomial(probs, num_samples=1) or by explicitly implementin...