ModelSamplingParams interface

Represents a set of parameters used to control the sampling behavior of a language model during text generation.

Properties

max_completion_tokens	The maximum number of tokens allowed in the completion.
seed	The random seed for reproducibility. Defaults to 42.
temperature	The temperature parameter for sampling. Defaults to 1.0.
top_p	The top-p parameter for nucleus sampling. Defaults to 1.0.

The maximum number of tokens allowed in the completion.

max_completion_tokens?: number

number

The random seed for reproducibility. Defaults to 42.

seed?: number

number

The temperature parameter for sampling. Defaults to 1.0.

temperature?: number

number

The top-p parameter for nucleus sampling. Defaults to 1.0.

top_p?: number

number