Interface: GenerationConfig

Defined in: types/llm.ts:379

Object configuring generation settings.

Properties

batchTimeInterval?

optional batchTimeInterval: number

Defined in: types/llm.ts:387

Upper limit on the time interval between consecutive token batches.

---

minP?

optional minP: number

Defined in: types/llm.ts:384

Minimum probability threshold: tokens with prob < minP * max_prob are excluded. 0 disables filtering.

---

outputTokenBatchSize?

optional outputTokenBatchSize: number

Defined in: types/llm.ts:386

Soft upper limit on the number of tokens in each token batch (in certain cases there can be more tokens in given batch, i.e. when the batch would end with special emoji join character).

---

repetitionPenalty?

optional repetitionPenalty: number

Defined in: types/llm.ts:385

Multiplicative penalty applied to logits of recently generated tokens. Values > 1 discourage repetition. 1 disables the penalty.

---

temperature?

optional temperature: number

Defined in: types/llm.ts:380

Scales output logits by the inverse of temperature. Controls the randomness / creativity of text generation.

---

topp?

optional topp: number

Defined in: types/llm.ts:383

Deprecated. Use topP instead.

---

topP?

optional topP: number

Defined in: types/llm.ts:381

Only samples from the smallest set of tokens whose cumulative probability exceeds topP.