Function: createLLMRunner()

createLLMRunner(modelPath, tokenizerPath, modalities?): LLMRunner

Defined in: extensions/llm/llmRunner.ts:164

Experimental

Creates a native ExecuTorch LLM runner instance. This API is experimental and might change in future releases. It relies on experimental ExecuTorch runtime extensions and injected member-pointer accessors to manage KV cache state that may evolve across releases.

Parameters

modelPath

string

Path to the local .pte model file.

tokenizerPath

string

Path to the local tokenizer configuration file (e.g. tokenizer.json).

modalities?

readonly ("image" | "audio")[]

List of supported input non-text modalities (e.g. ['image']). When omitted, defaults to text-only.

Returns

LLMRunner

A native LLMRunner instance.