Class: LLMModule
Defined in: modules/natural_language_processing/LLMModule.ts:17
Module for managing a Large Language Model (LLM) instance.
Methods
configure()
configure(
config):void
Defined in: modules/natural_language_processing/LLMModule.ts:132
Configures chat and tool calling and generation settings. See Configuring the model for details.
Parameters
config
Configuration object containing chatConfig, toolsConfig, and generationConfig.
Returns
void
delete()
delete():
void
Defined in: modules/natural_language_processing/LLMModule.ts:224
Method to delete the model from memory. Note you cannot delete model while it's generating. You need to interrupt it first and make sure model stopped generation.
Returns
void
deleteMessage()
deleteMessage(
index):Message[]
Defined in: modules/natural_language_processing/LLMModule.ts:183
Deletes all messages starting with message on index position.
After deletion it will call messageHistoryCallback() containing new history.
It also returns it.
Parameters
index
number
The index of the message to delete from history.
Returns
Message[]
- Updated message history after deletion.
forward()
forward(
input,imagePaths?):Promise<string>
Defined in: modules/natural_language_processing/LLMModule.ts:145
Runs model inference with raw input string.
You need to provide entire conversation and prompt (in correct format and with special tokens!) in input string to this method.
It doesn't manage conversation context. It is intended for users that need access to the model itself without any wrapper.
If you want a simple chat with model the consider using sendMessage
Parameters
input
string
Raw input string containing the prompt and conversation history.
imagePaths?
string[]
Optional array of local image paths for multimodal inference. Each entry may be either file:///absolute/path or /absolute/path — the controller normalizes the path before passing it to native code.
Returns
Promise<string>
The generated response as a string.
generate()
generate(
messages,tools?):Promise<string>
Defined in: modules/natural_language_processing/LLMModule.ts:156
Runs model to complete chat passed in messages argument. It doesn't manage conversation context.
For multimodal models, set mediaPath on user messages to include images.
Parameters
messages
Message[]
Array of messages representing the chat history. User messages may include a mediaPath field with a local image path.
tools?
Object[]
Optional array of tools that can be used during generation.
Returns
Promise<string>
The generated response as a string.
getGeneratedTokenCount()
getGeneratedTokenCount():
number
Defined in: modules/natural_language_processing/LLMModule.ts:199
Returns the number of tokens generated in the last response.
Returns
number
The count of generated tokens.
getPromptTokensCount()
getPromptTokensCount():
number
Defined in: modules/natural_language_processing/LLMModule.ts:207
Returns the number of prompt tokens in the last message.
Returns
number
The count of prompt token.
getTotalTokensCount()
getTotalTokensCount():
number
Defined in: modules/natural_language_processing/LLMModule.ts:215
Returns the number of total tokens from the previous generation. This is a sum of prompt tokens and generated tokens.
Returns
number
The count of prompt and generated tokens.
interrupt()
interrupt():
void
Defined in: modules/natural_language_processing/LLMModule.ts:191
Interrupts model generation. It may return one more token after interrupt.
Returns
void
sendMessage()
sendMessage(
message,media?):Promise<Message[]>
Defined in: modules/natural_language_processing/LLMModule.ts:168
Method to add user message to conversation.
After model responds it will call messageHistoryCallback() containing both user message and model response.
It also returns them.
Parameters
message
string
The message string to send.
media?
Optional media object containing a local image path for multimodal models.
imagePath?
string
Returns
Promise<Message[]>
- Updated message history including the new user message and model response.
setTokenCallback()
setTokenCallback(
tokenCallback):void
Defined in: modules/natural_language_processing/LLMModule.ts:119
Sets new token callback invoked on every token batch.
Parameters
tokenCallback
Callback function to handle new tokens.
tokenCallback
(token) => void
Returns
void
fromCustomModel()
staticfromCustomModel(modelSource,tokenizerSource,tokenizerConfigSource,onDownloadProgress?,tokenCallback?,messageHistoryCallback?):Promise<LLMModule>
Defined in: modules/natural_language_processing/LLMModule.ts:94
Creates an LLM instance with a user-provided model binary.
Use this when working with a custom-exported LLM.
Internally uses 'custom' as the model name for telemetry.
Required model contract
The .pte model binary must be exported following the
ExecuTorch LLM export process.
The native runner expects the standard ExecuTorch text-generation interface — KV-cache
management, prefill/decode phases, and logit sampling are all handled by the runtime.
Parameters
modelSource
A fetchable resource pointing to the model binary.
tokenizerSource
A fetchable resource pointing to the tokenizer JSON file.
tokenizerConfigSource
A fetchable resource pointing to the tokenizer config JSON file.
onDownloadProgress?
(progress) => void
Optional callback to monitor download progress, receiving a value between 0 and 1.
tokenCallback?
(token) => void
Optional callback invoked on every generated token.
messageHistoryCallback?
(messageHistory) => void
Optional callback invoked when the model finishes a response, with the full message history.
Returns
Promise<LLMModule>
A Promise resolving to an LLMModule instance.
fromModelName()
staticfromModelName(namedSources,onDownloadProgress?,tokenCallback?,messageHistoryCallback?):Promise<LLMModule>
Defined in: modules/natural_language_processing/LLMModule.ts:47
Creates an LLM instance for a built-in model.
Parameters
namedSources
An object specifying the model name, model source, tokenizer source, tokenizer config source, and optional capabilities.
capabilities?
readonly "vision"[]
modelName
modelSource
tokenizerConfigSource
tokenizerSource
onDownloadProgress?
(progress) => void
Optional callback to monitor download progress, receiving a value between 0 and 1.
tokenCallback?
(token) => void
Optional callback invoked on every generated token.
messageHistoryCallback?
(messageHistory) => void
Optional callback invoked when the model finishes a response, with the full message history.
Returns
Promise<LLMModule>
A Promise resolving to an LLMModule instance.
Example
import { LLMModule, LLAMA3_2_3B } from 'react-native-executorch';
const llm = await LLMModule.fromModelName(LLAMA3_2_3B);