TextToImageModule

TypeScript API implementation of the useTextToImage hook.

API Reference

• For detailed API Reference for TextToImageModule see: TextToImageModule API Reference.
• For all text to image models available out-of-the-box in React Native ExecuTorch see: Text to Image Models.

High Level Overview

import {
  TextToImageModule,
  BK_SDM_TINY_VPRED_256,
} from 'react-native-executorch';

const input = 'a castle';

// Creating an instance and loading the model
const textToImageModule = await TextToImageModule.fromModelName(
  BK_SDM_TINY_VPRED_256
);

// Running the model
const image = await textToImageModule.forward(input);

Methods

All methods of TextToImageModule are explained in details here: TextToImageModule API Reference

Loading the model

Use the static fromModelName factory method. It accepts a model config object (e.g. BK_SDM_TINY_VPRED_256) containing:

• schedulerSource - Location of the used scheduler.
• tokenizerSource - Location of the used tokenizer.
• encoderSource - Location of the used encoder.
• unetSource - Location of the used unet.
• decoderSource - Location of the used decoder.
• inferenceCallback - Optional callback invoked at each denoising step.

And an optional onDownloadProgress callback. It returns a promise resolving to a TextToImageModule instance.

For more information on loading resources, take a look at loading models page.

Running the model

To run the model, you can use the forward method. It accepts four arguments: a text prompt describing the requested image, a size of the image in pixels, a number of denoising steps, and an optional seed value, which enables reproducibility of the results.

The image size must fall within the range from 128 to 512 unless specified differently, and be a multiple of 32 due to the architecture of the U-Net and VAE models.

The seed value should be a positive integer.

Listening for inference steps

To monitor the progress of image generation, you can pass an inferenceCallback in the model config object passed to fromModelName. The callback is invoked at each denoising step (for a total of numSteps + 1 times), yielding the current step index that can be used, for example, to display a progress bar.

Deleting the model from memory

To delete the model from memory, you can use the delete method.