ImageEmbeddingsModule

TypeScript API implementation of the useImageEmbeddings hook.

Reference

import {
  ImageEmbeddingsModule,
  CLIP_VIT_BASE_PATCH32_IMAGE,
} from 'react-native-executorch';

// Creating an instance
const imageEmbeddingsModule = new ImageEmbeddingsModule();

// Loading the model
await imageEmbeddingsModule.load(CLIP_VIT_BASE_PATCH32_IMAGE);

// Running the model
const embedding = await imageEmbeddingsModule.forward(
  'https://url-to-image.jpg'
);

Methods

Method	Type	Description
load	(model: { modelSource: ResourceSource }, onDownloadProgressCallback?: (progress: number) => void): Promise<void>	Loads the model, where modelSource is a string that specifies the location of the model binary.
forward	(imageSource: string): Promise<Float32Array>	Executes the model's forward pass, where imageSource is a URI/URL to image that will be embedded.
onDownloadProgress	(callback: (downloadProgress: number) => void): any	Subscribe to the download progress event.

Type definitions

type ResourceSource = string | number | object;

Loading the model

To load the model, use the load method. It accepts an object:

model - Object containing the model source.

• modelSource - A string that specifies the location of the model binary.

onDownloadProgressCallback - (Optional) Function called on download progress.

This method returns a promise, which can resolve to an error or void.

For more information on loading resources, take a look at loading models page.

Running the model

It accepts one argument, which is a URI/URL to an image you want to encode. The function returns a promise, which can resolve either to an error or an array of numbers representing the embedding.