Image Classification

Image classification analyzes an input image and predicts the most likely visual categories it belongs to, along with confidence scores for each prediction. Unlike object detection (which locates multiple items with bounding boxes), classification evaluates the image as a whole.

It is ideal for visual search, photo organization, quality inspection, and accessibility tagging. Because inference runs entirely on-device, images never leave the user's phone.

iOS	Android

Quick Start

The useClassifier hook handles model downloading, initialization, and lifecycle management:

import { models, useClassifier } from 'react-native-executorch';
import type { ImageBuffer } from 'react-native-executorch/cv';

function MyComponent() {
  const classifier = useClassifier(models.classification.EFFICIENTNET_V2_S.DEFAULT);

  // Hook state:
  // classifier.isReady          — true once model is downloaded and loaded in memory
  // classifier.downloadProgress — 0 to 100 download progress
  // classifier.error            — Error instance if download or load failed
  // classifier.resource         — resolved config with all URLs replaced by local file paths

  const handleClassify = async (imageBuffer: ImageBuffer) => {
    if (!classifier.isReady || !classifier.classify) return;

    // Run inference on background thread
    const predictions = await classifier.classify(imageBuffer, { topk: 3 });
    console.log('Top prediction:', predictions[0]);
  };

  // Trigger handleClassify from an image picker, button press, or camera frame
}

Full Interactive Example in Gallery App

See src/app/(screens)/image-classification.tsx in the React Native ExecuTorch Gallery for a complete, runnable screen with photo picker, result overlays, and latency tracking.

Output Format

classify() returns an array of Classification objects sorted from highest to lowest confidence:

type Classification<L = string> = {
  /** The predicted class label string */
  readonly label: L;
  /** Normalized confidence score between 0.0 and 1.0 */
  readonly confidence: number;
};

Example result:

[
  { label: 'golden_retriever', confidence: 0.912 },
  { label: 'cocker_spaniel', confidence: 0.043 },
  { label: 'labrador_retriever', confidence: 0.018 },
];

Configuration & Options

Pass a ClassifyOptions object to classify():

Option	Type	Default	Description
topk	number	undefined	Maximum number of top-scoring predictions to return. When omitted, returns all classes in the vocabulary.

Imperative API

For background jobs, headless services, or manual lifecycle management outside React components, instantiate the pipeline directly with createClassifier:

import { createClassifier, download, models } from 'react-native-executorch';

// Download and cache model assets before creating the pipeline
const model = await download(models.classification.EFFICIENTNET_V2_S.DEFAULT);
const classifier = await createClassifier(model);

try {
  const results = await classifier.classify(imageBuffer, { topk: 5 });
  console.log('Top prediction:', results[0]);
} finally {
  // Always release native resources when finished
  classifier.dispose();
}

Synchronous Execution

For high-throughput loops like camera frame processors, createClassifier exposes a synchronous classifyWorklet function. This executes directly inside a worklet runtime without Promise scheduling overhead:

// Called synchronously inside a VisionCamera frame processor on the UI worklet thread
const results = classifier.classifyWorklet(frameBuffer, { topk: 1 });

See Worklets & Threading for details on dispatching tasks and sharing models across threads.

Available Models

The library provides ready-to-use models from the Software Mansion HuggingFace Classification Collection, pre-configured with ImageNet-1k vocabulary and normalization parameters in models.classification:

Model Family	Variants	Size Range	Supported Backends	Dataset / Vocabulary	Notes
EfficientNetV2-S	See	21.9 MB – 81.7 MB	XNNPACK (CPU), Core ML (Apple)	IMAGENET1K_LABELS (1,000 classes)	Fast, lightweight general image recognition and tagging on mobile.

Using Custom Models

To use your own fine-tuned classification .pte model, pass a ClassifierModel configuration object to useClassifier or createClassifier:

const customClassifier = await createClassifier({
  modelPath: 'https://example.com/my-model.pte',
  modelOpts: {
    resizeMode: 'stretch',
    interpolation: 'linear',
    normalizeOpts: { alpha: 1 / 255.0, beta: 0.0 },
    labels: ['cat', 'dog', 'bird'],
  },
});

The pipeline automatically verifies that the model's exported input and output shapes match its requirements. To prepare and export your own .pte model to match this pipeline, see Exporting Custom Models.

API Reference

Hooks & Pipelines

• useClassifier() — React hook for model downloading, inference state, and automatic memory cleanup.
• createClassifier() — Imperative factory for background jobs, services, and worklet execution.

Types & Options

• Classifier — Classifier task runner interface with classify and classifyWorklet.
• Classification — Result prediction object with label and confidence.
• ClassifyOptions — Configuration options for the classify call (topk).
• ClassifierModel — Model configuration spec for custom and preset models.
• ClassifierOptions — Preprocessing and label vocabulary configuration.
• ImageBuffer — Input image buffer structure (data, width, height, format).

Model Presets

• models.classification — Pre-configured classification models registry.

Source Code

View the implementation on GitHub:

• src/extensions/cv/tasks/classification.ts ↗