Optical Character Recognition (OCR)

Optical Character Recognition (OCR) detects and extracts text from images. The pipeline identifies text regions with oriented quadrilateral boundaries (Quad) and transcribes their characters in reading order (top-to-bottom, left-to-right columns).

It is used for document digitizing, receipt scanning, license plate reading, sign translation, and invoice processing. Because inference runs entirely on-device with zero network latency, sensitive documents never leave the phone.

iOS	Android

Quick Start

The useOpticalCharacterRecognizer hook manages model downloading, character set loading, and lifecycle:

import { models, useOpticalCharacterRecognizer } from 'react-native-executorch';
import type { ImageBuffer } from 'react-native-executorch/cv';

function MyComponent() {
  const ocr = useOpticalCharacterRecognizer(models.ocr.PADDLE.PPOCRV6_SMALL.DEFAULT);

  // Hook state:
  // ocr.isReady          — true once model and charset are downloaded and loaded
  // ocr.downloadProgress — 0 to 100 download progress
  // ocr.error            — Error instance if download or load failed
  // ocr.resource         — resolved config with all URLs replaced by local file paths

  const handleRecognize = async (imageBuffer: ImageBuffer) => {
    if (!ocr.isReady || !ocr.recognizeCharacters) return;

    // Run inference on background thread
    const textLines = await ocr.recognizeCharacters(imageBuffer, {
      confidenceThreshold: 0.5,
    });
    console.log('Recognized lines:', textLines);
  };

  // Trigger handleRecognize from an image picker, button press, or camera frame
}

Full Interactive Example in Gallery App

See src/app/(screens)/ocr.tsx in the React Native ExecuTorch Gallery for a complete, runnable screen with photo picker, oriented text bounding boxes, and latency tracking.

Output Format

recognizeCharacters() returns an array of OcrDetection objects in natural reading order:

type OcrDetection = {
  /** Transcribed text string */
  readonly text: string;
  /** Mean per-character probability score (between 0.0 and 1.0) */
  readonly confidence: number;
  /**
   * Oriented quadrilateral corners in pixel coordinates:
   * top-left, top-right, bottom-right, bottom-left
   */
  readonly quad: Quad;
};

Example result:

[
  {
    text: 'RECEIPT TOTAL: $42.50',
    confidence: 0.96,
    quad: [
      { x: 45.0, y: 120.5 },
      { x: 380.2, y: 122.0 },
      { x: 380.0, y: 155.4 },
      { x: 44.8, y: 154.0 },
    ],
  },
];

Configuration & Options

Pass a RecognizeCharactersOptions object to recognizeCharacters():

Option	Type	Default	Description
confidenceThreshold	number	0.5	Minimum mean confidence score for a text region to be returned.

Imperative API

For background processing, document scanners, or manual lifecycle management outside React components, create the pipeline using createPaddleOcr:

import { createPaddleOcr, download, models } from 'react-native-executorch';

// Download and cache model assets before creating the pipeline
const model = await download(models.ocr.PADDLE.PPOCRV6_SMALL.DEFAULT);
const ocr = await createPaddleOcr(model);

try {
  const lines = await ocr.recognizeCharacters(imageBuffer, {
    confidenceThreshold: 0.5,
  });
  console.log('Recognized text:', lines.map((l) => l.text).join('\n'));
} finally {
  // Always release native resources when finished
  ocr.dispose();
}

Synchronous Execution

For high-throughput loops or live camera text detection, createPaddleOcr exposes a synchronous recognizeCharactersWorklet function. This runs directly on the worklet thread with zero Promise scheduling overhead:

// Called synchronously inside a worklet runtime
const lines = ocr.recognizeCharactersWorklet(frameBuffer, {
  confidenceThreshold: 0.5,
});

See Worklets & Threading for details on worklet execution contexts and zero-copy host objects.

Available Models

The library provides mixed-precision fused PP-OCRv6 models from the Software Mansion HuggingFace OCR Collection, available in models.ocr:

Model Family	Variants	Size Range	Supported Backends	Notes
PP-OCRv6 Small	See	7.9 MB – 25.0 MB	XNNPACK (CPU), Core ML (Apple), Vulkan (Android)	Full end-to-end on-device text detection & recognition in a single pipeline.

Legacy CRAFT Models & Future EasyOCR Package

The HuggingFace OCR collection may also list legacy CRAFT text detector models. Direct CRAFT support has been deprecated in core react-native-executorch in favor of the significantly faster and lighter fused PP-OCRv6 pipeline. Advanced EasyOCR-style recognition features will be introduced in a dedicated companion package.

Using Custom Models

To use your own custom-trained PaddleOCR .pte model and character set, pass a PaddleOcrModel configuration object to useOpticalCharacterRecognizer or createPaddleOcr:

const customOcr = await createPaddleOcr({
  modelPath: 'https://example.com/my-ocr.pte',
  charsetPath: 'https://example.com/charset.json',
  modelOpts: {
    defaultConfidenceThreshold: 0.5,
  },
});

The pipeline automatically verifies that the model's exported input and output shapes match its requirements. To prepare and export your own .pte model to match this pipeline, see Exporting Custom Models.

API Reference

Hooks & Pipelines

• useOpticalCharacterRecognizer() — React hook for OCR model downloading, state, and lifecycle.
• createPaddleOcr() — Imperative factory for PP-OCRv6 pipelines.

Types & Options

• PaddleOcr — OCR runner interface (recognizeCharacters, recognizeCharactersWorklet, dispose).
• OcrDetection — Single recognized text line with text, confidence, and quad.
• RecognizeCharactersOptions — Inference options (confidenceThreshold).
• PaddleOcrModel — Model configuration spec with modelPath and charsetPath.
• PaddleOcrModelOptions — Model options (defaultConfidenceThreshold).
• Quad — Oriented 4-corner polygon tuple [Point, Point, Point, Point] in pixel coordinates (top-left, top-right, bottom-right, bottom-left).
• ImageBuffer — Input image buffer structure.

Model Presets

• models.ocr — Pre-configured OCR models registry.

Source Code

View the implementation on GitHub:

• src/extensions/cv/tasks/paddleOcr.ts ↗