Neural Style Transfer

Neural style transfer renders an input image in the artistic style of another image (such as famous paintings or pattern textures) while preserving the semantic content and structure of the original photo.

Because the models run locally in real time on mobile hardware accelerators, you can apply artistic filters to live camera frames or photos without uploading user media to external servers.

iOS	Android

Quick Start

The useStyleTransfer hook manages model downloading, initialization, and lifecycle:

import { models, useStyleTransfer } from 'react-native-executorch';
import type { ImageBuffer } from 'react-native-executorch/cv';

function MyComponent() {
  const styler = useStyleTransfer(models.styleTransfer.CANDY.DEFAULT);

  // Hook state:
  // styler.isReady          — true once model is downloaded and loaded in memory
  // styler.downloadProgress — 0 to 100 download progress
  // styler.error            — Error instance if download or load failed
  // styler.resource         — resolved config with all URLs replaced by local file paths

  const handleTransfer = async (imageBuffer: ImageBuffer) => {
    if (!styler.isReady || !styler.transferStyle) return;

    // Run inference on background thread
    const styledBuffer = await styler.transferStyle(imageBuffer);
    console.log('Styled image dimensions:', styledBuffer.width, styledBuffer.height);
  };

  // Trigger handleTransfer from an image picker, button press, or camera frame
}

Full Interactive Example in Gallery App

See src/app/(screens)/style-transfer.tsx in the React Native ExecuTorch Gallery for a complete, runnable screen with photo picker, side-by-side style comparisons, and latency tracking.

Output Format

transferStyle() returns an ImageBuffer object containing the styled RGBA image rendered at the input dimensions:

type ImageBuffer = {
  readonly width: number;
  readonly height: number;
  readonly format: 'rgba';
  readonly data: Uint8Array;
};

The resulting buffer contains raw uncompressed RGBA pixel bytes that can be rendered directly via React Native Skia or passed into subsequent processing steps.

Imperative API

For background photo processing, headless workflows, or manual lifecycle management outside React components, create the pipeline using createStyleTransfer:

import { createStyleTransfer, download, models } from 'react-native-executorch';

// Download and cache model assets before creating the pipeline
const model = await download(models.styleTransfer.CANDY.DEFAULT);
const styler = await createStyleTransfer(model);

try {
  const styledBuffer = await styler.transferStyle(imageBuffer);
  console.log('Styled output byte length:', styledBuffer.data.byteLength);
} finally {
  // Always release native resources when finished
  styler.dispose();
}

Synchronous Execution

For high-throughput loops like live viewfinder styling or video recording, createStyleTransfer exposes a synchronous transferStyleWorklet function. This runs directly on the worklet thread with zero Promise scheduling overhead:

// Called synchronously inside a VisionCamera frame processor on the UI worklet thread
const styledBuffer = styler.transferStyleWorklet(frameBuffer);

See Worklets & Threading for details on worklet execution contexts and zero-copy host objects.

Available Models

The library provides ready-to-use style transfer models from the Software Mansion HuggingFace Style Transfer Collection, available in models.styleTransfer:

Model Family	Variants	Size Range	Supported Backends	Notes
Candy	See	1.8 MB – 6.5 MB	XNNPACK (CPU), Core ML (Apple)	Vibrant, colorful candy aesthetic with bold outlines.
Mosaic	See	1.8 MB – 6.5 MB	XNNPACK (CPU), Core ML (Apple)	Classical geometric tile mosaic texture.
Rain Princess	See	1.8 MB – 6.5 MB	XNNPACK (CPU), Core ML (Apple)	Painterly expressionist oil painting style.
Udnie	See	1.8 MB – 6.5 MB	XNNPACK (CPU), Core ML (Apple)	Francis Picabia abstract modernist art style.

Using Custom Models

To use your own trained feed-forward style transfer .pte model, pass a StyleTransferModel configuration object to useStyleTransfer or createStyleTransfer:

const customStyler = await createStyleTransfer({
  modelPath: 'https://example.com/my-style.pte',
  modelOpts: {
    resizeMode: 'stretch',
    interpolation: 'linear',
    outInterpolation: 'lanczos',
    normalizeOpts: { alpha: 1 / 255.0, beta: 0.0 },
    outNormalizeOpts: { alpha: 255.0, beta: 0.0 },
  },
});

The pipeline automatically verifies that the model's exported input and output shapes match its requirements. To prepare and export your own .pte model to match this pipeline, see Exporting Custom Models.

API Reference

Hooks & Pipelines

• useStyleTransfer() — React hook for style transfer model downloading, state, and lifecycle.
• createStyleTransfer() — Imperative factory for style transfer pipelines.

Types & Options

• StyleTransfer — Style transfer runner interface (transferStyle, transferStyleWorklet).
• StyleTransferModel — Model configuration spec for style transfer models.
• StyleTransferOptions — Options defining normalization, interpolation, and resize modes.
• ImageBuffer — Input and output image buffer structure.

Model Presets

• models.styleTransfer — Pre-configured artistic style transfer models registry.

Source Code

View the implementation on GitHub:

• src/extensions/cv/tasks/styleTransfer.ts ↗