Skip to main content
Version: 0.10.0

Text-to-Speech (TTS)

The Text-to-Speech extension synthesizes natural, expressive spoken audio waveforms directly on-device from input text.

Speech synthesis operates through multi-stage neural pipelines that combine phonetic transcription or character indexing, duration prediction, acoustic modeling, and neural vocoder audio decompression. Because different model families use fundamentally distinct multi-stage architectures, the library provides dedicated pipelines for each:

  • Supertonic (createSupertonicTextToSpeech): A 4-stage multilingual flow-matching model (44.1 kHz) that coordinates a text encoder, duration predictor, vector estimator, and vocoder with multi-speaker voice style conditioning.
  • Kokoro (createKokoroTextToSpeech): A 2-stage phoneme-driven model (24 kHz) that pairs language-specific grapheme-to-phoneme (G2P) transcription with a duration predictor, acoustic synthesizer, and voice embedding matrices.

Both pipelines are wrapped uniformly by the useTextToSpeech React hook.

iOSAndroid

Quick Start​

The useTextToSpeech hook manages downloading all sub-model weights, phonemizers, and voice files. To achieve low Time-to-First-Audio (TTFA) and seamless gapless playback, pipe the generated chunks directly to an audio buffer queue such as react-native-audio-api:

import { useState } from 'react';
import { models, useTextToSpeech, KOKORO_SAMPLE_RATE } from 'react-native-executorch';
import { useAudioPlayer } from './hooks/useAudioPlayer'; // Custom helper hook built with react-native-audio-api

function SpeechComponent() {
  const [prompt, setPrompt] = useState('');
  const tts = useTextToSpeech(models.textToSpeech.KOKORO.EN_US.DEFAULT);
  const player = useAudioPlayer(KOKORO_SAMPLE_RATE); // 24000 Hz

  // Hook state:
  // tts.isReady          — true once all sub-models and voice assets are loaded
  // tts.downloadProgress — 0 to 100 download progress across all files
  // tts.error            — Error instance if download or load failed
  // tts.resource         — resolved config with all URLs replaced by local file paths

  const handleSpeak = async () => {
    if (!tts.isReady || !tts.synthesize || !prompt.trim()) return;

    // Start synthesis stream (yielding sentence-by-sentence chunks)
    const chunksStream = tts.synthesize(prompt, { voice: 'af_heart' });

    // Stream chunks directly into the audio buffer queue for instant playback
    await player.playStream(chunksStream);
  };

  const handleStop = () => {
    tts.synthesizeStop?.(); // Abort background generation
    player.stop(); // Clear audio buffers and stop playback
  };
}
Full Interactive Example in Gallery App

See src/app/(screens)/text-to-speech.tsx in the React Native ExecuTorch Gallery for a complete, runnable screen featuring voice selection, buffer queue streaming, Time-to-First-Audio (TTFA) benchmarking, and live waveform visualization.

Output Format​

synthesize() returns an AsyncGenerator yielding audio chunks (KokoroTtsChunk or SupertonicTtsChunk) sequentially as each sentence finishes synthesis:

type TtsChunk = {
  /** Float32 PCM audio samples normalized in [-1.0, 1.0] */
  readonly audio: Float32Array;
  /** Audio sampling rate in Hz (44100 for Supertonic, 24000 for Kokoro) */
  readonly sampleRate: number;
  /** Duration of this synthesized chunk in seconds */
  readonly duration: number;
  /** Zero-based index of this chunk */
  readonly chunkIndex: number;
  /** Total number of chunks partitioned from the input text */
  readonly totalChunks: number;
};

How Streaming Works​

On-device text-to-speech is built for instant audio feedback:

  • Sentence-by-Sentence Streaming: Long text is automatically split into natural phrases. Instead of waiting for an entire paragraph to finish generating, audio chunks are yielded one by one as each sentence is synthesized.
  • Immediate Playback: Your app can start playing the first sentence right away while subsequent sentences are generated seamlessly in the background.
  • Smooth UI: Audio generation runs in the background so your app's user interface and animations stay completely smooth.
  • Cancellation: Calling synthesizeStop() signals the generator to stop, halting synthesis before subsequent chunks are computed.

Imperative Pipelines​

For background services, audio workers, or non-React component logic, you can instantiate the pipelines imperatively using createSupertonicTextToSpeech or createKokoroTextToSpeech:

import { createKokoroTextToSpeech, download, models } from 'react-native-executorch';

// Download and cache remote model weights, phonemizers, and voice files
const model = await download(models.textToSpeech.KOKORO.EN_US.DEFAULT);
const tts = await createKokoroTextToSpeech(model);

try {
  const chunksStream = tts.synthesize('Hello from offline text-to-speech!', {
    voice: 'af_heart',
    speed: 1.0,
  });

  for await (const chunk of chunksStream) {
    console.log(`Chunk generated: ${chunk.duration.toFixed(2)}s`);
  }
} finally {
  // Always release native model memory and buffers when done
  tts.dispose();
}

Available Models​

The library provides ready-to-use Text-to-Speech models from the Software Mansion HuggingFace Text to Speech Collection, pre-packaged with neural G2P phonemizers and voice presets in models.textToSpeech:

ModelVariantsSub-Models & AssetsSample RateSize RangeSupported BackendsSupported LanguagesNotes
Supertonic 3SeeText Encoder, Duration Predictor, Vector Estimator, Vocoder, Voice Styles44.1 kHz398 MBXNNPACK (CPU), MLX (Apple), Vulkan (Android)English, Spanish, French, German, Korean, Japanese, Chinese & moreFaster generation and broad multilingual coverage across 10 bundled speaker styles, with slightly lower voice naturalness than Kokoro.
Kokoro (Language Packages)SeeDuration Predictor, Synthesizer, G2P Lexicon / Neural Model, Language Voices24.0 kHz332 MB eachXNNPACK (CPU), Core ML (Apple)English (US/GB), Spanish, French, Italian, Portuguese, Hindi, Polish, GermanExceptional voice naturalness and intonation per language package, but heavier compute per synthesized chunk.
Model-Specific Pipelines

Because Text-to-Speech architectures require distinct multi-model orchestration in TypeScript (coordinating phonemizers, duration predictors, flow-matching loops, and neural vocoders), TTS pipelines are model-specific. To use custom voices or fine-tuned weights, provide your modified .pte models or custom voice JSON/BIN files matching the SupertonicTtsModel or KokoroTtsModel specifications.

API Reference​

Hooks & Pipelines​

Types & Options​

Constants & Model Presets​

Source Code

View the implementation on GitHub:

We are Software Mansion.

Hire us