Architecting In-Browser Whisper Speech AI: Running ONNX Quantized Speech Recognition with WebAssembly, WebGPU, and Off-Thread Workers

01 The In-Browser Edge AI Speech Paradigm Shift

Traditional web applications transcribing spoken voice capture audio chunks through the browser's MediaStream and stream raw PCM or Opus frames across WebSockets to cloud-hosted ASR servers. This architecture incurs substantial network bandwidth overhead, introduces latency, and exposes spoken dialogue to third-party interception.

In-browser execution reverses this paradigm by downloading pre-quantized ONNX model weights once, caching them in browser CacheStorage / IndexedDB, and executing all neural forward passes locally on CPU/GPU:

02 Off-Thread Web Worker Pipeline & Zero UI Thread Blocking

Audio transformer forward passes involve intense mathematical operations (matrix multiplications across 80-channel mel-spectrogram filterbanks and multi-head attention layers). Executing inference on the browser's main JavaScript thread freezes UI rendering, drops animation frames, and makes the application completely unresponsive.

To ensure a smooth 60fps interface, the execution pipeline runs inside a dedicated background Web Worker (public/whisper-worker.js):

[Main UI Thread]                                            [Background Web Worker]
       │                                                               │
       ├─► User records mic / selects telephony preset                │
       ├─► Decodes audio & renders real-time Canvas Waveform          │
       │                                                               │
       │ (postMessage: { type: 'transcribe', audio, model })           │
       ├──────────────────────────────────────────────────────────────>│
       │                                                               ├─► Downloads ONNX weights
       │                                                               ├─► Computes log-mel spectrogram
       │                                                               ├─► Executes INT8 forward pass
       │ (postMessage: { type: 'result', transcript, chunks, rtf })    │
       │<──────────────────────────────────────────────────────────────┤
       │                                                               │
       ├─► Updates UI transcript, renders timestamps, exports SRT      │
    

03 16kHz Mono PCM Audio Resampling & Oscilloscope Visualization

OpenAI Whisper requires input audio formatted strictly as a single-channel (mono) Float32Array sampled at exactly 16,000 Hz. Browsers, however, capture audio at 44,100 Hz or 48,000 Hz depending on the operating system and hardware microphone.

AIConnect's audio decoder utilizes the Web Audio API's OfflineAudioContext to perform hardware-accelerated linear acoustic downsampling directly in memory before passing buffers to the inference worker. Concurrently, an HTML5 Canvas oscilloscope renders synchronized waveform oscillations during playback.

04 INT8 ONNX Quantization & Real-Time Factor (RTF) Performance

Raw FP32 Whisper models require hundreds of megabytes of memory. Using 8-bit integer quantization (INT8), model footprints shrink dramatically:

• Whisper Tiny English (`Xenova/whisper-tiny.en`): ~39 MB download, sub-100ms inference per audio second.
• Whisper Base (`Xenova/whisper-base`): ~73 MB download, enhanced acoustic precision for accented speech.

The application computes the Real-Time Factor (RTF): $RTF = rac{T_{execution}}{T_{audio}}$. An RTF of 0.20 means 10 seconds of speech are transcribed in just 2 seconds of execution time.

05 Production Web Worker Implementation: Whisper Transformers.js

Below is the core implementation of the dedicated Web Worker managing off-thread pipeline loading and inference:

import { pipeline, env } from '@xenova/transformers';

// Configure Transformers.js to allow local browser caching
env.allowLocalModels = false;
env.useBrowserCache = true;

class PipelineFactory {
  static task = 'automatic-speech-recognition';
  static model = 'Xenova/whisper-tiny.en';
  static instance = null;

  static async getInstance(progress_callback = null) {
    if (this.instance === null) {
      this.instance = pipeline(this.task, this.model, {
        quantized: true,
        progress_callback
      });
    }
    return this.instance;
  }
}

self.addEventListener('message', async (event) => {
  const { type, audio, model } = event.data;

  if (type === 'load') {
    if (model) PipelineFactory.model = model;
    await PipelineFactory.getInstance((data) => {
      self.postMessage({ type: 'progress', data });
    });
    self.postMessage({ type: 'ready' });
  } else if (type === 'transcribe') {
    const transcriber = await PipelineFactory.getInstance();
    const startTime = performance.now();

    const output = await transcriber(audio, {
      chunk_length_s: 30,
      stride_length_s: 5,
      return_timestamps: true
    });

    const elapsedMs = performance.now() - startTime;
    self.postMessage({
      type: 'result',
      transcript: output.text,
      chunks: output.chunks,
      elapsedMs
    });
  }
});

06 Live Testing with AIConnect's In-Browser Whisper Demo

Experience in-browser speech AI live directly in your browser with pre-loaded realistic call center presets: