How AI Background Removal Works: WebAssembly, ONNX & Deep Neural Networks

Have you ever wondered what happens behind the scenes when an AI model separates a person's hair or a sneaker's laces from a cluttered background in a split second?

In this technical breakdown, we look at the mathematical foundations, neural network architectures, and modern browser APIs that power cutting-edge on-device image segmentation.

1. Salient Object Detection (SOD) & Dichotomous Image Segmentation (DIS)

Background removal is technically an instance of **Salient Object Detection (SOD)** combined with **Dichotomous Image Segmentation (DIS)**.

Unlike semantic segmentation (which categorizes every pixel into arbitrary classes like "car", "road", or "sky"), DIS focuses on identifying the primary subject intended by the photographer and generating an **alpha matte** $\alpha \in [0, 1]$ where: - $\alpha = 1.0$ corresponds to 100% opaque foreground. - $\alpha = 0.0$ corresponds to 100% transparent background. - Values between $0 < \alpha < 1$ represent semi-transparent regions like hair, smoke, glass, or motion-blurred edges.

Modern architectures like **IS-Net (Intermediate Supervision Network)** use multi-scale feature extractors with deep intermediate supervision, forcing early layers to capture high-frequency spatial boundaries while deeper layers understand semantic context.

2. The Shift from Python Servers to In-Browser WebAssembly

Historically, running an IS-Net or U2-Net model required a Python backend with PyTorch, CUDA, and an NVIDIA A10G GPU. Each request incurred cold starts, high GPU server bills, and privacy liability.

Today, three open web standards allow us to run neural networks directly on the client:

1. **ONNX Runtime Web:** The Open Neural Network Exchange runtime compiled directly to WebAssembly with SIMD (Single Instruction Multiple Data) optimizations. 2. **WebGPU / WebGL Shaders:** Math operations (matrix multiplication and 2D convolutions) are delegated to the user's local GPU hardware via WebGPU compute shaders. 3. **WebWorkers:** Offloads tensor computations to background threads so the 60fps UI thread remains responsive and smooth.

3. Image Compositing & Edge Refinement

Once the alpha matte is generated: ``` Output_RGBA = (Foreground_RGB * Alpha) + (NewBackground_RGB * (1 - Alpha)) ```

The result is drawn onto an offscreen HTML5 2D Canvas where users can dynamically composite custom solid colors, multi-stop CSS gradients, or real-time gaussian blur layers before exporting a lossless 32-bit PNG file.

Check out our [FAQ page](/faq) for more technical details on browser compatibility and hardware acceleration.