Quantization

Quantization introduces an irreversible approximation: each continuous value is rounded to the nearest representable digital level. The maximum quantization error is ±½ LSB (Least Significant Bit). For an N-bit system, the quantization step size is (full range) / 2^N. This error manifests as banding in images (posterization) and as staircase artifacts in intensity profiles.

The quantization noise floor sets a theoretical limit on measurement precision. For uniformly distributed quantization error, the RMS noise is approximately (step size) / √12.

Quantization determines the finest intensity difference that can be distinguished in your data. If two cell populations differ by less than one quantization step, no amount of post-processing can separate them. This is why matching the ADC bit depth to the sensor's actual dynamic range is essential for quantitative analysis.

At 8-bit (256 levels), the minimum distinguishable intensity difference is 1/256 ≈ 0.4% of full range. At 16-bit, it's 1/65,536 ≈ 0.0015%. For ratio-metric measurements like FRET efficiency, where differences of 1-2% are biologically significant, 8-bit quantization is insufficient — 12-bit or higher is required.