A FIELD GUIDE TO LOW-PRECISION AI
A smaller
footprint
for AI.
Big ideas, fewer bits. Explore the small number format helping neural networks do more with less.
Look inside FP8FORMAT / 008FLOATING POINT
8 BITS
One byte. Many possibilities.
One byte. Many possibilities.
01 / UNDER THE HOOD
Every bit has a job.
FP8 uses eight bits to encode a number. How those bits are divided changes the balance between precision and range.
THE SAME VALUE: +1.0
00111000
Sign 1 bit Exponent 4 bits Mantissa 3 bits
A little more precision.
E4M3 gives three bits to the mantissa, with a maximum finite magnitude of 448. A common choice for weights and forward activations.
02 / KEEP EXPLORING
Small format.
Deep subject.
Lower precision is a design choice. Scaling, hardware support, and accuracy checks all matter.
From bits to
neural networks
Read the FP8 primer: formats, scaling, and mixed-precision training.
Read the technical guide