Extropic Introduces Z1T Sparse Models for Probabilistic Hardware

Extropic has released a technical overview of Z1T, a new family of transformer-like models specifically designed for its Z1 probabilistic sub-threshold CMOS hardware. This work represents an initial study in co-designing sparse neural networks and thermodynamic computing, aiming for significant energy efficiency gains in AI inference. The Z1T architecture uses 4-bit weights and a modified gated-convolutional-attention design, with operations assigned to Z1 estimated to be about three orders of magnitude more energy efficient than equivalent GPU operations.