architectures
Mamba
A selective state space model architecture that offers an alternative to transformers for sequence modeling. Mamba achieves linear-time inference scaling with sequence length while matching transformer performance on many tasks.
In practice
Mamba can process very long sequences much faster than transformers because it avoids the quadratic attention computation.