AI21 Labs releases Jamba, a hybrid SSM-Transformer model
Combining Mamba state-space layers with transformer attention, it handled a 256K-token context and fit on a single 80GB GPU, which AI21 said pure transformers could not match.
Models & capabilities · Open weights & ecosystem