Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

Understand and test language model architectures on synthetic tasks.

Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"

An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivation. It is probably the code which is the most close to select…

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

VMamba: Visual State Space Models，code is based on mamba

Mamba SSM architecture

Official implementation of the paper "Frequency-domain MLPs are More Effective Learners in Time Series Forecasting"

Simba

Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)