[Project Showcase] Disk Sparse Adam (DSA) — A disk-backed SparseAdam for large embeddings and GNNs

Hi PyTorch Community! I wanted to share an open-source library I’ve developed to solve a common memory bottleneck when training large-scale sparse models: Disk Sparse Adam (DSA). 🔴 The Problem: Memory Overheads in Large Sparse Embeddings When training massive embedding tables (e.g., Knowledge Graph Embeddings or Graph Neural Networks with millions/billions of nodes), standard torch.optim.SparseAdam requires storing first and second momentum states ($m$ and $v$) for every single para...