DeepSpeed (microsoft/DeepSpeed): Microsoft’s primary deep learning optimization library. It introduces breakthroughs like ZeRO (Zero Redundancy Optimizer) to make distributed training and inference of massive GenAI models efficient on accessible hardware. 👉 microsoft/DeepSpeed GitHub Repository [1, 2]
Unsloth (unslothai/unsloth): A highly popular research project that heavily accelerates the fine-tuning of open-source models (like Llama and Mistral). It rewrites the underlying GPU kernels using OpenAI’s Triton language, making fine-tuning up to 30 times faster while using 60% less VRAM. 👉 unslothai/unsloth GitHub Repository [1, 2]
Axolotl (OpenAccess-AI-Collective/axolotl): A streamlined research framework designed for the open-source community to easily fine-tune various AI models. It supports advanced academic training techniques like LoRA, QLoRA, and various alignment methods (DPO, IPO). 👉 Axolotl GitHub Repository [1]