A Library to build MoE from HF models

9 points

zmy999

2 years ago

Mergoo is a library that allows you to easily integrate multiple open-source LLMs from Hugging Face and efficiently train a mixture of experts without starting from scratch. https://github.com/Leeroo-AI/mergoo