louisbrulenaudet/mergeKit
Tools for merging pretrained Large Language Models and create Mixture of Experts (MoE) from open-source models.
MergeKit helps AI practitioners combine multiple pre-trained large language models (LLMs) into a single, more capable model, or create Mixture of Experts (MoE) models. You provide existing open-source LLMs, and it outputs a new, merged model along with a basic README file for sharing. This is for AI engineers and researchers who want to customize and enhance LLMs without vast computational resources.
No commits in the last 6 months.
Use this if you want to combine several open-source Large Language Models or create a Mixture of Experts model efficiently, even with limited GPU memory.
Not ideal if you are looking for a tool to train LLMs from scratch or fine-tune them on new datasets rather than merging existing ones.
Stars
8
Forks
—
Language
Jupyter Notebook
License
Apache-2.0
Category
Last pushed
Sep 18, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/louisbrulenaudet/mergeKit"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Tencent/AngelSlim
Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.
nebuly-ai/optimate
A collection of libraries to optimise AI model performances
antgroup/glake
GLake: optimizing GPU memory management and IO transmission.
kyo-takano/chinchilla
A toolkit for scaling law research ⚖
liyucheng09/Selective_Context
Compress your input to ChatGPT or other LLMs, to let them process 2x more content and save 40%...