ilyalasy/moe-routing
Analysis of token routing for different implementations of Mixture of Experts
This tool helps researchers and AI practitioners understand how different Mixture of Experts (MoE) Large Language Models (LLMs) distribute input tokens to their specialized 'expert' subnetworks. You provide a RedPajama dataset, and it produces data and visualizations showing how tokens are routed. This is primarily for those researching or working with the architecture and efficiency of MoE LLMs.
No commits in the last 6 months.
Use this if you are developing or studying Mixture of Experts LLMs and need to analyze the token routing patterns to optimize performance or understand architectural behavior.
Not ideal if you are looking for a tool to train, fine-tune, or simply use an LLM for text generation or other end-user applications.
Stars
10
Forks
—
Language
Jupyter Notebook
License
—
Category
Last pushed
Mar 22, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/ilyalasy/moe-routing"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AdaptiveMotorControlLab/CEBRA
Learnable latent embeddings for joint behavioral and neural analysis - Official implementation of CEBRA
theolepage/sslsv
Toolkit for training and evaluating Self-Supervised Learning (SSL) frameworks for Speaker...
PaddlePaddle/PASSL
PASSL包含 SimCLR，MoCo v1/v2，BYOL，CLIP，PixPro，simsiam, SwAV, BEiT，MAE 等图像自监督算法以及 Vision...
YGZWQZD/LAMDA-SSL
30 Semi-Supervised Learning Algorithms
ModSSC/ModSSC
ModSSC: A Modular Framework for Semi Supervised Classification