RoyZry98/T-REX-Pytorch
[Arxiv 2025] Official code for T-REX: Mixture-of-Rank-One-Experts with semantic-aware Intuition for Multi-task Large Language Model Finetuning
This project helps machine learning engineers fine-tune large language models (LLMs) to perform multiple tasks more efficiently. By using an advanced technique called Mixture-of-Rank-One-Experts, it takes a base LLM and a multi-task dataset to produce a specialized LLM capable of handling various language-related tasks. This is ideal for ML engineers working on applications requiring a single model to excel at diverse functions like summarization, translation, and question-answering.
No commits in the last 6 months.
Use this if you need to fine-tune a large language model to perform well across multiple distinct natural language processing tasks simultaneously.
Not ideal if you only need to fine-tune a model for a single specific task or if you are not comfortable working with command-line tools and machine learning frameworks.
Stars
17
Forks
1
Language
Python
License
—
Category
Last pushed
May 16, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/RoyZry98/T-REX-Pytorch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
EfficientMoE/MoE-Infinity
PyTorch library for cost-effective, fast and easy serving of MoE models.
raymin0223/mixture_of_recursions
Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation...
AviSoori1x/makeMoE
From scratch implementation of a sparse mixture of experts language model inspired by Andrej...
thu-nics/MoA
[CoLM'25] The official implementation of the paper
jaisidhsingh/pytorch-mixtures
One-stop solutions for Mixture of Expert modules in PyTorch.