OpenSparseLLMs/LLaMA-MoE-v2
🚀 LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
This project offers a collection of pre-trained, smaller, and more efficient language models built from LLaMA3, designed for advanced natural language processing. It takes existing large language models and outputs versions optimized for specific tasks, making them more affordable to run. Data scientists, machine learning engineers, and AI researchers would use this to deploy specialized AI.
No commits in the last 6 months.
Use this if you need to deploy a high-performing language model for specific tasks, but are concerned about the computational cost and resource demands of larger models.
Not ideal if you need an out-of-the-box, general-purpose large language model that doesn't require specialized fine-tuning or model construction.
Stars
93
Forks
13
Language
Python
License
Apache-2.0
Category
Last pushed
Dec 03, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/OpenSparseLLMs/LLaMA-MoE-v2"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
facebookresearch/LayerSkip
Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024
FareedKhan-dev/train-llm-from-scratch
A straightforward method for training your LLM, from downloading data to generating text.
kmeng01/rome
Locating and editing factual associations in GPT (NeurIPS 2022)
datawhalechina/llms-from-scratch-cn
仅需Python基础，从0构建大语言模型；从0逐步构建GLM4\Llama3\RWKV6， 深入理解大模型原理