feifeibear/Odysseus-Transformer
Odysseus: Playground of LLM Sequence Parallelism
This project is a playground for developers who are pushing the boundaries of large language models (LLMs) with extremely long input sequences. It explores and compares different methods for 'sequence parallelism,' which is a way to distribute the computational load of processing long texts across multiple GPUs. Developers can experiment with different parallelization techniques to find the most efficient way to train or run these advanced models.
No commits in the last 6 months.
Use this if you are a machine learning engineer or researcher developing or fine-tuning large language models and need to optimize their performance when working with very long input texts, especially on multi-GPU setups.
Not ideal if you are a general LLM user or a developer looking for a high-level API to simply use an existing LLM, as this project focuses on low-level parallelization strategies.
Stars
79
Forks
6
Language
Python
License
—
Category
Last pushed
Jun 17, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/feifeibear/Odysseus-Transformer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AI-Hypercomputer/maxtext
A simple, performant and scalable Jax LLM!
rasbt/reasoning-from-scratch
Implement a reasoning LLM in PyTorch from scratch, step by step
mindspore-lab/mindnlp
MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore with seamless...
mosaicml/llm-foundry
LLM training code for Databricks foundation models
rickiepark/llm-from-scratch
<밑바닥부터 만들면서 공부하는 LLM>(길벗, 2025)의 코드 저장소