BorealisAI/flora-opt
This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.
Training large AI models, especially large language models (LLMs), often requires immense GPU memory, which can be a significant bottleneck. Flora dramatically reduces the GPU memory needed for pre-training and fine-tuning these models without sacrificing performance or noticeably slowing down the process. AI/ML researchers and engineers working on deep learning model development can use this tool to develop and experiment with large models on more constrained hardware.
106 stars. No commits in the last 6 months.
Use this if you need to pre-train or fine-tune large deep learning models, such as LLMs, and are running into GPU memory limitations.
Not ideal if you are working with smaller models that do not push the limits of your GPU memory, as the benefits might not outweigh the integration effort.
Stars
106
Forks
5
Language
Python
License
LGPL-3.0
Category
Last pushed
Jul 01, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/BorealisAI/flora-opt"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ThilinaRajapakse/simpletransformers
Transformers for Information Retrieval, Text Classification, NER, QA, Language Modelling,...
jsksxs360/How-to-use-Transformers
Transformers 库快速入门教程
google/deepconsensus
DeepConsensus uses gap-aware sequence transformers to correct errors in Pacific Biosciences...
Denis2054/Transformers-for-NLP-2nd-Edition
Transformer models from BERT to GPT-4, environments from Hugging Face to OpenAI. Fine-tuning,...
abhimishra91/transformers-tutorials
Github repo with tutorials to fine tune transformers for diff NLP tasks