HomebrewML/HomebrewNLP-torch
A case study of efficient training of large language models using commodity hardware.
This project helps machine learning engineers and researchers explore how to train large language models effectively using standard computer hardware, rather than specialized, expensive systems. It takes in large text datasets and outputs a trained language model, demonstrating practical approaches for optimizing training on commodity machines. This is for professionals focused on advanced natural language processing and model development.
No commits in the last 6 months.
Use this if you are a machine learning engineer or researcher looking to understand and apply techniques for training large language models efficiently on widely available hardware.
Not ideal if you are looking for a ready-to-use language model for deployment or do not have experience with model training and optimization.
Stars
68
Forks
8
Language
Python
License
BSD-2-Clause
Category
Last pushed
Aug 04, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/HomebrewML/HomebrewNLP-torch"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AliHaiderAhmad001/GPT-from-Scratch-with-Tensorflow
Implementation for "Improving Language Understanding by Generative Pre-Training" paper
akshat0123/GPT-1
Pytorch implementation of GPT-1
qiqiApink/MotionGPT
The official PyTorch implementation of the paper "MotionGPT: Finetuned LLMs are General-Purpose...
nawnoes/pytorch-gpt-x
An implementation of an autoregressive language model using an improved Transformer and...
Shenggan/atp
Adaptive Tensor Parallelism for Foundation Models