procesaur/Scratch2LM
Training transformer models (e.g. RoBERTa, GPT2 and GPT-J) from scratch.
This helps data scientists and machine learning engineers create custom language models tailored to their specific data. You provide raw text data, and it trains a transformer model (like RoBERTa or GPT2) from the ground up, generating a specialized language model ready for downstream tasks.
No commits in the last 6 months.
Use this if you need to train a large language model from scratch on your unique dataset, rather than fine-tuning a pre-existing general-purpose model.
Not ideal if you're looking for a simple plug-and-play solution for common NLP tasks, or if you don't have extensive text data and computational resources.
Stars
7
Forks
1
Language
Python
License
GPL-3.0
Category
Last pushed
Oct 01, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/procesaur/Scratch2LM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
tabularis-ai/be_great
A novel approach for synthesizing tabular data using pretrained large language models
EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron...
shibing624/textgen
TextGen: Implementation of Text Generation models, include LLaMA, BLOOM, GPT2, BART, T5, SongNet...
ai-forever/ru-gpts
Russian GPT3 models.
AdityaNG/kan-gpt
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold...