logic-OT/Decoder-Only-LLM
This repository features a custom-built decoder-only language model (LLM) with a total of 37 million parameters 🔥. I train the model to be able to ask question from a given context
This project helps machine learning engineers or researchers build and pretrain a custom language model for specific text generation tasks. You provide your own conversational dataset, and the model learns to generate relevant text based on a given context. It's designed for those who need to develop specialized LLMs rather than use off-the-shelf solutions.
No commits in the last 6 months.
Use this if you are a machine learning engineer or researcher looking to experiment with or build a bespoke decoder-only large language model from scratch and train it on your own unique dataset.
Not ideal if you need a ready-to-use language model for general text generation or question answering without custom training or deep technical involvement.
Stars
16
Forks
4
Language
Jupyter Notebook
License
—
Category
Last pushed
Aug 27, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/logic-OT/Decoder-Only-LLM"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
liangyuwang/Tiny-DeepSpeed
Tiny-DeepSpeed, a minimalistic re-implementation of the DeepSpeed library
catherinesyeh/attention-viz
Visualizing query-key interactions in language + vision transformers (VIS 2023)
microsoft/Text2Grad
🚀 Text2Grad: Converting natural language feedback into gradient signals for precise model...
FareedKhan-dev/Building-llama3-from-scratch
LLaMA 3 is one of the most promising open-source model after Mistral, we will recreate it's...