villagecomputing/superpipe
Superpipe - optimized LLM pipelines for structured data
Building and refining workflows that use large language models (LLMs) to process text and extract specific information can be challenging. This project helps you define clear steps for your LLM, feed it raw text data like job histories, and reliably get back structured data such as job departments. It's designed for data scientists, analysts, or product managers who need to extract consistent, categorized information from unstructured text at scale.
109 stars. No commits in the last 6 months.
Use this if you need to reliably classify, extract, or transform text data into structured formats using LLMs, and want to systematically test and improve your results for accuracy, speed, and cost.
Not ideal if you are looking for a pre-built, ready-to-use LLM application and don't intend to customize or evaluate the underlying data transformation steps.
Stars
109
Forks
2
Language
Python
License
—
Category
Last pushed
Jun 18, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/villagecomputing/superpipe"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
AI-Hypercomputer/maxtext
A simple, performant and scalable Jax LLM!
rasbt/reasoning-from-scratch
Implement a reasoning LLM in PyTorch from scratch, step by step
mindspore-lab/mindnlp
MindSpore + 🤗Huggingface: Run any Transformers/Diffusers model on MindSpore with seamless...
mosaicml/llm-foundry
LLM training code for Databricks foundation models
rickiepark/llm-from-scratch
<밑바닥부터 만들면서 공부하는 LLM>(길벗, 2025)의 코드 저장소