vtuber-plan/langport
Langport is a language model inference service
This tool helps developers serve large language models quickly and efficiently in their applications. You provide a language model (like LLaMa or GPT2) and it creates a robust, distributed service that can handle many requests. This is for software developers building applications that need to interact with large language models, allowing them to integrate advanced AI capabilities into their products.
No commits in the last 6 months. Available on PyPI.
Use this if you are a software developer building applications that need to quickly and reliably respond to user input using large language models, especially when you need high throughput or distributed inference.
Not ideal if you are looking for a no-code solution to experiment with AI models, or if you only need a basic local setup for occasional use rather than a production-ready serving platform.
Stars
94
Forks
13
Language
Python
License
MIT
Category
Last pushed
Sep 09, 2024
Commits (30d)
0
Dependencies
15
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/vtuber-plan/langport"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
GURPREETKAURJETHRA/END-TO-END-GENERATIVE-AI-PROJECTS
End to End Generative AI Industry Projects on LLM Models with Deployment_Awesome LLM Projects
ariannamethod/molequla
molequla.ai. live ecology of GPT organisms
ictnlp/Stream-Omni
Stream-Omni is a GPT-4o-like language-vision-speech chatbot that simultaneously supports...
ashleykleynhans/text-generation-docker
Docker image for the Text Generation Web UI: A Gradio web UI for Large Language Models with...
josStorer/selfhostedAI
A collection of one-click self-hosted AI