jlonge4/local_llama
This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.
This tool helps you quickly get answers from your documents without an internet connection or sharing them with external services. You input PDF, TXT, DOCX, or MD files, and it allows you to ask questions and receive summaries or specific information based on their content. Anyone who needs to extract information from their own documents while ensuring privacy and offline access would find this useful.
298 stars. No commits in the last 6 months.
Use this if you need to privately chat with your documents and retrieve information without uploading them to cloud-based AI services or requiring an internet connection.
Not ideal if you're looking for a general-purpose AI chatbot that can answer questions beyond the scope of your uploaded files or if you prefer a fully managed cloud solution.
Stars
298
Forks
47
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 12, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/jlonge4/local_llama"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
containers/ramalama
RamaLama is an open-source developer tool that simplifies the local serving of AI models from...
av/harbor
One command brings a complete pre-wired LLM stack with hundreds of services to explore.
RunanywhereAI/runanywhere-sdks
Production ready toolkit to run AI locally
runpod-workers/worker-vllm
The RunPod worker template for serving our large language model endpoints. Powered by vLLM.
foldl/chatllm.cpp
Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)