seonglae/llama2gptq
Chat to LLaMa 2 that also provides responses with reference documents over vector database. Locally available model using GPTQ 4bit quantization.
This tool allows you to chat with an AI that uses your own documents to provide answers, complete with references. You feed it your existing text files, like Notion data or other documents, and it generates conversational responses backed by those sources and even suggests related webpages. This is ideal for researchers, analysts, or anyone who needs to quickly get answers from their private data without sending it to external AI services.
No commits in the last 6 months.
Use this if you need a confidential, AI-powered assistant that can answer questions and provide sources based on your specific private documents or knowledge base.
Not ideal if you're looking for a cloud-based AI solution or don't have the technical setup (like a CUDA-compatible GPU) to run a local model.
Stars
31
Forks
—
Language
Python
License
MIT
Category
Last pushed
Nov 25, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/seonglae/llama2gptq"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jakobdylanc/llmcord
Make Discord your LLM frontend - Supports any OpenAI compatible API (Ollama, xAI, Gemini,...
xNul/chat-llama-discord-bot
A Discord Bot for chatting with LLaMA, Vicuna, Alpaca, MPT, or any other Large Language Model...
amanvirparhar/weebo
A real-time speech-to-speech chatbot powered by Whisper Small, Llama 3.2, and Kokoro-82M.
innightwolfsleep/old_llm_telegram_bot
Connect llama-cpp, transformers or text-generation-webui to telegram bot api.
ma2za/telegram-llm-bot
Telegram LLM bot backed by OpenAI, Whisper, Beam, LLaMA, Weaviate, MinIO and MongoDB