aahouzi/llama2-chatbot-cpu
A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.
This project helps you run a LLaMA2-7b chatbot directly on your computer's CPU, rather than needing specialized hardware. You provide your approved LLaMA2 model and get an interactive chatbot that remembers your conversation history. This is for AI developers, researchers, or data scientists who need to deploy and test large language models efficiently on standard CPU infrastructure.
No commits in the last 6 months.
Use this if you need to deploy and interact with a LLaMA2-7b chatbot on a CPU with improved speed and memory efficiency.
Not ideal if you are a casual user looking for a ready-to-use chatbot without any technical setup or model access requirements.
Stars
15
Forks
—
Language
Python
License
MIT
Category
Last pushed
Feb 27, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/aahouzi/llama2-chatbot-cpu"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jakobdylanc/llmcord
Make Discord your LLM frontend - Supports any OpenAI compatible API (Ollama, xAI, Gemini,...
xNul/chat-llama-discord-bot
A Discord Bot for chatting with LLaMA, Vicuna, Alpaca, MPT, or any other Large Language Model...
amanvirparhar/weebo
A real-time speech-to-speech chatbot powered by Whisper Small, Llama 3.2, and Kokoro-82M.
innightwolfsleep/old_llm_telegram_bot
Connect llama-cpp, transformers or text-generation-webui to telegram bot api.
ma2za/telegram-llm-bot
Telegram LLM bot backed by OpenAI, Whisper, Beam, LLaMA, Weaviate, MinIO and MongoDB