manuelescobar-dev/LLM-Tools
Open-source calculator for LLM system requirements.
This tool helps AI engineers and researchers estimate the computing memory needed to run or train large language models (LLMs). You input details like the model's size, desired precision, batch size, and sequence length, and it outputs an estimate of the total memory required for inference or training. This allows you to plan your hardware resources more effectively.
175 stars. No commits in the last 6 months.
Use this if you need to quickly estimate the GPU or system memory required to run or train a large language model and ensure you have adequate hardware.
Not ideal if you need a precise, real-time memory monitor for an already deployed LLM, as this provides estimates for planning, not live diagnostics.
Stars
175
Forks
29
Language
Python
License
MIT
Category
Last pushed
Dec 18, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/manuelescobar-dev/LLM-Tools"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
thu-pacman/chitu
High-performance inference framework for large language models, focusing on efficiency,...
NotPunchnox/rkllama
Ollama alternative for Rockchip NPU: An efficient solution for running AI and Deep learning...
sophgo/LLM-TPU
Run generative AI models in sophgo BM1684X/BM1688
Deep-Spark/DeepSparkHub
DeepSparkHub selects hundreds of application algorithms and models, covering various fields of...
howard-hou/VisualRWKV
VisualRWKV is the visual-enhanced version of the RWKV language model, enabling RWKV to handle...