SqueezeAILab/LLMCompiler
[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
This project helps developers integrate large language models (LLMs) with custom tools or functions more efficiently. It automatically figures out which tasks an LLM needs to perform using these tools can run at the same time and which ones depend on each other. Developers provide the LLM with a set of tools and example prompts, and the project optimizes how the LLM calls these tools to complete complex problems faster and more accurately. This is for developers building applications that use LLMs to solve problems that require multiple steps and external data or computations.
1,828 stars. No commits in the last 6 months.
Use this if you are a developer building LLM-powered applications and want to improve the speed, cost-efficiency, and accuracy of multi-function calling by enabling parallel execution of tasks.
Not ideal if you are looking for an off-the-shelf application to solve a specific problem, as this is a framework for developers to build with.
Stars
1,828
Forks
126
Language
Python
License
MIT
Category
Last pushed
Jul 10, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/SqueezeAILab/LLMCompiler"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
alibaba/MNN
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering...
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source,...
tensorzero/tensorzero
TensorZero is an open-source stack for industrial-grade LLM applications. It unifies an LLM...