TrentPierce/PolyCouncil
PolyCouncil is an open-source multi-model deliberation engine for LM Studio. It runs multiple LLMs in parallel, gathers their answers, scores each response using a shared rubric, and produces a final, consensus-driven result. Designed for testing, comparing, and orchestrating local models with ease.
PolyCouncil helps you compare and orchestrate multiple large language models (LLMs) from various providers, whether they are running locally or hosted online. You provide a prompt, and it gathers answers from several LLMs, scores each response based on a shared rubric, and delivers a final, consensus-driven result. This is ideal for researchers, developers, or evaluators who need to systematically test and compare different LLMs.
Use this if you need to objectively compare the performance of multiple LLMs for a specific task or want to combine their insights into a single, refined output.
Not ideal if you only work with a single LLM at a time or are not interested in comparing or deliberating between different model responses.
Stars
26
Forks
3
Language
Python
License
—
Category
Last pushed
Feb 18, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/prompt-engineering/TrentPierce/PolyCouncil"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
langfuse/langfuse
🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management,...
Arize-ai/phoenix
AI Observability & Evaluation
Mirascope/mirascope
The LLM Anti-Framework
Agenta-AI/agenta
The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM...
Helicone/helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓