Prompt Optimization Systems Prompt Engineering Tools

Tools for iteratively testing, evaluating, and automatically improving LLM prompts through structured experimentation, search algorithms, and performance metrics. Does NOT include prompt storage/repositories, workflow orchestration, or general prompt writing guides.

There are 61 prompt optimization systems tools tracked. 1 score above 70 (verified tier). The highest-rated is dottxt-ai/outlines at 74/100 with 13,552 stars. 1 of the top 10 are actively maintained.

Get all 61 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=prompt-engineering&subcategory=prompt-optimization-systems&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	dottxt-ai/outlines Structured Outputs	74	Verified	13,552	Python
2	takashiishida/arxiv-to-prompt Transform arXiv papers into a single LaTeX source that can be used as a...	54	Established	133	Python
3	microsoft/promptpex Test Generation for Prompts	51	Established	158	TeX
4	Spr-Aachen/LLM-PromptMaster A simple LLM-Powered chatbot software.	43	Emerging	36	Python
5	AI-secure/aug-pe [ICML 2024 Spotlight] Differentially Private Synthetic Data via Foundation...	43	Emerging	57	Python
6	equinor/promptly A prompt collection for testing and evaluation of LLMs.	43	Emerging	27	Jupyter Notebook
7	microsoft/Evoke Everything for the Paper: 'Evoke: Evoking Critical Thinking Abilities in...	37	Emerging	19	—
8	hunterooc/mudplayersolo LLMs playing MUDs, no world model, self-improving prompts	36	Emerging	4	Python
9	step2-technology/ga-llm GA + LLM hybrid framework for structured text generation and task optimization.	35	Emerging	5	Python
10	Spr-Aachen/LLM-PromptMaster-Backend Backend for LLM-PromptMaster	34	Emerging	3	Python
11	hammer-mt/thumb A simple prompt testing library for LLMs.	33	Emerging	28	Python
12	mistir-nigusse/LLM_Prompt_Engine AI-driven prompt generation and evaluation system, designed to optimize the...	32	Emerging	3	Python
13	colindoyle0000/llms-as-method-actors LLMs as Method Actors: A Model for Prompt Engineering and Architecture	31	Emerging	46	Python
14	pyladiesams/optimize-ai-prompts-jun2025 Optimize AI prompts to reduce verbosity and carbon footprint with ScaleDown	31	Emerging	2	Jupyter Notebook
15	JRC1995/ZeroPromptSearch Implementation of an LLM prompting pipeline combined with wrappers for...	29	Experimental	15	Python
16	tddschn/langwhat pyWhat LLM version \| Answer "What is it?" on the command line with the power...	29	Experimental	7	Python
17	rosaia/langworks Flexibly instruct and prompt LLMs in Python	28	Experimental	1	Python
18	BashMocha/Prompting-LLMs-for-Aerial-Navigation Prompts and source code for applying LLMs to UAV-based navigation tasks with...	24	Experimental	59	Python
19	canto-lang/canto-lang Language for programming LLMs	23	Experimental	7	Python
20	husain-zaidi/wordle-llm-solver Wordle solver agent using Semantic kernel to prompt Large language models.	23	Experimental	4	Python
21	othercodes/prompt-lab Test prompt variants across LLM providers with LLM-as-judge evaluation	22	Experimental	1	Python
22	HebeHH/llm-prompt-experiments Experiment with multifactor analysis of different prompting strategies.	22	Experimental	6	TypeScript
23	ibrahimcesar/prompt-entropy-experiment Quantifying prompt quality using information theory: entropy and mutual...	22	Experimental	3	Jupyter Notebook
24	pedrocvaranda/notebooklm-power-prompt High-performance prompt generator to maximize learning in NotebookLM	22	Experimental	1	HTML
25	gsavla6-hue/llm-prompt-optimizer Systematic LLM prompt optimization framework with A/B testing, metrics...	22	Experimental	—	Python
26	3aKHP/Neural-Narratology Research on LLM Interactive Narrative: From Reverse Engineering to Automated...	21	Experimental	—	PowerShell
27	parth-shinge/Hybrid-Prompt-Generator Hybrid prompt generation framework combining templates, LLMs, ensemble...	21	Experimental	—	Python
28	marklysze/AutoGenPromptTesting Prompt testing Local LLMs for Microsoft's AutoGen	20	Experimental	10	Python
29	tkhang1999/repair-prompts Various prompt templates for automatic program repair using LLMs	19	Experimental	3	—
30	pwenker/prompt_teacher The Prompt Teacher is an interactive and educational prompt engineering...	19	Experimental	29	Python
31	zhoukuncheng/llm-vocabulary-reminder This project is a Telegram bot that sends daily reminders with English...	19	Experimental	3	Python
32	Pro-GenAI/Auto-Precise-Prompts Automated LLM-based Prompt Engineering for Structured Data Processing	19	Experimental	4	Jupyter Notebook
33	taylorsatula/TeaLeaves End-to-end pipeline for seeing how LLMs actually process your prompts....	19	Experimental	20	Python
34	estebanpdl/narrative-blueprint AI-driven analysis workflow that uses LLMs to analyze narratives, content...	19	Experimental	2	Python
35	elanzini/steer Steer: Enhancing Control Flow and Steerability for Large Language Models	18	Experimental	2	Python
36	KishoreMuruganantham/LipiForge A reproducible narrative transformation engine that uses RAG-style pipelines...	17	Experimental	—	Python
37	Dr-AneeshJoseph/Noumen-Loom-Game 🕸️ THE NOUMENAL LOOM: An Ontological Roguelike played inside LLMs....	17	Experimental	—	—
38	breezy89757/LlmPromptLab AI-powered Prompt Engineering Lab with Meta-Prompting and Auto-Evaluation...	17	Experimental	—	C#
39	Justmalhar/prompt-engineer-agent AI Prompt Engineer to automatically enhance and optimize prompts for large...	17	Experimental	1	Python
40	GuilhermeRuy97/prompt-evaluation-langsmith A project to take a suboptimal prompt from Langsmith, enhance it, submit it...	17	Experimental	6	Python
41	Anmolian/Prompt_Eval_LLM_Judge Prompt Design & LLM Judge	17	Experimental	1	Python
42	keon/prompt-psychology Code used for The Prompt Psychology Myth article	16	Experimental	3	Python
43	xyz9911/KPPO Official Implementation of "Beyond Elicitation: Provision-based Prompt...	15	Experimental	5	—
44	tomazf8/invariant-well A framework for inducing structural invariants in LLMs via non-linear...	14	Experimental	—	HTML
45	kgkazakos/notebook_lm_prompt_generator A command-line Python tool that acts as an AI creative director for Google's...	14	Experimental	—	Python
46	MercanCanpolat06/prompt-engineering-practice Trying various prompt engineering techniques and challenging them with both...	14	Experimental	1	Python
47	stevenittileon/Wordle-Bot A Python-based Wordle-solving bot that uses frequency analysis and prompt...	13	Experimental	—	Jupyter Notebook
48	Saharsh1005/autonomous-prompting Developing an autonomous system for prompt selection for Large Language...	13	Experimental	—	Jupyter Notebook
49	DevonFulcher/prompt-teacher An LLM prompt that helps to improve LLM prompts for AI applications by...	13	Experimental	—	Python
50	thatoldfarm/llminux An LLM playground for finding out what happens when LLM bugs are turned into...	13	Experimental	—	TypeScript
51	l0d0v1c/anteprompt Prepromt generator for LLM model, focused on personality	13	Experimental	—	HTML
52	Yuyang-Yao75/Prompt-Enginneer-on-Recommend-System 传统的推荐系统依赖明确建模和大量特征工程，难以快速适配新的场景。本项目探索如何利用大语言模型（LLM）...	13	Experimental	—	Python
53	barun-saha/CodePrompt Evaluation of code generation, with sequential trials, using PaLM 2	11	Experimental	—	Python
54	RamonMeffert/ltp-verifiability Course project on prompt engineering for automated verifiability checking of...	11	Experimental	3	Python
55	lintyourcode/promptu Write dynamic LLM prompts as natural language	11	Experimental	—	Python
56	genaivitbcommunity/prompt_analyzer A hybrid machine learning system for scoring LLM prompts. Features a...	11	Experimental	—	Jupyter Notebook
57	sukiboo/bct_experiments Collection of scripts to perform various experiments with behavioral messages	11	Experimental	—	Python
58	NoraEr/System_Prompt_Pipeline This project helps you structure your system prompt for LLM's using the...	11	Experimental	—	Python
59	AMfeta99/LLM-vs.-Human-Minds "Will you be able to win?" This is the game that challenges Human to face...	11	Experimental	—	Python
60	Wingtail/devils-advocate A simple but devilishly effective LLM inference framework for planning and refinement	10	Experimental	2	Python
61	079035/nx1_llm A tool to iteratively optimize a chain-of-thought prompt for classifying SEC...	10	Experimental	1	HTML

Comparisons in this category

promptpex and promptly (51 vs 43)