ronantakizawa/sleeptimecompute
A Demo of Running Sleep-time Compute to Reduce LLM Latency
If you manage applications that use AI language models for tasks like document question-answering or coding assistance, this project helps reduce the waiting time for model responses and improve accuracy. It works by analyzing the context of your application (like a document or codebase) during idle periods, then uses these insights to answer user queries faster and more precisely. This is designed for AI application developers and system architects looking to optimize the performance of their stateful LLM applications.
No commits in the last 6 months.
Use this if you need to reduce latency and improve accuracy in AI applications where users ask multiple related questions about a consistent body of information.
Not ideal if your application handles only single, unpredictable queries or if the context changes very frequently, as the benefits of pre-computation won't be realized.
Stars
16
Forks
2
Language
Jupyter Notebook
License
—
Category
Last pushed
May 17, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/ronantakizawa/sleeptimecompute"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
SepineTam/stata-mcp
Let LLM help you achieve your regression with Stata. Evolve from reg monkey to causal thinker.
datawhalechina/code-your-own-llm
一份全栈式大语言模型参考指南，用最简洁的代码帮助你端到端定义模型从零训练到工程落地的每一个细节
leonid20000/odin-slides
This is an advanced Python tool that empowers you to effortlessly draft customizable PowerPoint...
onejune2018/Awesome-LLM-Eval
Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs...
axhiao/QuickNote
Capture what you want with LLM