ahnjaewoo/timechara
🧙🏻 Code and benchmark for our Findings of ACL 2024 paper - "TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models"
This project helps researchers and developers evaluate how well large language models (LLMs) can maintain a character's traits and knowledge at specific points in a story, preventing them from 'hallucinating' information. You provide a story, like the Harry Potter series, and an LLM's responses, and it assesses the accuracy of the character's statements or actions against the story's timeline. This tool is for those who are developing or rigorously testing role-playing LLMs to ensure they stay true to their assigned personas.
No commits in the last 6 months.
Use this if you need to objectively measure and benchmark how consistently your large language models embody a specific character's knowledge and history over time within a narrative.
Not ideal if you are looking for a tool to generate character-driven narratives or if your focus is on general factual accuracy rather than point-in-time character consistency.
Stars
21
Forks
1
Language
Python
License
MIT
Category
Last pushed
Dec 20, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/ahnjaewoo/timechara"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
SimonAytes/SoT
Official code repository for Sketch-of-Thought (SoT)
xuyige/SoftCoT
ACL'2025: SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs. and preprint:...
Fr0zenCrane/UniCoT
[ICLR 2026] Uni-CoT: Towards Unified Chain-of-Thought Reasoning Across Text and Vision
logikon-ai/cot-eval
A framework for evaluating the effectiveness of chain-of-thought reasoning in language models.
nicolay-r/THOR-ECAC
The official fork of THoR Chain-of-Thought framework, enhanced and adapted for Emotion Cause...