aalok-sathe/surprisal
A unified interface for computing surprisal (log probabilities) from language models! Supports neural, symbolic, and black-box API models.
This tool helps researchers in linguistics, psychology, and cognitive science measure how surprising a word is within a sentence using various language models. You input sentences or text, and it outputs numerical 'surprisal' scores for each word or a chosen segment, indicating how unexpected that word was in its context. It's designed for anyone analyzing human language processing, text comprehension, or language model behavior.
No commits in the last 6 months. Available on PyPI.
Use this if you need to quantify the predictability or unexpectedness of words in text, for example, to understand reading difficulty or assess a language model's fluency.
Not ideal if you need to generate text, translate languages, or perform general text classification, as this tool focuses specifically on surprisal calculation.
Stars
51
Forks
11
Language
Python
License
MIT
Category
Last pushed
Aug 06, 2025
Commits (30d)
0
Dependencies
3
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/aalok-sathe/surprisal"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
EvolvingLMMs-Lab/lmms-engine
A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.
FunnySaltyFish/Better-Ruozhiba
【逐条处理完成】人为审核+修改每一条的弱智吧精选问题QA数据集
reasoning-machines/pal
PaL: Program-Aided Language Models (ICML 2023)
microsoft/monitors4codegen
Code and Data artifact for NeurIPS 2023 paper - "Monitor-Guided Decoding of Code LMs with Static...
YutongWang1216/DocMTAgent
Code and data releases for the paper -- DelTA: An Online Document-Level Translation Agent Based...