capjamesg/pysurprisal
Calculate surprisal for words in text.
This tool helps linguists and cognitive scientists analyze text by quantifying how unexpected each word is within a given passage. You input a piece of text, and it outputs a numerical value for each word, indicating its 'surprisal' or how much new information it conveys. This is designed for researchers studying language comprehension or text complexity.
No commits in the last 6 months. Available on PyPI.
Use this if you need to objectively measure the information content or predictability of individual words in a text for linguistic analysis.
Not ideal if you need a tool for broad sentiment analysis, topic modeling, or general natural language understanding beyond word-level surprisal.
Stars
8
Forks
—
Language
Python
License
MIT
Category
Last pushed
Dec 15, 2023
Commits (30d)
0
Dependencies
1
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/capjamesg/pysurprisal"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
sloria/TextBlob
Simple, Pythonic, text processing--Sentiment analysis, part-of-speech tagging, noun phrase...
chrismattmann/tika-python
Tika-Python is a Python binding to the Apache Tika™ REST services allowing Tika to be called...
cltk/cltk
The Classical Language Toolkit
allenai/scispacy
A full spaCy pipeline and models for scientific/biomedical documents.
wi2trier/cbrkit
Customizable Case-Based Reasoning (CBR) toolkit for Python with a built-in API and CLI.