cx0/llm-typos
Impact of typos and common misspellings on LLM task performance.
This project helps evaluate how well Large Language Models (LLMs) can handle text containing typos and common misspellings. It takes a piece of text and a target word (which may or may not be misspelled) and measures the LLM's accuracy in finding the words immediately before and after it. This is useful for anyone working with LLMs who needs to understand their robustness when processing imperfect real-world text inputs, such as content from speech-to-text, user-generated content, or unproofread documents.
No commits in the last 6 months.
Use this if you are a researcher or practitioner working with LLMs and need to assess how well they perform text retrieval tasks when faced with common typographical errors, homophones, or other subtle misspellings.
Not ideal if you are looking for a tool to correct typos in text or a general-purpose LLM evaluation framework that goes beyond spelling variations.
Stars
19
Forks
1
Language
Python
License
—
Category
Last pushed
Mar 22, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/cx0/llm-typos"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
google/langfun
OO for LLMs
tanaos/artifex
Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.
preligens-lab/textnoisr
Adding random noise to a text dataset, and controlling very accurately the quality of the result
vulnerability-lookup/VulnTrain
A tool to generate datasets and models based on vulnerabilities descriptions from @Vulnerability-Lookup.
masakhane-io/masakhane-mt
Machine Translation for Africa