go-air/dupi
A tool to find all duplicates in large sets of text documents.
This tool helps you quickly find identical or highly similar passages within very large collections of text documents. You provide it with thousands of documents, and it identifies and lets you query for all the repeated chunks of text. It's ideal for anyone managing large document archives, like legal professionals, researchers, or content managers.
No commits in the last 6 months.
Use this if you need to detect plagiarism, identify redundant content, or clean up massive text datasets by finding all duplicate sections.
Not ideal if you're looking for semantic similarity where wording is different but the meaning is the same, or if you only have a few documents to check.
Stars
16
Forks
3
Language
Go
License
Apache-2.0
Category
Last pushed
Sep 29, 2021
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/go-air/dupi"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.