faisaltareque/Multilingual-Sentence-Tokenizer

This Python package is designed for tokenizing sentences in over 40 languages. It serves as a wrapper around various open-source libraries. The package was created to support our work XL-HeadTags. To use it, simply provide the word and its corresponding language to the stemmer, and it will return the stemmed version of the word.

11
/ 100
Experimental

No commits in the last 6 months.

No License Stale 6m No Package No Dependents
Maintenance 0 / 25
Adoption 3 / 25
Maturity 8 / 25
Community 0 / 25

How are scores calculated?

Stars

3

Forks

—

Language

Python

License

—

Category

word-stemming-stemmers

Last pushed

Aug 11, 2024

Commits (30d)

0

GitHub
Word Stemming Stemmers · 56 tools

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/nlp/faisaltareque/Multilingual-Sentence-Tokenizer"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

hplt-project/sacremoses

Python port of Moses tokenizer, truecaser and normalizer

68

Blake-Madden/OleanderStemmingLibrary

Porter stemming library (C++)

54

adbar/simplemma

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

51

htaghizadeh/PersianStemmer-Python

PersianStemmer-Python

51

michmech/lemmatization-lists

Machine-readable lists of lemma-token pairs in 23 languages.

50

Explore NLP Tools

All categories Trending NLP directory Insights