shibing624/pinyin-tokenizer
pinyintokenizer, 拼音分词器，将连续的拼音切分为单字拼音列表。
This tool helps you break down continuous Pinyin sequences into individual Pinyin syllables, which is useful for processing spoken or typed Chinese text. For example, if you input "nihao", it outputs ['ni', 'hao']. This is designed for anyone working with Pinyin, especially those involved in natural language processing (NLP) for Chinese, like transcribers, linguists, or developers building Pinyin-to-Hanzi conversion systems.
No commits in the last 6 months.
Use this if you need to accurately segment a string of Pinyin into its constituent syllables, often as a first step before converting Pinyin to Chinese characters.
Not ideal if your input includes English words, as it is specifically designed for Pinyin and may produce unexpected results.
Stars
31
Forks
1
Language
Python
License
Apache-2.0
Category
Last pushed
Feb 05, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/shibing624/pinyin-tokenizer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PyThaiNLP/pythainlp
Thai natural language processing in Python
hankcs/HanLP
Natural Language Processing for the next decade. Tokenization, Part-of-Speech Tagging, Named...
jacksonllee/pycantonese
Cantonese Linguistics and NLP
dongrixinyu/JioNLP
中文 NLP 预处理、解析工具包，准确、高效、易用 A Chinese NLP Preprocessing & Parsing Package www.jionlp.com
hankcs/pyhanlp
中文分词