Japanese Text Processing NLP Tools

Tools for Japanese-specific morphological analysis, text normalization, kana-kanji conversion, and character processing. Does NOT include general multilingual NLP, machine translation systems, or language learning applications (unless text processing is the primary focus).

There are 97 japanese text processing tools tracked. 1 score above 70 (verified tier). The highest-rated is EmilStenstrom/conllu at 73/100 with 320 stars.

Get all 97 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=nlp&subcategory=japanese-text-processing&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	EmilStenstrom/conllu A CoNLL-U parser that takes a CoNLL-U formatted string and turns it into a...	73	Verified	320	Python
2	OpenPecha/Botok 🏷 བོད་ཏོག [pʰøtɔk̚] Tibetan word tokenizer in Python	66	Established	78	Python
3	zaemyung/sentsplit A flexible sentence segmentation library using CRF model and regex rules	60	Established	31	Python
4	taishi-i/nagisa A Japanese tokenizer based on recurrent neural networks	60	Established	415	Python
5	natasha/razdel Rule-based token, sentence segmentation for Russian language	56	Established	279	Python
6	polm/cutlet Japanese to romaji converter in Python	55	Established	373	Python
7	textlint-rule/sentence-splitter Split {Japanese, English} text into sentences.	54	Established	136	TypeScript
8	azooKey/AzooKeyKanaKanjiConverter Kana-Kanji Conversion Module written in Swift, supporting Neural Kana-Kanji...	53	Established	138	Swift
9	ku-nlp/rhoknp Yet another Python binding for Juman++/KNP/KWJA	53	Established	38	Python
10	himkt/konoha 🌿 An easy-to-use Japanese Text Processing tool, which makes it possible to...	51	Established	261	Python
11	gold-silver-copper/english World's most accurate and fast procedural English conjugation library	48	Emerging	41	Rust
12	akaza-im/akaza Yet another Japanese IME for IBus/Linux	48	Emerging	247	Rust
13	PKSHATechnology-Research/tdmelodic A Japanese accent dictionary generator	48	Emerging	123	Python
14	togatoga/karukan Japanese Input Method System for Linux, Neural Kana-Kanji Conversion Engine...	48	Emerging	234	Rust
15	polm/fugashi A Cython MeCab wrapper for fast, pythonic Japanese tokenization and...	47	Emerging	515	C++
16	KoichiYasuoka/SuPar-UniDic Tokenizer POS-tagger Lemmatizer and Dependency-parser for modern and...	47	Emerging	20	Jupyter Notebook
17	kevincobain2000/jProcessing Japanese Natural Langauge Processing Libraries	46	Emerging	148	OpenEdge ABL
18	LibreTranslate/MiniSBD Free and open source library for fast sentence boundary detection	45	Emerging	3	Python
19	SOMJANG/Mecab-ko-for-Google-Colab Use Mecab Library(NLP Library) in Google Colab	45	Emerging	78	Shell
20	fnl/syntok Text tokenization and sentence segmentation (segtok v2)	45	Emerging	209	Python
21	rabbit19981023/yomigana-ebook The fastest converter to add furigana(readings) to Japanese epub eBooks	45	Emerging	26	Python
22	ku-nlp/jumanpp Juman++ (a Morphological Analyzer Toolkit)	44	Emerging	409	C++
23	hamanlp/hama 🦛 Hangul Morphological Analyzer	44	Emerging	4	TypeScript
24	miurahr/pykakasi Lightweight converter from Japanese Kana-kanji sentences into Kana-Roman.	44	Emerging	444	Python
25	ikegami-yukino/neologdn Japanese text normalizer for mecab-neologd	44	Emerging	287	Cython
26	mediacloud/sentence-splitter Text to sentence splitter using heuristic algorithm by Philipp Koehn and...	44	Emerging	259	Python
27	andreihar/taibun Taiwanese Hokkien Transliterator and Tokeniser	43	Emerging	44	Python
28	Kensuke-Mitsuzawa/JapaneseTokenizers aim to use JapaneseTokenizer as easy as possible	43	Emerging	137	Python
29	javierarce/silabea Node package that split Spanish words into syllables.	43	Emerging	6	JavaScript
30	ikegami-yukino/neologdn-java Japanese text normalizer for mecab-neologd	43	Emerging	5	Java
31	koshort/pyeunjeon (deprecated) 은전한닢 프로젝트와 mecab 기반의 한국어 형태소 분석기의 독립형 python 인터페이스	42	Emerging	60	C++
32	mkartawijaya/dango An easy to use tokenizer for Japanese text, aimed at language learners and...	42	Emerging	25	Python
33	ikegami-yukino/mozcpy Kana-Kanji converter using Mozc dictionary	42	Emerging	47	Python
34	alinear-corp/kuzukiri Japanese Text Segmenter for Python written in Rust	41	Emerging	6	Jupyter Notebook
35	loomchild/segment Program used to split text into segments	40	Emerging	28	Java
36	thammin/juman-bin a User-Extensible Morphological Analyzer for Japanese. 日本語形態素解析システム	40	Emerging	3	JavaScript
37	andreihar/taibun.js Taiwanese Hokkien Transliterator and Tokeniser	40	Emerging	4	JavaScript
38	neelguha/legal-segmenter A simple library for segmenting legal texts	38	Emerging	18	Python
39	wwwcojp/ja_sentence_segmenter japanese sentence segmentation library for python	38	Emerging	74	Python
40	LanguageMachines/mbt MBT: Memory-based tagger generation and tagging MBT is a memory-based...	38	Emerging	10	C++
41	craigtrim/fast-sentence-segment Fast and Efficient Sentence Segmentation	37	Emerging	2	Python
42	medspacy/sectionizer A rule-based Python module for spitting documents into sections.	36	Emerging	12	Jupyter Notebook
43	LR-POR/cl-conllu tool for working with conllu files in CL	36	Emerging	13	Common Lisp
44	typedgrammar/typed-japanese 🌸 Learn Japanese grammar with TypeScript	36	Emerging	1,920	TypeScript
45	azu/morpheme-match match function that match token(形態素解析) with sentence.	36	Emerging	14	JavaScript
46	gpizzorno/conllu_tools A Python toolkit for working with CoNLL-U files, Universal Dependencies...	35	Emerging	3	Python
47	ABTdomain/dksplit DKSplit — fast word segmentation for Python. Split domain names and...	34	Emerging	2	Python
48	tokuhirom/jawiki-kana-kanji-dict Generate SKK/MeCab dictionary from Wikipedia(Japanese edition)	34	Emerging	60	Python
49	ejossev/hypherator-java Java Hyphenation Iterator	34	Emerging	3	Java
50	ku-nlp/knp A Japanese Parser	34	Emerging	33	C
51	cronokirby/ginkou Japanese sentence bank program. Add and find sentences for language learning.	33	Emerging	5	Rust
52	yoshoku/suika Suika 🍉 is a Japanese morphological analyzer written in pure Ruby	33	Emerging	51	Ruby
53	luxiant/sentence_segmentation A rule-based sentence_segmenter, inspired by ruby pragmatic segmenter by...	32	Emerging	2	Rust
54	retarfi/jptranstokenizer Japanese Tokenizer for transformers library	32	Emerging	5	Python
55	tasukuigarashi/j-liwc2015 Japanese version of LIWC2015	32	Emerging	13	—
56	junhewk/RcppMeCab RcppMeCab: Rcpp Interface of CJK Morpheme Analyzer MeCab	32	Emerging	27	C++
57	ArthurDevNL/CoNLL-U A lightweight NuGet package for parsing CoNLL-U files in C#	31	Emerging	4	C#
58	agatan/yoin A Japanese Morphological Analyzer written in pure Rust	30	Emerging	26	Rust
59	uribo/sudachir R Interface to 'Sudachi'	30	Emerging	6	R
60	KOLANICH-libs/WordSplitAbs.py An abstraction layer around word splitters for python	29	Experimental	1	Python
61	NonJishoKei/NonJishoKei [WIP] This is a lightweight morphological analyzer designed for Japanese...	29	Experimental	102	HTML
62	azagniotov/solr-lucene-analyzer-sudachi A Japanese morphological analyzer Sudachi as a Solr plugin.	29	Experimental	9	Java
63	hephaex/mecab-ko MeCab-Ko: Rust로 구현된 한국어 형태소 분석기. 세종 코퍼스 호환 97% 정확도.	29	Experimental	2	C++
64	MiguelNecoechea/Complexa Yet another Chrome extension for learning Japanese	28	Experimental	2	TypeScript
65	junhewk/RmecabKo RmecabKo: R wrapper for eunjeon project (mecab-ko)	28	Experimental	10	Shell
66	apakabarlabs/syllabreak-swift Multilingual library for accurate and deterministic hyphenation and syllable...	27	Experimental	2	Swift
67	taipalogy/taipa 台灣語形態素解析(Taiwanese morphological parsing)	27	Experimental	2	TypeScript
68	tchin25/japanese-dependency-visualizer A dependency visualizer for Japanese to help beginners deconstruct complex...	25	Experimental	11	JavaScript
69	rmalouf/treesearch High-performance toolkit for querying linguistic dependency parses	25	Experimental	2	Rust
70	cryshin22/Cutlet-Japan Japanese to romaji converter in Python	23	Experimental	17	Python
71	hppRC/jawiki-cleaner 🧹Japanese Wikipedia Cleaner 🧹	21	Experimental	6	Python
72	GINK03/boosting-tree-tokenizer Gradient Boosting Dicision...	21	Experimental	9	C++
73	atsumari-io/mecab-service Web app for tokenizing Japanese text using MeCab	21	Experimental	1	TypeScript
74	btrkeks/jp-deinflector A high-performance Rust crate for deinflecting Japanese words using perfect...	21	Experimental	7	Rust
75	tetutaro/mecab_dictionaries create various dictionaries for MeCab and MeCab CLI using fugashi	20	Experimental	8	Python
76	megagonlabs/desuwa Feature annotator to morphemes and phrases based on KNP rule files (pure-Python)	20	Experimental	6	Emacs Lisp
77	rakutentech/pisah Sentence Splitter Library (C++ port of pySBD)	20	Experimental	5	C++
78	Shusei-E/RcppJagger RcppJagger is a wrapper package for Jagger	19	Experimental	4	C++
79	bureaucratic-labs/conllu CoNLL-U format parser	19	Experimental	3	Go
80	jeffhuen/plurality Fast English plural and singular noun inflection for Elixir. Convert plural...	19	Experimental	—	Elixir
81	akiomik/vibrato-dict-ipa-neologd A compiled mecab-ipadic-neologd dictionary for vibrato	19	Experimental	4	Rust
82	cronokirby/nicer-mecab Japanese morphological analysis. Wrapper over mecab.	19	Experimental	4	Rust
83	BrambleXu/jp-stopword-filter A lightweight Python library designed to filter stopwords from Japanese text...	19	Experimental	4	Python
84	proycon/hyphertool Command-line tool for syllabification and hyphenisation for multiple languages	19	Experimental	1	Rust
85	milovatjp/hazuki Japanese complexity analysis app within JLPT framework.	18	Experimental	2	Jupyter Notebook
86	QuyAnh2005/StyleTTS-VC-Japanese StyleTTS Voice Conversions for Japanese	18	Experimental	2	Python
87	d108/Samazama Save keystrokes for iOS and macOS users by comparing shorthand input against...	17	Experimental	1	Swift
88	TaygaHoshi/japanese-i-plus-one-filter Finds i+1 sentences for a specific word from Jisho.org.	17	Experimental	1	Python
89	luckasRanarison/kaiseki A japanese tokenizer and morphological analyzer	17	Experimental	1	Rust
90	mkpoli/ainu-wiktionary アイヌ語Wiktionary入力補助ツール	15	Experimental	—	Svelte
91	PenguinCabinet/Aiueo-sort-with-Kanji-reading 漢字の読みを考慮した、あいうえお順ソートツール	12	Experimental	1	TypeScript
92	whelk-io/hy-phen-a-tion Java OSS library for calculating syllables and hyphenation based on Frank...	12	Experimental	3	Java
93	ku-nlp/jumanpp-jumandic Scripts for training Jumandic Juman++ model	12	Experimental	7	Makefile
94	ensan-hcl/KanaKanjiConversionSamples KanaKanjiConversionSamples is a sample package which implements very simple...	11	Experimental	3	Swift
95	ru-ka/syllable-divider A WebAssembly library for syllable division in XML/DOM trees	11	Experimental	—	Rust
96	evamaxfield/cue-queue Transcript segmentation using the average semantic encodings of cue sentences.	11	Experimental	2	Python
97	apakabarlabs/syllabreak-kotlin Kotlin library for multilingual syllabification and hyphenation	11	Experimental	—	Kotlin