CLUEbenchmark/CLUECorpus2020
Large-scale Pre-training Corpus for Chinese 100G 中文预训练语料
This project offers a massive, cleaned collection of Chinese text data, perfect for training language models or generating Chinese text. It takes raw Chinese web content and refines it into a high-quality corpus, ready for use in various natural language processing applications. Data scientists, AI researchers, or developers working on Chinese language technologies would find this valuable.
1,002 stars.
Use this if you need a large, pre-processed dataset of Chinese text to train or evaluate language models and generation tasks.
Not ideal if you require text in languages other than simplified Chinese, as traditional Chinese, English, Japanese, and Korean tokens are not fully supported.
Stars
1,002
Forks
83
Language
—
License
MIT
Category
Last pushed
Feb 06, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/CLUEbenchmark/CLUECorpus2020"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
acl-org/acl-anthology
Data and software for building the ACL Anthology.
anoopkunchukuttan/indic_nlp_library
Resources and tools for Indian language Natural Language Processing
KennethEnevoldsen/scandinavian-embedding-benchmark
A Scandinavian Benchmark for sentence embeddings
Separius/awesome-sentence-embedding
A curated list of pretrained sentence and word embedding models
AndyTheFactory/romanian-nlp-datasets
A list of Romanian NLP Datasets