twang2218/vocab-coverage
语言模型中文认知能力分析
This project helps researchers and developers evaluate how well large language models (LLMs) understand Chinese characters and words. It takes an LLM as input and generates detailed reports and visualizations showing its 'literacy rate' for Chinese characters and the semantic distribution of its Chinese word embeddings. Anyone building, selecting, or fine-tuning LLMs for Chinese language tasks would find this valuable.
236 stars. No commits in the last 6 months. Available on PyPI.
Use this if you need to understand the Chinese language capabilities of a large language model, particularly its grasp of different character sets and the semantic quality of its word representations.
Not ideal if you are looking to analyze non-Chinese language models or evaluate aspects of LLMs beyond character recognition and word embedding quality.
Stars
236
Forks
24
Language
Python
License
Apache-2.0
Category
Last pushed
Sep 09, 2023
Commits (30d)
0
Dependencies
17
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/twang2218/vocab-coverage"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
MinishLab/model2vec
Fast State-of-the-Art Static Embeddings
AnswerDotAI/ModernBERT
Bringing BERT into modernity via both architecture changes and scaling
tensorflow/hub
A library for transfer learning by reusing parts of TensorFlow models.
Embedding/Chinese-Word-Vectors
100+ Chinese Word Vectors 上百种预训练中文词向量
Santosh-Gupta/SpeedTorch
Library for faster pinned CPU <-> GPU transfer in Pytorch