cisnlp/MEXA
🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment
This tool helps you evaluate how well an English-centric large language model (LLM) understands other languages. You provide a dataset of parallel sentences (e.g., English and Spanish), and it calculates an "alignment score" that shows how similar the LLM's understanding of different languages is to its English understanding. This helps AI researchers and engineers understand how effective an LLM will be for multilingual applications without extensive testing.
No commits in the last 6 months.
Use this if you need to quickly estimate how well an English-centric LLM will perform on tasks in various non-English languages based on its English performance.
Not ideal if you need a direct, task-specific performance metric for a non-English language or if your LLM is not primarily English-centric.
Stars
11
Forks
1
Language
Python
License
Apache-2.0
Category
Last pushed
Apr 06, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/cisnlp/MEXA"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
MilaNLProc/contextualized-topic-models
A python package to run contextualized topic modeling. CTMs combine contextualized embeddings...
vinid/cade
Compass-aligned Distributional Embeddings. Align embeddings from different corpora
spcl/ncc
Neural Code Comprehension: A Learnable Representation of Code Semantics
criteo-research/CausE
Code for the Recsys 2018 paper entitled Causal Embeddings for Recommandation.
vintasoftware/entity-embed
PyTorch library for transforming entities like companies, products, etc. into vectors to support...