shimo-lab/modelmap
Embedding language models in probability space via log-likelihood vectors
This tool helps AI researchers and practitioners understand the relationships between different language models. You input a collection of diverse language models and a dataset of texts, and it outputs a visual 'map' showing how similar or different these models are to each other based on how they assign probabilities to the texts. This allows you to quickly grasp which models perform similarly and which are unique.
Use this if you need to visualize and compare a large number of language models to understand their underlying characteristics and relationships, especially in research and development settings.
Not ideal if you are looking for a tool to evaluate the performance of a single language model or to fine-tune a model for a specific task.
Stars
16
Forks
1
Language
Jupyter Notebook
License
—
Category
Last pushed
Oct 25, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/shimo-lab/modelmap"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
ContextualAI/gritlm
Generative Representational Instruction Tuning
xlang-ai/instructor-embedding
[ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings
liuqidong07/LLMEmb
[AAAI'25 Oral] The official implementation code of LLMEmb
hpcaitech/CachedEmbedding
A memory efficient DLRM training solution using ColossalAI
ritesh-modi/embedding-hallucinations
This repo shows how foundational model hallucinates and how we can fix such hallucinations using...