AnswerDotAI/ModernBERT
Bringing BERT into modernity via both architecture changes and scaling
This is a research repository for pre-training and evaluating advanced language models, specifically ModernBERT. It takes large text datasets and outputs a trained language model and its performance evaluations, which can then be used for various natural language understanding tasks. Machine learning researchers and practitioners who want to experiment with cutting-edge language model architectures and training methods would use this.
1,642 stars.
Use this if you are a machine learning researcher or engineer interested in experimenting with and pre-training advanced BERT-like language models for natural language processing tasks, and you are comfortable with a research-focused codebase.
Not ideal if you are looking for a plug-and-play language model for immediate use in an application; for that, you should head to the ModernBERT Collection on HuggingFace.
Stars
1,642
Forks
144
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 01, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/AnswerDotAI/ModernBERT"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
MinishLab/model2vec
Fast State-of-the-Art Static Embeddings
tensorflow/hub
A library for transfer learning by reusing parts of TensorFlow models.
Embedding/Chinese-Word-Vectors
100+ Chinese Word Vectors 上百种预训练中文词向量
twang2218/vocab-coverage
语言模型中文认知能力分析
Santosh-Gupta/SpeedTorch
Library for faster pinned CPU <-> GPU transfer in Pytorch