CPJKU/wechsel
Code for WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language models.
Training large language models for new languages is incredibly resource-intensive. This tool helps natural language processing (NLP) researchers and machine learning engineers adapt existing English language models (like RoBERTa or GPT-2) to new languages without having to train them from scratch. It takes an English language model and a target language corpus, and outputs a version of the model initialized to understand the new language, making subsequent training much faster and more efficient.
No commits in the last 6 months. Available on PyPI.
Use this if you need to quickly adapt a pre-trained English language model to understand and process a new language, especially for languages with limited training data.
Not ideal if you are looking to train a language model completely from scratch in a new language without leveraging an existing English model.
Stars
89
Forks
12
Language
Python
License
MIT
Category
Last pushed
Sep 12, 2024
Commits (30d)
0
Dependencies
8
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/CPJKU/wechsel"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
LoicGrobol/zeldarose
Train transformer-based models.
yuanzhoulvpi2017/zero_nlp
中文nlp解决方案(大模型、数据、模型、训练、推理)
minggnim/nlp-models
A repository for training transformer based models
IntelLabs/nlp-architect
A model library for exploring state-of-the-art deep learning topologies and techniques for...
MahmoudWahdan/dialog-nlu
Tensorflow and Keras implementation of the state of the art researches in Dialog System NLU