christinakim/scaling-laws-for-language-transfer
code for Scaling Laws for Language Transfer Learning
This project helps machine learning researchers understand how much pre-training a language model on English data benefits its performance when fine-tuning for other languages like Chinese, Spanish, or German. It takes existing English pre-trained models and then applies them to non-English text datasets, producing insights into the 'scaling laws' of language transfer. A machine learning scientist or researcher focused on multilingual natural language processing would find this useful.
No commits in the last 6 months.
Use this if you are a machine learning researcher exploring the optimal strategies for developing multilingual language models, specifically concerning the role of English pre-training.
Not ideal if you are a practitioner looking for a ready-to-use, production-grade multilingual model for direct application.
Stars
9
Forks
1
Language
Python
License
—
Category
Last pushed
Apr 18, 2021
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/christinakim/scaling-laws-for-language-transfer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
openvinotoolkit/nncf
Neural Network Compression Framework for enhanced OpenVINO™ inference
huggingface/optimum
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers...
NVIDIA/Megatron-LM
Ongoing research training transformer models at scale
huggingface/optimum-intel
🤗 Optimum Intel: Accelerate inference with Intel optimization tools
eole-nlp/eole
Open language modeling toolkit based on PyTorch