AndreaCossu/continual-pretraining-nlp-vision
Code to reproduce experiments from the paper "Continual Pre-Training Mitigates Forgetting in Language and Vision" https://arxiv.org/abs/2205.09357
This project helps researchers and machine learning engineers working with large language models and computer vision models. It provides the tools and code to explore how to update these models with new information over time without 'forgetting' previously learned knowledge. You input pre-trained language or vision models and various datasets, and the output helps you understand strategies to continuously update them effectively.
No commits in the last 6 months.
Use this if you are developing or studying methods to continually pre-train NLP or computer vision models and want to mitigate catastrophic forgetting.
Not ideal if you are looking for a ready-to-use application or a high-level library to perform continual learning tasks without diving into the underlying research and model training specifics.
Stars
22
Forks
1
Language
Jupyter Notebook
License
MIT
Category
Last pushed
Oct 23, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/AndreaCossu/continual-pretraining-nlp-vision"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
galilai-group/stable-pretraining
Reliable, minimal and scalable library for pretraining foundation and world models
CognitiveAISystems/MAPF-GPT
[AAAI-2025] This repository contains MAPF-GPT, a deep learning-based model for solving MAPF...
UKPLab/gpl
Powerful unsupervised domain adaptation method for dense retrieval. Requires only unlabeled...
larslorch/avici
Amortized Inference for Causal Structure Learning, NeurIPS 2022
svdrecbd/mhc-mlx
MLX + Metal implementation of mHC: Manifold-Constrained Hyper-Connections by DeepSeek-AI.