neulab/knn-transformers
PyTorch + HuggingFace code for RetoMaton: "Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval" (ICML 2022), including an implementation of kNN-LM and kNN-MT
This project helps machine learning researchers and practitioners enhance existing language models and machine translation models by integrating k-nearest neighbor (kNN) retrieval. You start with a pre-trained language model, which is then augmented with a 'datastore' of training examples. The output is a more accurate language model or machine translation model, specifically useful for those working on improving text generation, summarization, or translation quality.
286 stars. No commits in the last 6 months.
Use this if you are an NLP researcher or machine learning engineer looking to improve the performance of Hugging Face's `transformers` language models or machine translation models using retrieval-augmented methods without extensive code modifications.
Not ideal if you are a business user or an application developer who needs a ready-to-use API or a pre-packaged solution for a specific NLP task without delving into model architecture or training pipelines.
Stars
286
Forks
23
Language
Python
License
MIT
Category
Last pushed
Oct 20, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/neulab/knn-transformers"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
lucidrains/x-transformers
A concise but complete full-attention transformer with a set of promising experimental features...
kanishkamisra/minicons
Utility for behavioral and representational analyses of Language Models
lucidrains/simple-hierarchical-transformer
Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT
lucidrains/dreamer4
Implementation of Danijar's latest iteration for his Dreamer line of work
Nicolepcx/Transformers-in-Action
This is the corresponding code for the book Transformers in Action