kyegomez/DifferentialTransformer
An open source community implementation of the model from "DIFFERENTIAL TRANSFORMER" paper by Microsoft.
This project offers an implementation of a 'Differential Transformer' model, designed to improve the quality of machine learning models that process sequential data like text or time series. It takes raw input data (e.g., text sequences) and produces cleaner, more focused representations by reducing irrelevant 'noise'. This is for machine learning engineers and researchers building advanced AI models.
Use this if you are developing transformer-based models and need to enhance their performance by reducing 'attention noise' in sequential data processing.
Not ideal if you are looking for an out-of-the-box application to solve a specific business problem rather than a component for model development.
Stars
39
Forks
—
Language
Python
License
MIT
Category
Last pushed
Mar 09, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/kyegomez/DifferentialTransformer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
transformerlab/transformerlab-app
The open source research environment for AI researchers to seamlessly train, evaluate, and scale...
naru-project/naru
Neural Relation Understanding: neural cardinality estimators for tabular data
neurocard/neurocard
State-of-the-art neural cardinality estimators for join queries
danielzuegner/code-transformer
Implementation of the paper "Language-agnostic representation learning of source code from...
salesforce/CodeTF
CodeTF: One-stop Transformer Library for State-of-the-art Code LLM