BenChaliah/Superposition-Transformer
a novel architecture that leverages Autoencoders to superimpose the hidden representations of a base model and a fine-tuned model within a shared parameter space. Using B-spline-based blending coefficients and autoencoders that adaptively reconstruct the original hidden states based on the input data distribution.
This project helps machine learning engineers and researchers adapt large language models (LLMs) to new tasks or languages without losing their original abilities. It takes an existing LLM and a version fine-tuned for a specific task or domain, combining their knowledge to produce a single, more versatile model. This is for AI practitioners who need to deploy models that can handle multiple specialized tasks while retaining general knowledge.
No commits in the last 6 months.
Use this if you need to fine-tune a large language model for a new task or domain and want to avoid the problem of 'catastrophic forgetting,' where the model loses its previous knowledge.
Not ideal if you are looking for a pre-trained model ready for immediate deployment or if you are not comfortable working with deep learning architectures and model training.
Stars
79
Forks
3
Language
Jupyter Notebook
License
—
Category
Last pushed
Aug 01, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/BenChaliah/Superposition-Transformer"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
huggingface/text-generation-inference
Large Language Model Text Generation Inference
OpenMachine-ai/transformer-tricks
A collection of tricks and tools to speed up transformer models
poloclub/transformer-explainer
Transformer Explained Visually: Learn How LLM Transformer Models Work with Interactive Visualization
IBM/TabFormer
Code & Data for "Tabular Transformers for Modeling Multivariate Time Series" (ICASSP, 2021)
tensorgi/TPA
[NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (T6)...