Mathematical Reasoning Transformers Transformer Models

Tools for training transformers to solve mathematical and symbolic reasoning problems through techniques like pretraining, reinforcement learning, and neuro-symbolic methods. Does NOT include general question-answering, commonsense reasoning without mathematical focus, or pure symbolic solvers without neural components.

There are 94 mathematical reasoning transformers models tracked. 3 score above 50 (established tier). The highest-rated is galilai-group/stable-pretraining at 56/100 with 133 stars.

Get all 94 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=transformers&subcategory=mathematical-reasoning-transformers&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Model	Score	Tier	Stars	Language
1	galilai-group/stable-pretraining Reliable, minimal and scalable library for pretraining foundation and world models	56	Established	133	Python
2	CognitiveAISystems/MAPF-GPT [AAAI-2025] This repository contains MAPF-GPT, a deep learning-based model...	53	Established	119	C++
3	UKPLab/gpl Powerful unsupervised domain adaptation method for dense retrieval. Requires...	52	Established	340	Python
4	larslorch/avici Amortized Inference for Causal Structure Learning, NeurIPS 2022	49	Emerging	72	Python
5	svdrecbd/mhc-mlx MLX + Metal implementation of mHC: Manifold-Constrained Hyper-Connections by...	47	Emerging	3	Python
6	kyegomez/MHMoE Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch	47	Emerging	29	Python
7	chaitjo/learning-tsp Code for the paper 'Learning TSP Requires Rethinking Generalization' (CP 2021)	46	Emerging	241	Jupyter Notebook
8	ai4co/routefinder [TMLR 2025 + ICML 2024 FM-Wild Oral] RouteFinder: Towards Foundation Models...	46	Emerging	111	Python
9	Cognitive-AI-Systems/MAPF-GPT-DDG [IROS-2025] MAPF-GPT-DDG is a scalable decentralized multi-agent pathfinding...	46	Emerging	61	Python
10	eloialonso/iris Transformers are Sample-Efficient World Models. ICLR 2023, notable top 5%.	45	Emerging	870	Python
11	deep-symbolic-mathematics/TPSR [NeurIPS 2023] This is the official code for the paper "TPSR:...	44	Emerging	81	Python
12	IntelLabs/causality-lab Causal discovery algorithms and tools for implementing new ones	44	Emerging	247	Jupyter Notebook
13	RobertCsordas/modules The official repository for our paper "Are Neural Nets Modular? Inspecting...	41	Emerging	46	Python
14	pjlab-sys4nlp/llama-moe ⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual...	41	Emerging	1,002	Python
15	ai4co/parco [NeurIPS 2025] PARCO: Parallel AutoRegressive Combinatorial Optimization	40	Emerging	44	Python
16	vmicheli/delta-iris Efficient World Models with Context-Aware Tokenization. ICML 2024	40	Emerging	119	Python
17	softengg-manoj/dreamer4 🌟 Implement Dreamer 4 for training agents within scalable world models,...	40	Emerging	4	Python
18	IDSIA/automated-cl Official repository for the paper "Automating Continual Learning"	39	Emerging	18	Python
19	IDSIA/lmtool-fwp PyTorch Language Modeling Toolkit for Fast Weight Programmers	39	Emerging	19	Python
20	microsoft/COCO-LM [NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for...	39	Emerging	118	Python
21	deep-symbolic-mathematics/Multimodal-Symbolic-Regression [ICLR 2024 Spotlight] SNIP on Symbolic Regression: Deep Symbolic Regression...	38	Emerging	21	Python
22	IDSIA/fpainter Official repository for the paper "Images as Weight Matrices: Sequential...	37	Emerging	12	Python
23	deep-symbolic-mathematics/Multimodal-Math-Pretraining [ICLR 2024 Spotlight] This is the official code for the paper "SNIP:...	36	Emerging	58	Python
24	srvCodes/continual_learning_with_vit Code for our CVPR 2022 workshop paper "Towards Exemplar-Free Continual...	35	Emerging	24	Python
25	cifkao/context-probing Black-box language model explanation by context length probing	34	Emerging	9	Jupyter Notebook
26	IDSIA/modern-srwm Official repository for the paper "A Modern Self-Referential Weight Matrix...	34	Emerging	176	Python
27	czg1225/CoDe [CVPR 2025] CoDe: Collaborative Decoding Makes Visual Auto-Regressive...	34	Emerging	108	Python
28	levashi/reprobe Phase-aware LLM activation steering and linear probing. A memory-efficient,...	33	Emerging	2	Python
29	softsys4ai/differentiable-proving Code and data for the paper "Pretrained Language Models are Symbolic...	32	Emerging	12	Python
30	alexliap/greek_gpt MoE Decoder Transformer implementation with MLX	32	Emerging	6	Python
31	AIRI-Institute/Probing_framework Framework for probing tasks	32	Emerging	31	Python
32	yyDing1/GNER [ACL 2024 Findings] Code implementation of Paper "Rethinking Negative...	31	Emerging	60	Python
33	elijahnzeli1/CausalTorch CausalTorch is a PyTorch library for building generative models with...	31	Emerging	5	Python
34	microsoft/AMOS [ICLR 2022] Pretraining Text Encoders with Adversarial Mixture of Training...	30	Emerging	26	Python
35	OrigamiDream/CoRT CoRT: Contrastive Rhetorical Tagging - KISTI 2022 AI/ML Competition	30	Emerging	6	Python
36	Shekswess/tiny-reasoning-language-model Code repository dedicated to experimenting and research with tiny reasoning...	30	Emerging	49	Python
37	NellyW8/VeriReason This is the Github Repo for the paper: VeriReason: Reinforcement Learning...	30	Emerging	21	Python
38	relign-ai/relign post train language models on multi-step reasoning with reinforcement learning	30	Emerging	20	Python
39	ianchute/generative-reflections A two-model system for reasonable text generation	29	Experimental	1	Jupyter Notebook
40	DataArcTech/ChartMoE [ICLR2025 Oral] ChartMoE: Mixture of Diversely Aligned Expert Connector for...	29	Experimental	94	Jupyter Notebook
41	Ultron09/Mirror_mind A production-ready adaptive meta-learning framework for continuous...	29	Experimental	5	Python
42	IDSIA/recurrent-fwp Official repository for the paper "Going Beyond Linear Transformers with...	28	Experimental	51	Python
43	anastadimi/Contra-Sformer Code for 'Keep Your Eye on the Best: Contrastive Regression Transformer for...	28	Experimental	12	Python
44	cpuheater/cause-life-is-a-game Solving games with reinforcement learning	28	Experimental	7	Python
45	ImMohammadHosseini/MKP-RL :sparkles: Solve multi_dimensional multiple knapsack problem using...	27	Experimental	13	Python
46	cui-shaobo/causal-strength evaluating the causal strength between cause and effect	27	Experimental	2	Python
47	AndreaCossu/continual-pretraining-nlp-vision Code to reproduce experiments from the paper "Continual Pre-Training...	26	Experimental	22	Jupyter Notebook
48	cattolatte/reflective-reasoning-transformer 🧠 R2T Prototype: An LLM pre-trained on causal graphs (not just text) to...	25	Experimental	2	Python
49	RitoCryo/DeepRWKV-Reasoning 🔍 Enhance reasoning in Large Language Models with DeepRWKV-Reasoning, using...	24	Experimental	1	Python
50	ashimmortallp/mHC-manifold-constrained-hyper-connections 🔍 Explore mHC for manifold-constrained hyper-connections in PyTorch,...	23	Experimental	—	Python
51	The-Swarm-Corporation/MoF This work introduces Flow Matching Mixture of Experts (FM-MoE), a framework...	23	Experimental	2	Python
52	Pomilon-Intelligence-Lab/ALSI Early baby steps towards a long-term vision regarding Mamba-2's state...	22	Experimental	1	Python
53	aliuyar1234/proberoute Research code for ProbeRoute, a probe-initialized sparse routing method for...	22	Experimental	—	Python
54	matlok-ai/bampe-weights This repository is for profiling, extracting, visualizing and reusing...	22	Experimental	9	Python
55	capybara-brain346/moe-router A small Mixture-of-Experts (MoE) Transformer trained from scratch to learn...	21	Experimental	2	Python
56	discover-Austin/Architectural-Emergence-of-Synchronization Modular Recursive Workspace (MRW) - Complete Phase Transition Detection...	21	Experimental	—	Python
57	Eran-BA/MoP Mixture of Products (MoP) for Transformers — research prototype	21	Experimental	6	Python
58	CheongWoong/impact_of_cooccurrence A repository for analyzing the impact of co-occurrence statistics on factual...	21	Experimental	10	Jupyter Notebook
59	nlx-group/Shortcutted-Commonsense-Reasoning Code for the article "Shortcutted Commonsense: Data Spuriousness in Deep...	21	Experimental	10	Jupyter Notebook
60	axonura/axonura-X1 The First AI Model Of Axonura	21	Experimental	—	Python
61	AndrewBoessen/neural-game-engine Neural network approach for modeling interactive game environments using...	20	Experimental	5	Python
62	torotoki/reasoning-minimal Minimal code to train reasoning model with reinforcement learning.	20	Experimental	3	Python
63	NISL-MSU/MultiSetSR Decomposable Neuro Symbolic Regression	20	Experimental	2	Python
64	The-Swarm-Corporation/ClusterMoE A novel neural network architecture that extends Mixture of Experts (MoE)...	20	Experimental	4	Python
65	mduffster/self-referent-test Testing role-based pathways on small LLMs	20	Experimental	1	Python
66	nlx-group/Commonsense-Reasoning-Neuro-only-vs-Neuro-Symbolic-Methods Code for the article "Commonsense Reasoning: how do Neuro-only and hybrid...	19	Experimental	4	Python
67	gpt-reasoning/ReasoningCombinatorials [NeurIPS'25] Teaching Transformers to Solve Combinatorial Problems through...	19	Experimental	—	C
68	omron-sinicx/transformer4sr [NeurIPS 2023 AI4Science] "A Transformer Model for Symbolic Regression...	19	Experimental	18	Python
69	UIC-Liu-Lab/CPT [EMNLP 2022] Continual Training of Language Models for Few-Shot Learning	19	Experimental	44	Python
70	AdamG012/moe-paper-models A sumary of MoE experimental setups across a number of different papers.	19	Experimental	16	—
71	kreasof-ai/stable-latent-reasoning Stable Latent Reasoning --- Enhancing Inference in Large Language Models...	18	Experimental	2	—
72	cyan-ide/nn_models Neural network / AI models / LLM models - implementations from scratch in pytorch	18	Experimental	1	Jupyter Notebook
73	alessoh/ssi1 Developing neural-symbolic transformer models for superintelligence method	17	Experimental	1	Python
74	eljandoubi/PaliGemma Coding PaliGemma from scratch using pytorch for inference.	17	Experimental	1	Python
75	TheAeryan/strips-transformer Code for work "From Next Token Prediction to (STRIPS) World Models --...	17	Experimental	—	PDDL
76	neuro-symbolic-ai/latent_mathematical_reasoning Multi-Operational Mathematical Derivations in Latent Space	17	Experimental	1	Python
77	UKPLab/starsem2023-arithmetic-based-pretraining Code and data for the StarSem 2023 paper "Arithmetic-Based Pretraining --...	17	Experimental	1	Julia
78	moxin-org/CC-MoE Collaborative Compression for Large-Scale MoE Deployment on Edge	16	Experimental	4	Python
79	Reason-Wang/NAT [NAACL 2025] The official implementation of paper "Learning From Failure:...	15	Experimental	28	Python
80	bassrehab/steering-vectors-agents Runtime control of LLM agent behaviors through activation steering vectors....	14	Experimental	3	Python
81	anayebi/mental-sim Models of Mental Simulation	13	Experimental	10	Jupyter Notebook
82	pranavAL/DART Official Code Repo for the paper "Learning to Play Atari in a World of...	13	Experimental	11	—
83	chaowei312/dsan6650_final Recursive reasoning with tiny transformers (<1M params): TRM + MoE + MCTS...	13	Experimental	—	Jupyter Notebook
84	CheongWoong/knowledge_probing A repository for factual knowledge probing with large language models.	13	Experimental	—	Python
85	thesofakillers/infoshare Official repository for the paper: "Probing LLMs for Joint Encoding of...	12	Experimental	7	Python
86	neil-ab/probing-lms Probing language models for linguistic features in their representations	12	Experimental	5	Jupyter Notebook
87	torchipeppo/rc2024-wm A world model component for the technical challenge of RoboCup 2024 SPL	11	Experimental	—	Python
88	Masao-Taketani/multi-agent-env-generator My master research implementation titled 'Multi-Agent Simulated Environments...	11	Experimental	—	Python
89	bihani-g/rel-paradox This repository contains code and experiments for the paper 'The Reliability...	11	Experimental	—	Jupyter Notebook
90	Fanziyang-v/contrastive-decoding SOTA Contrastive Decoding Strategies Implementation	11	Experimental	4	Python
91	francesco-p/off-the-shelf-cl Simple off-the-shelf solution for Continual Learning of Computer Vision...	11	Experimental	3	Python
92	krishoncloud/SymbolicRegression-ML4sci-Krish-Malik For ML4sci Symbolic Regression Evaluation Tasks - Krish Malik	11	Experimental	3	Jupyter Notebook
93	andresnowak/Mixture-of-Experts-mlx Implementation of different Mixture of Experts in MLX	10	Experimental	1	Python
94	SimonOuellette35/MLC-ARC_gym Evaluating MLC method on ARC_gym	10	Experimental	2	Python

Comparisons in this category

MAPF-GPT and MAPF-GPT-DDG (53 vs 46)