SpecForge and TorchSpec
SpecForge and TorchSpec are competitors for training speculative decoding models, with SpecForge offering additional integration into the SGLang serving ecosystem while TorchSpec provides a PyTorch-native alternative.
About SpecForge
sgl-project/SpecForge
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
For those working with large language models, SpecForge helps you train specialized 'speculative decoding' models that can significantly speed up how fast your main LLM responds. You feed in your LLM and it outputs a more efficient version, ready to be used with the SGLang serving framework. This is for AI practitioners and researchers looking to optimize LLM inference performance.
About TorchSpec
torchspec-project/TorchSpec
A PyTorch native library for training speculative decoding models
This tool helps AI engineers optimize large language models by training specialized 'draft' models for speculative decoding. It takes hidden states from existing inference engines as input and produces a smaller, faster draft model. This allows for significant speed improvements when generating text with large language models.
Scores updated daily from GitHub, PyPI, and npm data. How scores work