Gunale0926/SORSA
SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models
This project helps machine learning engineers adapt large language models (LLMs) for specific tasks, like answering math problems or coding, more efficiently. It takes a pre-trained LLM and a dataset for a new task, applying a method called SORSA to fine-tune the model with fewer computational resources. The result is a fine-tuned LLM that performs well on the new task without increasing the model's size or slowing down its inference.
No commits in the last 6 months. Available on PyPI.
Use this if you need to fine-tune large language models for specific downstream tasks without the extensive computational demands or increased inference latency typically associated with full fine-tuning.
Not ideal if you are a general user looking for an off-the-shelf LLM application rather than a tool for model adaptation.
Stars
38
Forks
4
Language
Python
License
Apache-2.0
Category
Last pushed
Feb 24, 2025
Commits (30d)
0
Dependencies
2
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/Gunale0926/SORSA"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
unslothai/unsloth
Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek, Qwen, Llama,...
huggingface/peft
🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.
modelscope/ms-swift
Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.5, DeepSeek-R1, GLM-5,...
oumi-ai/oumi
Easily fine-tune, evaluate and deploy gpt-oss, Qwen3, DeepSeek-R1, or any open source LLM / VLM!
linkedin/Liger-Kernel
Efficient Triton Kernels for LLM Training