orion-orion/Distributed-ML-PySpark
🔨 使用Spark/Pytorch实现分布式算法，包括图/矩阵计算（graph/matrix computation）、随机算法、优化（optimization）和机器学习。参考刘铁岩《分布式机器学习》和CME 323课程
This project provides implementations of classic distributed machine learning algorithms using PySpark and PyTorch. It helps data scientists and machine learning engineers process very large datasets by distributing the computational load. You can input large graphs, matrices, or general datasets, and it outputs trained machine learning models, optimized calculations, or analytical results, making complex tasks tractable on big data.
No commits in the last 6 months.
Use this if you need to run machine learning, graph analysis, or optimization algorithms on datasets that are too large to process on a single machine.
Not ideal if your datasets are small enough to be handled by a single computer or if you are not comfortable with distributed computing environments like Spark.
Stars
39
Forks
6
Language
Python
License
MIT
Category
Last pushed
Jun 27, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/orion-orion/Distributed-ML-PySpark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PaddlePaddle/Paddle
PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice...
fastai/fastai
The fastai deep learning library
openvinotoolkit/openvino_notebooks
📚 Jupyter notebook tutorials for OpenVINO™
PaddlePaddle/docs
Documentations for PaddlePaddle
msuzen/bristol
Parallel random matrix tools and complexity for deep learning