mars-project/mars
Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
This framework helps data professionals, data scientists, and engineers to speed up their large-scale data analysis and machine learning workflows. It takes existing NumPy arrays, pandas DataFrames, or scikit-learn models and processes them much faster, whether on a single machine or across a cluster. The result is quicker computations and model training for vast datasets.
2,748 stars. No commits in the last 6 months.
Use this if you are working with extremely large datasets that make standard Python data libraries like NumPy, pandas, or scikit-learn too slow or run out of memory.
Not ideal if your data processing needs are small to medium-sized, as the overhead of a distributed framework might outweigh the benefits.
Stars
2,748
Forks
323
Language
Python
License
Apache-2.0
Category
Last pushed
Jan 02, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/mars-project/mars"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference...
helmholtz-analytics/heat
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
bsc-wdc/dislib
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.