bsc-wdc/dislib
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.
This library helps researchers and data scientists run complex machine learning tasks on massive datasets using supercomputers or large clusters. You provide your Python-based machine learning code, and the library optimizes it to run efficiently across many processors, delivering results much faster than on a single machine. It's designed for those who need to scale their data analysis beyond typical computing limits.
Available on PyPI.
Use this if you are a researcher or data scientist needing to execute high-performance machine learning algorithms on very large datasets using distributed computing environments like supercomputers or large clusters.
Not ideal if you are working with small to medium-sized datasets or do not have access to distributed computing infrastructure.
Stars
52
Forks
26
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 13, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/bsc-wdc/dislib"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related frameworks
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference...
helmholtz-analytics/heat
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
learning-at-home/hivemind
Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across...