maxpumperla/elephas
Distributed Deep learning with Keras & Spark
This project helps data scientists and machine learning engineers train deep learning models much faster by distributing the workload across a cluster of machines. You provide your Keras deep learning model and large datasets, and it leverages Apache Spark to process the data in parallel, producing a trained model ready for predictions. This is ideal for those working with massive datasets that overwhelm a single machine.
1,578 stars. No commits in the last 6 months.
Use this if you are a data scientist or machine learning engineer struggling with long training times for Keras deep learning models on very large datasets.
Not ideal if your datasets are small enough to be handled efficiently on a single machine without distributed computing.
Stars
1,578
Forks
309
Language
Python
License
MIT
Category
Last pushed
May 01, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/maxpumperla/elephas"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Compare
Related frameworks
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference...
helmholtz-analytics/heat
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
bsc-wdc/dislib
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.