deepfinch/XLearning-GPU
qihoo360 xlearning with GPU support; AI on Hadoop
This project helps data scientists and machine learning engineers run their AI and deep learning models on large datasets using existing Hadoop clusters, leveraging GPU resources. You provide your deep learning model code and data, and it manages the distributed execution, producing trained models and results. This is for professionals building and deploying machine learning solutions at scale within an enterprise Hadoop environment.
No commits in the last 6 months.
Use this if you need to train deep learning models on large datasets already stored in Hadoop HDFS, and you want to utilize GPU resources across your cluster efficiently.
Not ideal if you are working with small datasets, prefer local training, or do not operate within a Hadoop YARN ecosystem.
Stars
24
Forks
9
Language
Java
License
—
Category
Last pushed
Jan 10, 2018
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/deepfinch/XLearning-GPU"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference...
helmholtz-analytics/heat
Distributed tensors and Machine Learning framework with GPU and MPI acceleration in Python
hpcaitech/ColossalAI
Making large AI models cheaper, faster and more accessible
horovod/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.
bsc-wdc/dislib
The Distributed Computing library for python implemented using PyCOMPSs programming model for HPC.