flink-extended/dl-on-flink

Deep Learning on Flink aims to integrate Flink and deep learning frameworks (e.g. TensorFlow, PyTorch, etc) to enable distributed deep learning training and inference on a Flink cluster.

/ 100

Established

This project helps data scientists and machine learning engineers run their deep learning models across many machines efficiently. It takes existing TensorFlow or PyTorch models and training data, and then distributes the training or inference workload on a Flink cluster. This allows for faster processing of large datasets and more robust operations.

695 stars. No commits in the last 6 months.

Use this if you need to scale deep learning model training or inference workflows on large datasets using a Flink cluster.

Not ideal if you are working with small datasets, prefer a single-machine setup, or are not already using Apache Flink for your data processing.

distributed-machine-learning deep-learning-engineering big-data-analytics model-training ml-operations

Stale 6m No Package No Dependents

Maintenance 0 / 25

Adoption 10 / 25

Maturity 16 / 25

Community 25 / 25

How are scores calculated?

Stars

695

Forks

199

Language

Java

License

Apache-2.0

Related frameworks

lensacom/sparkit-learn

PySpark + Scikit-learn = Sparkit-learn

Angel-ML/angel

A Flexible and Powerful Parameter Server for large-scale machine learning

MingChen0919/learning-apache-spark

Notes on Apache Spark (pyspark)

mahmoudparsian/data-algorithms-book

MapReduce, Spark, Java, and Scala for Data Algorithms Book

endymecy/spark-ml-source-analysis

spark ml 算法原理剖析以及具体的源码实现分析

Explore ML Frameworks

All categories Trending ML Framework directory Insights