Chabane/bigdata-playground
A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL
This project provides a comprehensive big data environment designed for engineers and architects to experiment with batch processing, real-time streaming, and machine learning workflows. It integrates various popular big data technologies, allowing users to input raw data and test the output of complex data processing jobs locally or on a resource manager like Yarn. It's ideal for those setting up or evaluating big data application stacks.
210 stars. No commits in the last 6 months.
Use this if you are a big data engineer or architect who needs a pre-configured, end-to-end environment to prototype, test, and develop big data applications using a variety of tools.
Not ideal if you are looking for a simple, single-purpose tool to solve a specific data problem or if you lack experience with big data infrastructure and development.
Stars
210
Forks
74
Language
TypeScript
License
Apache-2.0
Category
Last pushed
Feb 01, 2019
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/mlops/Chabane/bigdata-playground"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
jrzaurin/ml-pipeline
Using Kafka-Python to illustrate a ML production pipeline
jaumpedro214/ml-streming-kafka-cdc
Deploying a Machine Learning model streaming application with Apache Kafka
IPVS-AS/MMP-Backend
A Model Management Platform (MMP) for Industry 4.0 Environments (Backend)
jmeisele/ml-ops-kafka
Kafka variant of the MLOps Level 1 stack
zzzzz1st/predictorML
Machine learning and prediction service for Niagara NX platform.