SETL-Framework/setl
A simple Spark-powered ETL framework that just works 🍺
This framework helps data scientists and data engineers build and manage data processing pipelines more effectively. You provide raw data sources and data transformation rules, and it outputs cleaned, transformed datasets ready for analysis or storage. It's designed for professionals who need to develop and maintain robust ETL (Extract, Transform, Load) solutions using Apache Spark.
185 stars. No commits in the last 6 months.
Use this if you are a data scientist or data engineer working with Scala and Apache Spark, and you need a structured way to build, organize, and debug your data transformation projects.
Not ideal if you are not using Scala or Apache Spark, or if your data processing needs are very simple and don't require a full ETL framework.
Stars
185
Forks
33
Language
Scala
License
Apache-2.0
Category
Last pushed
Oct 02, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/data-engineering/SETL-Framework/setl"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
supabase/supabase-py
Python Client for Supabase. Query Postgres from Flask, Django, FastAPI. Python user...
elastic/eland
Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
sicara/sicarator
Instant Setup & Best Quality for Data Projects!
Omoluabi1003/ETL-AI
AI-enabled data pipeline framework for transforming raw data into structured, decision-ready intelligence
sosannaunregenerate143/gcp-financial-data-platform
Build and manage a production-grade financial data platform on GCP with integrated ingestion,...