realdatadriven/etlx
ETL / ELT Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.
This tool helps data professionals manage and automate their data pipelines. You provide data from various sources like databases or data lakes, along with Markdown files containing SQL and configuration details. It then processes and loads this data into your desired destination, providing a clear, auditable record of the entire workflow. This is ideal for data engineers, data analysts, or anyone responsible for ensuring data quality and availability.
Use this if you need to build and maintain data pipelines where transparency, reproducibility, and clear documentation are paramount, especially when working with SQL across multiple database systems.
Not ideal if you prefer graphical drag-and-drop interfaces for pipeline building or if your data integration needs are extremely simple and do not require extensive SQL transformations or auditing.
Stars
38
Forks
3
Language
Go
License
MIT
Category
Last pushed
Mar 18, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/data-engineering/realdatadriven/etlx"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PrefectHQ/prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
growthbook/growthbook
Open Source Feature Flags, Experimentation, and Product Analytics
koopjs/koop
Transform, query, and download geospatial data on the web.
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.