All Data Engineering Tools

Prefect is a workflow orchestration framework for building resilient data...

Open Source Feature Flags, Experimentation, and Product Analytics

Transform, query, and download geospatial data on the web.

Python ETL framework for stream processing, real-time analytics, LLM...

An orchestration platform for the development, production, and observation...

Python Client for Supabase. Query Postgres from Flask, Django, FastAPI....

data load tool (dlt) is an open source Python library that makes data...

Meltano: the declarative code-first data integration engine that powers your...

locopy: Loading/Unloading to Redshift and Snowflake using Python.

Convert documents to structured data effortlessly. Unstructured is...

Hop Orchestration Platform

Apache Superset is a Data Visualization and Data Exploration Platform

The leading data integration platform for ETL / ELT data pipelines from...

Clean APIs for data cleaning. Python implementation of R package Janitor

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability,...

The Public Utility Data Liberation Project provides analysis-ready energy...

Change data capture for a variety of databases. Please log issues at...

Quilt is a Scientific Data Management Platform on AWS that helps teams and...

ingestr is a CLI tool to copy data between any databases with a single...

Apache DevLake is an open-source dev data platform to ingest, analyze, and...

A dbt adapter for Databricks.

Egeria core

Flink CDC is a streaming data integration tool

phpMyFAQ - Open Source FAQ web application for PHP 8.3+ and MySQL,...

The AI-Native Infrastructure for Enterprise Apps. Powered by ObjectStack...

SeaTunnel is a multimodal, high-performance, distributed, massive data...

Blazing-fast Data-Wrangling toolkit

OpenMetadata is a unified metadata platform for data discovery, data...

OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg or Plain...

Supplementary Materials for the The Complete dbt (Data Build Tool) Bootcamp...

A Spark plugin for reading and writing Excel files

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

A high-performance observability data pipeline.

Database Reporting Tool and Tasks (.Net)

Dataform is a framework for managing SQL based data operations in BigQuery

Know your data better！Datavines is Next-gen Data Observability Platform,...

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL...

A fast and versatile ETL tool that can transfer data between RDBMS and NoSQL...

Data Contracts engine for the modern data stack. https://www.soda.io

CrateDB is a distributed and scalable SQL database for storing and analyzing...

Data pipelines for cloud config and security data. Build cloud asset...

A local-first workflow engine built the way it should be: declarative,...

Event streaming platform for agents, apps, and analytics. Continuously...

A local-first workflow engine built the way it should be: declarative,...

Free universal database tool and SQL client

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream,...

Relational data pipelines for the science lab

The leader in Customer Data Infrastructure

Fast, Simple and a cost effective tool to replicate data from Postgres to...

lakeFS - Data version control for your data lake | Git for data

KNIME Analytics Platform

🧙 Build, run, and manage data pipelines for integrating and transforming data.

ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data...

A wrapper around the python-irodsclient to allow for easy interaction with...

Scalable and efficient data transformation framework - backwards compatible with dbt.

visual data prep powered by python

Wikidata and Wiktionary language data extraction

Automation fetching data for Vietlott. Just for fun.

The premier open source Data Quality solution

The dbt-native data observability solution for data & analytics engineers....

A utility library for comparing and synchronizing different datasets.

Save or export your private and public Slack messages, threads, files, and...

🎲 Business Logic Code in a flow!

An extensible framework for Personal Data Management.

Consolidate, enrich and republish the data gathered by warn-scraper

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML,...

Sling is a CLI tool that extracts data from a source storage/database and...

Apache Hamilton helps data scientists and engineers define testable,...

⚡ Fastest SQL ETL pipeline in a single C++ binary, built for stream...

Snowflake Snowpark Python API

数据流引擎是一款面向数据集成、数据同步、数据交换、数据共享、任务配置、任务调度的底层数据驱动引擎。数据流引擎采用管执分离、多流层、插件库等体系应对大规模数...

Command-line interface for downloading WARN Act notices of qualified plant...

Use data to tell stories.An intelligent Visualization Narrative Development...

Server application to serve U.S. federal spending data via a RESTful API

A unified interface for distributed computing. Fugue executes SQL, Python,...

Desbordante is a high-performance data profiler that is capable of...

Open-source data framework for biology. Context and memory for datasets and...

dbt-sugar is a CLI tool that allows users of dbt to have fun and ease...

Inspection of tabular (csv, xls-like) files to guess the columns' content

ReplicaDB is open source tool for database replication, designed for...

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Fancy stream processing made operationally mundane

Yet another redundant workflow engine

A collection of tools for extracting FHIR resources and analytics services...

Use SQL to instantly query AWS resources across regions and accounts. Open...

The best place to learn data engineering. Built and maintained by the data...

为DataX(https://github.com/alibaba/DataX) 提供远程多语言调用（ThriftServer，HttpServer）...

🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.

log data pre processing, generation and shipping in python

The Fabric Metadata-Driven Framework (FMD) is a cutting-edge accelerator...

First open-source data discovery and observability platform. We make a life...

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

A Python package to load complex XML files into a relational database

A curated list of awesome system integration software and resources.

PyAirbyte brings the power of Airbyte to every Python developer.

100+ SQL Scripts - PostgreSQL, MySQL, Oracle, Google BigQuery, MariaDB, AWS...

R interface for Apache Spark

Icebird: JavaScript Iceberg Client

Official Neo4j JDBC Driver

Contribute to dlt verified sources 🔥