Trending Data Engineering Tools

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream,...

Data Contracts engine for the modern data stack. https://www.soda.io

visual data prep powered by python

Transform, query, and download geospatial data on the web.

🎲 Business Logic Code in a flow!

High Performance Storage and Streaming Solution for Data Acquisition Systems

Python Streaming DataFrames for Kafka

Use SQL to instantly query AWS resources across regions and accounts. Open...

The best place to learn data engineering. Built and maintained by the data...

Hop Orchestration Platform

Use SQL to instantly query repositories, users, gists and more from GitHub....

A fast and versatile ETL tool that can transfer data between RDBMS and NoSQL...

OpenSnowcat Enricher (Apache 2.0 License)

In this repository, you will find varies demo and presentations I have...

Code Snippets, Designs, and other things about building a Data Analytics...

asyncIO, Github Actions, GCP, dbt, Terraform, Docker

Community supported integrations for the Dagster platform.

Codebase for CCAO data infrastructure construction and management

A data lineage tool detects table dependencies from rendered SQL statements.

Convert your audio files into DuckDB or Parquet files

Stop Guessing. Start Dominating Your Market. The only data platform built...

Airflow provider package for SAP HANA

Documentation repository for the Egeria project.

The Public Utility Data Liberation Project provides analysis-ready energy...

log data pre processing, generation and shipping in python

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI

Dataform is a framework for managing SQL based data operations in BigQuery

ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data...

Data Pipeline Automation Framework to build MCP servers, data APIs, and data...

GovHub - Transformando Dados em Valor para Gestão Pública

Scalable and efficient data transformation framework - backwards compatible with dbt.

A hybrid CLI/TUI tool written in Go for viewing Pokémon data from the...

End-To-End Data Engineering Project. Made to learn some common data...

Categorical Query Language IDE

end-to-end data engineering project to get insights from PyPi using python,...

Extract transform load CLI tool for extracting small and middle data volume...

Flowfile is a visual ETL tool and Python library combining drag-and-drop...

Privacy and Security focused Segment-alternative, in Golang and React

🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.

Watchmen Platform is a low code data platform for data pipeline, meta data...

Configurable Extract, Transform, and Load

tracebloc data pipeline for training/test dataset setup

A deterministic orchestrator for composable micro-workflows with reusable modules

The premier open source Data Quality solution

Library for describing data transformation pipelines by compositing simple...

One ETL tool to rule them all

Cool DE Projects

Egeria core

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

A machine-readable, human-editable database of the Yu-Gi-Oh! Trading Card...

Mox-Data.com is a cloud-based data ingestion tool used to process raw data...

Frontend & BFF (Backend for frontend) for Olake. This includes the UI code...

bboss-datatran 由 bboss 开源的数据采集&流批一体化工具，提供数据采集、清洗转换处理以及流批一体化计算功能；...

A python package to create a database on the platform using our moj data...

Addax Admin is a web-based management console for Addax ETL jobs, offering...

A tool for converting FERC filings published in XBRL into SQLite databases

Pull and standardize data on cloud compute resources.

Transformation, web frontend, and API for the hbz catalog as LOD

A curated list of awesome system integration software and resources.

A movie journal coupled with open IMDb data, and a Flask web-app for easy...

No-code ETL tool, based on onETL + PySpark

ReplicaDB is open source tool for database replication, designed for...

Resilient data pipeline framework running on Apache Spark

Lightweight REST for Java

Declarative text based tool for data analysts and engineers to extract,...

Git all remotes. git cli tool that targets both Github and Gitlab

An automatically-updated collection of wikitexts from Yugipedia. Part of YAML Yugi.

Apache DevLake is an open-source dev data platform to ingest, analyze, and...

Simple HWM Store backend

Códigos de captura e tratamento de dados da SMTR

Airflow DAGS for migrating and managing ILS data into FOLIO along with other...

Basic ETL Entity classes for onETL

Packages for the Tenzir ecosystem.

Horizon HWM Store for onETL

Frontend for Syncmaster, no-code ETL tool. WIP

A collection of tools for the BNA.

Parse judgements from the Transformation Engine and load them into MarkLogic...

A collection of tools for extracting FHIR resources and analytics services...

go-etl is a toolset for data extraction, transformation and loading.

First open-source data discovery and observability platform. We make a life...

Home of the Open Data Contract Standard (ODCS).

Desbordante is a high-performance data profiler that is capable of...

Dagster Labs' open-source data platform, built with Dagster.

The data-validation toolkit for enhanced dbt (data build tool) PR review

Snowflake Snowpark Python API

PyAirbyte brings the power of Airbyte to every Python developer.

(Spatial) data harmonisation with hale»studio (formerly HUMBOLDT Alignment Editor)

The Open-Source Enterprise Data Platform in a single Portal

Go client for Chalk

DataOps Observability Integration Agents are part of DataKitchen's Open...

TAK Compatible, browser based Common Operation Picture & Situational Awareness tool

Installer for DataKitchen's Open Source Data Observability Products. Data...

Contribute to dlt verified sources 🔥

Dataform Tools - VS Code extension to run and visualise Dataform data...

Server application to serve U.S. federal spending data via a RESTful API

Drop-in replacement for Apache Spark UI

Use @xonsh wherever you go through the SSH without installation on the host.

Command-line interface for downloading WARN Act notices of qualified plant...

ETL / ELT Framework powered by DuckDB, designed to seamlessly integrate and...

Ingestão, preparação e disponibilização gratuita de dados de CNPJs de...