weld-project/weld
High-performance runtime for data analytics applications
Weld helps data engineers and developers accelerate data analytics applications that combine multiple data processing libraries. It takes complex data workflows as input, optimizes the entire computation across different frameworks to reduce data movement, and then executes it for faster results. The output is significantly improved performance for data-intensive tasks.
3,003 stars. No commits in the last 6 months.
Use this if you are building data analytics applications that suffer from performance bottlenecks due to extensive data movement between different libraries or frameworks.
Not ideal if your data analytics tasks are simple, single-library operations, or if you are not experiencing significant performance issues related to data transfer.
Stars
3,003
Forks
253
Language
Rust
License
BSD-3-Clause
Category
Last pushed
Jun 22, 2022
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/data-engineering/weld-project/weld"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
arkflow-rs/arkflow
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing...
strake-data/strake
The Data Layer for AI. A high-performance federated SQL engine that gives AI agents governed,...
idlab-discover/RustiFlow
Flow feature extraction tool built in Rust using eBPF
torchforge-rs/torchforge-data
Zero-copy, streaming data pipeline for edge-native machine learning in Rust.
SunDoge/fastdata
Fast Data Pipeline with TFRecord and io-uring