fkie-cad/Logprep
log data pre processing, generation and shipping in python
Logprep helps operations and security teams automatically clean up, standardize, and route their log data from various sources like Kafka or S3. It takes raw, messy log entries and transforms them into structured, enriched messages, ready for analysis in tools like OpenSearch. This is for IT professionals who need to manage and process large volumes of log data for monitoring, troubleshooting, or security.
Available on PyPI.
Use this if you need to continuously collect, modify, and distribute log messages from multiple sources to different destinations, such as enriching IP addresses with geographic data or stripping sensitive fields.
Not ideal if you're looking for a simple log viewer or a one-time log file analysis tool without continuous processing needs.
Stars
35
Forks
10
Language
Python
License
LGPL-2.1
Category
Last pushed
Mar 17, 2026
Commits (30d)
0
Dependencies
34
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/data-engineering/fkie-cad/Logprep"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
PrefectHQ/prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
growthbook/growthbook
Open Source Feature Flags, Experimentation, and Product Analytics
koopjs/koop
Transform, query, and download geospatial data on the web.
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
dagster-io/dagster
An orchestration platform for the development, production, and observation of data assets.