chonkie and rag-chunk
These are complements: Chonkie is a production-ready ingestion library for RAG pipelines, while rag-chunk is a benchmarking CLI tool for evaluating and selecting optimal chunking strategies before deployment.
About chonkie
chonkie-inc/chonkie
🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines
This is a lightweight tool for developers building Retrieval-Augmented Generation (RAG) applications. It takes various forms of text data, processes it by intelligently splitting it into smaller, meaningful parts (chunks), and then refines and embeds these chunks. The output is optimized text chunks ready to be stored in a vector database for efficient retrieval by large language models.
About rag-chunk
messkan/rag-chunk
A Python CLI to test, benchmark, and find the best RAG chunking strategy for your Markdown documents.
This tool helps developers working with Retrieval-Augmented Generation (RAG) by optimizing how text documents are broken down into smaller, searchable pieces. You input your Markdown documents, and the tool evaluates various chunking methods, showing you which one performs best in terms of retrieval accuracy. It's designed for machine learning engineers and AI practitioners building or fine-tuning RAG systems.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work