spapicchio/QATCH
Official implementation of QATCH: Benchmarking SQL-centric tasks with Table Representation Learning Models on Your Data
This project helps data engineers and machine learning practitioners benchmark how well different Table Representation Learning (TRL) models and Large Language Models (LLMs) understand and answer questions about their unique, private databases. It takes your proprietary database as input and generates a tailored set of SQL queries and natural language questions to test models. The output is a detailed evaluation showing where models succeed or fail, helping you compare models, validate their quality, or monitor performance over time.
No commits in the last 6 months.
Use this if you need to objectively compare, validate, or continuously monitor the performance of TRL models or LLMs on SQL-centric tasks like Question Answering and Semantic Parsing using your own specific datasets.
Not ideal if you are looking for a general-purpose model training or fine-tuning tool, or if you don't work with table representation learning models or LLMs for database interaction.
Stars
32
Forks
—
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 17, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/spapicchio/QATCH"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
PaddlePaddle/RocketQA
🚀 RocketQA, dense retrieval for information retrieval and question answering, including both...
shuaihuaiyi/QA
使用深度å¦ä¹ 算法实现的ä¸æ–‡é—®ç”系统
allenai/deep_qa
A deep NLP library, based on Keras / tf, focused on question answering (but useful for other NLP too)
worldbank/iQual
iQual is a package that leverages natural language processing to scale up interpretative...
fhamborg/Giveme5W1H
Extraction of the journalistic five W and one H questions (5W1H) from news articles: who did...