usnistgov/KAIROS
Scoring and analysis software for the evaluation of Knowledge Directed Artificial Intelligence Reasoning Over Schemas (KAIROS)
This software suite helps researchers and participants in the KAIROS program evaluate the performance of Knowledge Directed Artificial Intelligence Reasoning Systems. It takes in system outputs (SDF files and LDC annotations) from TA1 and TA2 systems and produces detailed score reports and statistical spreadsheets. This tool is designed for KAIROS program participants who need to assess and compare their AI system's ability to extract and reason over complex knowledge graphs.
No commits in the last 6 months.
Use this if you are a KAIROS program participant needing to formally evaluate the output of your TA1 or TA2 AI systems against NIST's criteria.
Not ideal if you are looking for a general-purpose AI model evaluation tool or if you are not involved in the KAIROS research program.
Stars
8
Forks
4
Language
Python
License
—
Category
Last pushed
May 03, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/usnistgov/KAIROS"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
nityansuman/marvin
Web app to automatically generate subjective or an objective test and evaluate user responses...
shibing624/judger
自动作文评分工具，支持中文、英文作文智能评分，支持评分模型自训练，支持WEKA处理模型数据，支持自定义评分算法。java开发。
shubhpawar/Automated-Essay-Scoring
Automated Essay Scoring on The Hewlett Foundation dataset on Kaggle
antrixsh/trusteval
Enterprise LLM Evaluation & Responsible AI Framework — Benchmark bias, hallucination, PII...
samiali12/debateai-server
A FastAPI-powered backend that manages structured debates, analyzes arguments, and generates...