microsoft/NeMoEval
A Benchmark Tool for Natural Language-based Network Management
This tool helps network engineers and operations teams evaluate how well natural language instructions can manage and analyze network operations. You provide natural language queries about network traffic or lifecycle management, and it assesses the quality of the generated code or actions. The output helps you understand the effectiveness of using AI for network tasks.
No commits in the last 6 months.
Use this if you are a network engineer, network architect, or operations manager evaluating the potential of large language models to assist with network traffic analysis or network lifecycle management tasks.
Not ideal if you need a plug-and-play solution to directly manage a live network, as this is a benchmark and evaluation tool, not an operational one.
Stars
29
Forks
7
Language
Python
License
MIT
Category
Last pushed
Jun 18, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/microsoft/NeMoEval"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
FudanSELab/ClassEval
Benchmark ClassEval for class-level code generation.
apartresearch/specificityplus
👩💻 Code for the ACL paper "Detecting Edit Failures in LLMs: An Improved Specificity Benchmark"
claws-lab/XLingEval
Code and Resources for the paper, "Better to Ask in English: Cross-Lingual Evaluation of Large...
HICAI-ZJU/SciKnowEval
SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models
nicolay-r/RuSentRel-Leaderboard
This is an official Leaderboard for the RuSentRel-1.1 dataset originally described in paper...