yihuaihong/ConceptVectors
[EMNLP 2025 Main] ConceptVectors Benchmark and Code for the paper "Intrinsic Evaluation of Unlearning Using Parametric Knowledge Traces"
This project helps researchers and developers working with large language models (LLMs) to intrinsically evaluate how effectively models "unlearn" specific concepts. It provides a benchmark dataset and code to analyze changes within the model's parameters when knowledge is supposed to be removed. The output helps model developers understand if unlearning methods truly erase information, making LLMs safer and more reliable.
No commits in the last 6 months.
Use this if you are developing or evaluating unlearning techniques for LLMs and need to assess whether specific concepts are genuinely removed from the model's internal knowledge representation, beyond just behavioral tests.
Not ideal if you are a general LLM user or a practitioner only interested in the high-level behavior of an unlearned model without delving into its internal parametric changes.
Stars
39
Forks
5
Language
Jupyter Notebook
License
CC-BY-4.0
Category
Last pushed
Aug 20, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/yihuaihong/ConceptVectors"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.