Nils Feldhus

@nfel.bsky.social

329 Followers 436 Following

Postdoctoral Researcher @ University of Groningen / GroNLP (@gronlp.bsky.social) interested in the interpretability and analysis of language models. Ex- BIFOLD, TU Berlin, DFKI. https://nfelnlp.github.io/

Concept description evaluation techniques categorized by metric, study, and the underlying quality being measured. Metrics are grouped into conceptual families: predictive simulation, input-based evaluation, output-based evaluation, semantic similarity, and human judgment.

Concept description techniques categorized by component/abstraction (Neurons, SAEs, Circuits, Attention Heads), description source, and target dataset.

Overview of descriptions for model components (neurons, attention heads) and model abstractions (SAE features, circuits).

Overview of our experimental pipeline, exemplified with the GECO dataset (Wilming et al., 2024). We begin by obtaining predictions for male/female sentence pairs. We then use feature attribution methods to explain the predictions and evaluate the explanations using various metrics. We finally analyze the distributions of evaluation scores per each metric for male and female sentences and observe if the evaluations differ significantly between the two genders, indicating gender bias and disparity in explanations.

Cross-Refine example of the question ``Where would you borrow coffee if you do not have any?'' from ECQA. The initial explanation by the generator has been accurately corrected and refined based on the feedback and explanations provided by the critic.