Interactive Data Lab

@idl.uw.edu

547 Followers 41 Following

Visualization & data analysis research at the University of Washington. In a prior life was the Stanford Vis Group. https://idl.uw.edu

Chart costs conditioned on task type, comparing Kim et al.’s experimental results to DracoGPT-Rank (A) and DracoGPT-Recommend Vega-Lite (B). For value tasks, results from both GPT4-Turbo models moderately correlate with a model fit to human performance data, but do not significantly correlate for summary tasks.
Overview of the DracoGPT-Rank pipeline. (1) User provides prompt templates for an LLM to rank chart pairs; (2) Draco featurizes charts and produces feature vectors consisting of constraint counts; (3) Draco learns constraint weights over LLM-labeled chart pairs by fitting a RankSVM model; (4) The fitted Draco model can be applied to score charts. Results at each stage of the pipeline afford insight into LLM ranking preferences.