Lucy Li

@lucy3.bsky.social

5851 Followers 668 Following

Postdoc at UW NLP 🏔️. #NLProc, computational social science, cultural analytics, responsible AI. she/her. Previously at Berkeley, Ai2, MSR, Stanford. Incoming assistant prof at Wisconsin CS. lucy3.github.io

an image of a sticker showing a badger with red and white hat hugging a jar of hazelnut spread branded as "slai"
a work in progress (blue lines) of a wisconsin badger with a jar of hazelnut spread with the research group logo "slai" on the front of the jar.
concept art for a badger hugging a jar with a rough sketch of the wisconsin social language and ai logo as the brand label
a screenshot of "your ip address has been blocked please contact dl-support@acm.org"
Question: How many dots did the student include in their array? For an erroneous student response: Model answer: 12. True answer: The student didn't include an array. True answer for a non-erroneous student response: The student included 12 dots in their array. Question: How many squares did the student draw to show the number of cups of red paint? For an erroneous student response: Model answer: The student drew 9 squares to show the number of cups of red paint. True answer: The student drew 12 squares to represent the cups of red paint. True answer for a non-erroneous student response: The student drew 9 squares to show the number of cups of red paint.
A bar chart disaggregating results for four VLMs across different question types. Content description QA consistently drives the gap in VLM performance between student responses that contain errors versus those that do not. In addition, questions related to students’ correctness and errors are still the most difficult.
Title, author list, and two figures from the paper. Title: The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors Authors: Li Lucy, Albert Zhang, Nathan Anderson, Ryan Knight, Kyle Lo Figure 1: On the left is a math problem, where students are asked to draw x < 5/2 on a number line. The right side shows two example student responses that differ in correctness. DrawEduMath pairs each math problem with one student response, and prompts VLMs to answer questions about the student response. Figure 2: VLMs consistently perform worse on answering DrawEduMath benchmark questions pertaining to erroneous student responses. Performance on non-erroneous student responses is labeled with specific VLMs’ names; that same model’s performance on erroneous student responses is directly below.
a screenshot of gemini chat history, where today the user prompted the model "Do you train on my inputs" and yesterday the user prompted the model "university of buffalo has got have real buffalo"
A staircase in the new School of Computer, Data & Information Sciences building at Wisconsin Madison. Tan wood structures surround tapestry art and a small indoor garden.
A white llama in a cafe setting, peering over the counter at the barista bartender.
a screenshot of code that says "from turtle import pd" "import torch" "from transformers import..."
pink frosted pop tarts with rainbow sprinkles
a white ragdoll cat rolling in a bathtub, with her belly upwards and arms and legs flailing
screenshot of text that says "*** System restart required *** =============== HELLO ==================" followed by an scared-looking ASCII cat
A screenshot of the title, authors, abstract, and figure 1 of the paper. Text in the abstract begins with: "Conventional bag-of-words approaches for topic modeling, like latent Dirichlet allocation (LDA), struggle with literary text. Literature challenges lexical methods because narrative language focuses on immersive sensory details instead of abstractive description or exposition: writers are advised to show, don’t tell. We propose Retell, a simple, accessible topic modeling approach for literature. Here, we prompt resource-efficient, generative language models (LMs) to tell what passages show, thereby translating narratives’ surface forms into higher-level concepts and themes."
a screenshot of an overleaf latex document titled "Lucy Dissertation" with 299 latex errors/warnings
Wisconsin-Madison's tree-filled campus, next to a big shiny lake
Green chili chicken tacos with avocado
A screenshot of Ai2 bluesky account post about a new "Ai2 Paper Finder", followed by my post about having tip-of-the-tongue paper retrieval problems
Figure caption: Fig. 2 | Collective outcomes in H–M social systems differ from those in human-only systems. Machines behave differently from humans, and in social systems with covert artificial agents the collective outcomes will differ simply because of this fact, —even if humans, unaware of the presence of machines, do not change their behaviour—. Furthermore, the two types of actors and their interactions are interdependent and influence each other. Thus, suspicion or awareness of machine presence can change human behaviour and interacting with a machine and observing M–M interactions can influence how humans act towards each other. Order of difference. First: Machine behaviour differs from human behaviour. Second: Humans interact differently with machines than with humans. Machines interact even if they are not explicitly designed to do so. Third: Human behaviour changes with the presence of machines, Human interactions are indirectly affected by machines and their interactions.
a screenshot of the drawedumath website, showing paper title Evaluating Vision Language Models with Expert-Annotated Students’ Hand-Drawn Math Images and author list Sami Baral, Li Lucy, Ryan Knight, Alice Ng, Luca Soldaini, Neil Heffernan, Kyle Lo with affiliations at WPI, Berkeley, Insource Services, Teaching Lab, and Ai2. Below that is an image showing an example image with question-answer pairs from the dataset.
a white cat paw next to a cursor on a computer monitor
drake meme, where the top shows drake gesturing "no" to the OLMo paper poster presented at a conference and the bottom shows him gesturing "yes" to a poster for OLMo 2 designed like the 2 fast 2 furious movie poster
a paper titled 2 OLMo 2 Furious co-authored by many people from the Allen Institute for AI and UW and NYU.
a screenshot of a paper title that says "Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning" authored by people at Princeton and Tsinghua, including Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen
A low sun peeking through trees. Ground covered in snow
an instant noodle packet, specifying chapagetti (jjajang noodle)
colorful succulents on patio table and chairs after rain
A screenshot of a breakdown that says "Here's everything I detected in your voice" with 49% English, 22% Korean, 14% Hindi/Urdu
A screenshot of "Here's everything I detected in your voice", where the results say 43% Persian, 19% English, and 18% Spanish
A screenshot of text that says "Hi Lucy, Thank you for contributing photos to eBird and the Macaulay Library. This November, we released a new version of Photo ID in Merlin, and we wanted to extend our gratitude for the important part you played in the creation of this new resource. 1 of your photos were used to train Merlin to identify birds in photos."
a photo of cat paws on a laptop keyboard, with one paw very close to the on/off button
a screenshot of a tweet that says 🚨 In 2025, @emnlpmeeting will take place in Suzhou, China from Nov. 5-9! with a picture of a. map and the conference venue, International Expo Centre
A screenshot of a pdf showing the title and abstract of a paper titled Content Analysis of Textbooks via Natural Language Processing: Findings on Gender, Race, and Ethnicity in Texas U.S. History Textbooks
A screenshot of the top of the first page of a paper titled Gender and Representation Bias in GPT-3 Generated Stories
A screenshot of the top of the first page of a paper titled AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters
A figure showing models’ problem solving performance, based on problems' grade levels of skills/concepts inferred by a language model.
A table of common reasons for why generated problems have no or partial alignment, obtained via open coding of teachers’ explanations. Provided examples in each row are cases where GPT-4 judges a problem to be fully aligned, but teachers do not.
A table showing summary data statistics from Achieve the Core and MathFish.
An example of a math problem involving operations in base ten in our dataset, along with standards it does and doesn't align with. This figure illustrates two task formats: verification and tagging.
a meme of a car driving on a road labeled "me" and two directions. One direction is labeled "journal papers with no deadlines" and another is labeled "conference papers with deadlines". The car is swerving to the latter
my white cat with dark lynx shaped face markings and blue eyes
a heatmap showing filtering rates for different subregions of the world (rows) and different filters (columns). In particular, eastern asia is filtered more than other subregions.
a table showing that game developers, game designers, data scientists, and software engineers are preferred by a classifier trained on OpenWebText2
a screenshot of the first page of a research paper titled "AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters" with a diagram showing a paraphrased excerpt for a website's about page, with common stated social dimensions emphasized with highlighting. The dimensions are: individuals or organizations, social roles, geographic locations, and topical interests.
A white cat with blue eyes and shocked expression next to a Christmas tree
A screenshot of TripAdvisor reviews of hotels that mention ghosts
A bar chart titled "Should replies change based on the identity-related feature in the message?" with five sets of bars, corresponding to names, roles, countries, dialect, and style. More judges favor always adapting to style than other types of features, while many responded "never" adapting to names.
My white cat smiling at me from the doorway
A white cat with her face in a cardboard box with holes on the outside and little balls on the inside
A white cat with dark ears in a giant banana
My charcuterie board containing salami, two types of round crackers, a hard cheese and Brie, strawberries, dried apricots, walnuts
Oregon ocean, foggy and gray yet so much green forest
Colorful chairs overlooking docked boats in Lake Union, Seattle