Michelle Lam

@mlam.bsky.social

940 Followers 250 Following

Stanford CS PhD student | hci, human-AI interaction (+ dance, design, doodling!) michelle123lam.github.io

Broader usage scenarios inclue multi-stakeholder collaboration (live mode, git for policy, policy forks, participatory maps) and model evaluation + auditing (policy test suite, policy audits)
An evaluation with 12 LLM safety experts found it was much easier to identify policy gaps and author policies with the system compared to in their normal work.
Policy maps chart LLM policy coverage over an unbounded space of model behaviors. Here, an AI practitioner is designing a policy for how an LLM should summarize violent text. Policy map abstractions (right) allow the policy designer to interactively author and test policies that govern a model’s behavior using if-then rules over concepts. The designer can create any desired concept by providing a simple text definition to capture cases of model behavior. Our Policy Projector tool (center) renders cases, concepts, and policies as visual map layers to aid iterative policy design.
High-level policies like "be helpful & polite" contrast against on-the-ground cases like "Gave unsolicited financial tips," "Avoided controversial topics," "Shared outdated medical advice," and "Amplified toxic user beliefs"