Health policy, technology, labor, and democracy. Political economy of health & collective governance for tech. MD/MPH. He/him/his. ਸਰਬੱਤ ਦਾ ਭਲਾ Let’s build a healthier world together!
![A politics of refusal
To build and govern technologies in healthcare, we cannot interrogate these tools as simply technical elements solving technical challenges. They exist embedded within and in relation with society as a singular coherent system. This sociotechnical understanding recognizes that the performance and impacts of technologies are results of both social dynamics and technical designs. They are inextricable from each other [5].
As such, we must recognize that these models are not deployed in isolation- a critical analysis of the context for the care settings in which interactions occur must also consider the relationships and tensions that exist outside of care settings. These are the worlds that shape the thoughts, stressors, and priorities of our patients who must continue living with their health after leaving the clinic, for caregivers who struggle with the burdens of their own lives as they try to help carry the burdens of others, and for providers who must navigate the hurdles and constraints of healthcare as they strive to practice medicine.
Relationship-Centered Care will remain a distant ambition if attempts at automation are not designed with an intention to augment the sense of agency for patients, caregivers, and providers. As clinicians, technologists, researchers, and compassionate members of society, we must therefore practice a politics of refusal when faced with tools that will further extract agency from these groups. We must set these boundaries firm, and build on histories of design knowledge to innovate new tools to make RCC a reality in the world of AI.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:6fbh3l4c6dxom5wc3b3n4vns/bafkreian3qzyrpqivupfxpbkqfg47zlcwc6a56r7fjz66z4xvqeic3tr4u)
![AI agents are built on principles of optimization, but in contrast to the early tools of algorithmic management, the architecture of contemporary models obfuscates their behavior and methodology [13]. Their underlying LLMs use hundreds of billions of parameters built to predict words in sequence, and demonstrate impressive capabilities in mimicking human language and tasks. With additional layers of fine tuning, filters, system prompts, and more, developers steer LLM ‘behavior’ to more closely align with user values and accomplish such goals [14]. Yet, LLMs may demonstrate their own emergent preferences and behavioral biases, and actively produce outputs to conceal such behavior [25,30,31].
In clinical settings, the weight of these decisions can cost patients their lives, and significant progress remains in addressing the risks that patients, caregivers, and providers are exposed to by these models. The framing of relationship-centered care (RCC) can help in catalyzing new practices in strengthening human relations in clinical settings. To underline this framing, I propose orienting methodologies to seriously consider the nature of human agency. Specifically, I call for practices in the design, deployment, and evaluation of algorithmic models to characterize and measure how a model will materially enhance the human agency experienced by patients, caregivers, and providers. By embedding a mandate to augment human agency, I argue that any approach to RCC that strips or fails to preserve their sense of agency cannot meaningfully enhance the human connections in healthcare, and must not advance to the clinical setting.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:6fbh3l4c6dxom5wc3b3n4vns/bafkreicvhch225kihmdtghfrjkafmjna4ed5nge4nzc7t3rfosz6g4pkse)
Health policy, technology, labor, and democracy. Political economy of health & collective governance for tech. MD/MPH. He/him/his. ਸਰਬੱਤ ਦਾ ਭਲਾ Let’s build a healthier world together!