Human-Computer Interaction Talk Series: Prof. Kate Donahue, "Hypothesis generation surfaces interpretable signals that improve clinical risk prediction."

- Sponsor
- Siebel School of Computing and Data Science
- Speaker
- Prof. Kate Donahue
- Contact
- Tal August
- taugust@illinois.edu
- Views
- 2
- Originating Calendar
- Siebel School Speakers Calendar
Abstract: In recent years, algorithmic tools have become increasingly used in high-stakes decision settings. However, such tools are typically built on tabular, pre-defined features (e.g. age, sex). By contrast, human experts, such as physicians, case workers, or judges, may have rich, complex data with complementary information to the tabular features. For example, clinical notes may capture key patient details, such as unexpected surgical complications. A key challenge in human--AI complementarity is reconciling the information across these sources. Existing approaches apply complex learning methods to raw trace data (like clinical notes), resulting in predictions that are accurate but highly uninterpretable, which is often an infeasible solution in high-stakes decision settings. We develop an alternative approach that leverages hypothesis generation to distill rich expert trace data into a small number of interpretable binary features that can be reliably extracted by LLMs. In a surgical risk prediction setting, we show that our approach is competitive with black-box approaches. We apply this approach to a medical risk prediction setting, leveraging an internal surgical dataset from a large hospital setting. Finally, we explore the impact of algorithmic tools on this "human-generated” data: surgeons are increasingly using templates to assist in note-writing, which could reduce the amount of signal in the data. However, we show that the degree of templating is anti-correlated with risk, suggesting that surgeons judiciously modify templates when necessary. As a result, our approach to leverage information from notes is robust to the degree of templating in notes.
Bio: Kate Donahue is an assistant professor of computer science at University of Illinois Urbana-Champaign (UIUC) and a research affiliate at MIT. Previously, she was a METEOR postdoc at MIT, working with Manish Raghavan. Donahue works on algorithmic problems relating to the societal impact of AI such as human/AI collaboration, benchmark design, and incentives in federated learning and data sharing. She completed her PhD in computer Science at Cornell, advised by Jon Kleinberg. During her PhD, she interned at Google (with Kostas Kollias and Sreenivas Gollapudi), at Amazon (with Krishnaram Kenthapadi and Alexandra Chouldechova) and at Microsoft Research (with Solon Barocas, in the NYC FATE group, and with Nicole Immorlica and Brendan Lucier in the NE EconCS group). Donahue has been selected as a "Rising Star in EECS" and her PhD research has been supported by an NSF Graduate Research Fellowship.