Description:

AI agents increasingly support people at work and in their everyday lives by performing tasks autonomously. Consider, for example, an AI agent that automatically manages your e-mails and answers them on your behalf. In such critical tasks, human oversight of the system is required to mitigate risk by ensuring responsibility and preserving human judgment in decisions affecting people’s lives. However, for human oversight to achieve its goals, it must be designed to be effective; you need to detect inaccurate or inadequate system outputs and intervene appropriately to prevent negative outcomes. At the same time, it is argued that effectiveness also depends on the sociotechnical conditions under which oversight occurs and is closely linked to moral responsibility: to be morally responsible, you must satisfy both an epistemic condition (understanding why the agent sent inappropriate e-mails and how it could have been prevented) and a causal condition (instructions that caused or contributed to the inappropriate response).

The goal of this project is to investigate how moral responsibility is perceived in human oversight settings. Your task is to design and implement an ecologically valid empirical study in which participants perform a realistic human oversight task. The study design should allow for measuring moral responsibility and generalizing it to real-world settings. The findings will contribute to our understanding of how moral responsibility is attributed in human oversight tasks and can be measured in user studies. The results may inform the design of future oversight systems, including high-stakes domains such as education, finance, healthcare, or law.

Requirements:

• Experience or strong interest in conducting empirical user studies and data analysis
• Good programming skills (e.g., Web-based, Python)

References:

• https://doi.org/10.48550/arXiv.2602.10701 
• https://doi.org/10.1145/3630106.3659051 
• https://doi.org/10.1016/j.clsr.2022.105681 
• https://doi.org/10.48550/arXiv.2510.19512 
• https://plato.stanford.edu/entries/computing-responsibility/ 

Contact:

Cedric Faas: faas@cs.uni-saarland.de