Reducing Moral Ambiguity in Partially Observed Human-Robot Interactions
Artist: Ari Weinkle
Reducing Moral Ambiguity in Partially Observed Human-Robot Interactions
Benn, C & Grastien, A 2021, ‘Reducing Moral Ambiguity in Partially Observed Human-Robot Interactions’, Advanced Robotics.
Robots need to abide by moral constraints. In interacting with humans though, it is not always sufficient for an agent to perform normatively permissible actions: they should be unambiguously so. This generally requires the agent to make suboptimal decisions. We consider a framework in which the agent is supposed to choose amongst a collection of permissible and impermissible options. In this framework, an observer sees what option was selected, but is unable to distinguish between some of the options. We consider several definitions of Virtue Signalling: in the strict sense, the agent is not allowed to select any option that looks like an impermissible one. As the strict constraint is often too stringent in practice, we enrich the domain with a preference relation (from the standpoint of the agent) between the options, so that a rational agent will try to choose their preferred option (while still remaining normatively acceptable). We assume that the observer partially knows these preferences. In this context, an option that could be misinterpreted as impermissible remains normatively acceptable if it is clear that the impermissible options were dominated by better options. We study different interpretations of Virtue Signalling. We also extend the definition for uncertain contexts in which the preference relation is best represented probabilistically. We propose to label an option acceptable if there is a low enough probability that the preferred option (amongst those matching the observation) is impermissible. As we believe that not all situations will require robotic agents to signal their virtue, we conclude by proposing experimental setups to verify the relevance of this theory in different contexts.
See the paper here.