How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review - Télécom Paris
Conference Papers Year : 2022

How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review

Abstract

The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation methods and cognitive constraints. Despite these advances, the community lacks a general vision of what and how cognitive biases affect explainability systems. To address this gap, we present a heuristic map which matches human cognitive biases with explainability techniques from the XAI literature, structured around XAI-aided decision-making. We identify four main ways cognitive biases affect or are affected by XAI systems: 1) cognitive biases affect how XAI methods are designed, 2) they can distort how XAI techniques are evaluated in user studies, 3) some cognitive biases can be successfully mitigated by XAI techniques, and, on the contrary, 4) some cognitive biases can be exacerbated by XAI techniques. We construct this heuristic map through the systematic review of 37 papers-drawn from a corpus of 285-that reveal cognitive biases in XAI systems, including the explainability method and the user and task types in which they arise. We use the findings from our review to structure directions for future XAI systems to better align with people's cognitive processes.
Fichier principal
Vignette du fichier
How Cognitive Biases Affect XAI-assisted Decision-making_rvwd.pdf (961.3 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03684457 , version 1 (01-06-2022)

Identifiers

Cite

Astrid Bertrand, Rafik Belloum, James Eagan, Winston Maxwell. How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review. AAAI/ACM Conference on Artificial intelligence, Ethics, and Society, Aug 2022, Oxford, United Kingdom. ⟨10.1145/3514094.3534164⟩. ⟨hal-03684457⟩
998 View
3164 Download

Altmetric

Share

More