Explanation and trust: what to tell the user in security and AI?

  • 13 Citations

Abstract

There is a common problem in artificial intelligence (AI) and information security. In AI, an expert system needs to be able to justify and explain a decision to the user. In information security, experts need to be able to explain to the public why a system is secure. In both cases, an important goal of explanation is to acquire or maintain the users’ trust. In this paper, I investigate the relation between explanation and trust in the context of computing science. This analysis draws on literature study and concept analysis, using elements from system theory as well as actor-network theory. I apply the conceptual framework to both AI and information security, and show the benefit of the framework for both fields by means of examples. The main focus is on expert systems (AI) and electronic voting systems (security). Finally, I discuss consequences of the analysis for ethics in terms of (un)informed consent and dissent, and the associated division of responsibilities.
Original languageUndefined
Pages (from-to)53-64
Number of pages12
JournalEthics and information technology
Volumeonline pre
Issue number1
DOIs
StatePublished - Mar 2011

Fingerprint

artificial intelligence
data security
Artificial intelligence
Security of data
knowledge-based system
Expert systems
explanation
actor-network-theory
Circuit theory
user
System theory
system theory
Security systems
analysis
voting
moral philosophy
electronics
system
security
responsibility

Keywords

  • EWI-18815
  • SCS-Cybersecurity
  • Trust
  • explanation
  • Systems theory
  • Information Security
  • confidence
  • METIS-271135
  • Actor-Network Theory
  • Expert Systems
  • IR-74620
  • informed consent

Cite this

Pieters, Wolter / Explanation and trust: what to tell the user in security and AI?

In: Ethics and information technology, Vol. online pre, No. 1, 03.2011, p. 53-64.

Research output: Scientific - peer-reviewArticle

@article{31158c0934244f068d93bfb120e9b1eb,
title = "Explanation and trust: what to tell the user in security and AI?",
abstract = "There is a common problem in artificial intelligence (AI) and information security. In AI, an expert system needs to be able to justify and explain a decision to the user. In information security, experts need to be able to explain to the public why a system is secure. In both cases, an important goal of explanation is to acquire or maintain the users’ trust. In this paper, I investigate the relation between explanation and trust in the context of computing science. This analysis draws on literature study and concept analysis, using elements from system theory as well as actor-network theory. I apply the conceptual framework to both AI and information security, and show the benefit of the framework for both fields by means of examples. The main focus is on expert systems (AI) and electronic voting systems (security). Finally, I discuss consequences of the analysis for ethics in terms of (un)informed consent and dissent, and the associated division of responsibilities.",
keywords = "EWI-18815, SCS-Cybersecurity, Trust, explanation, Systems theory, Information Security, confidence, METIS-271135, Actor-Network Theory, Expert Systems, IR-74620, informed consent",
author = "Wolter Pieters",
note = "Open Access",
year = "2011",
month = "3",
doi = "10.1007/s10676-010-9253-3",
volume = "online pre",
pages = "53--64",
journal = "Ethics and information technology",
issn = "1388-1957",
publisher = "Springer Netherlands",
number = "1",

}

Explanation and trust: what to tell the user in security and AI? / Pieters, Wolter.

In: Ethics and information technology, Vol. online pre, No. 1, 03.2011, p. 53-64.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - Explanation and trust: what to tell the user in security and AI?

AU - Pieters,Wolter

N1 - Open Access

PY - 2011/3

Y1 - 2011/3

N2 - There is a common problem in artificial intelligence (AI) and information security. In AI, an expert system needs to be able to justify and explain a decision to the user. In information security, experts need to be able to explain to the public why a system is secure. In both cases, an important goal of explanation is to acquire or maintain the users’ trust. In this paper, I investigate the relation between explanation and trust in the context of computing science. This analysis draws on literature study and concept analysis, using elements from system theory as well as actor-network theory. I apply the conceptual framework to both AI and information security, and show the benefit of the framework for both fields by means of examples. The main focus is on expert systems (AI) and electronic voting systems (security). Finally, I discuss consequences of the analysis for ethics in terms of (un)informed consent and dissent, and the associated division of responsibilities.

AB - There is a common problem in artificial intelligence (AI) and information security. In AI, an expert system needs to be able to justify and explain a decision to the user. In information security, experts need to be able to explain to the public why a system is secure. In both cases, an important goal of explanation is to acquire or maintain the users’ trust. In this paper, I investigate the relation between explanation and trust in the context of computing science. This analysis draws on literature study and concept analysis, using elements from system theory as well as actor-network theory. I apply the conceptual framework to both AI and information security, and show the benefit of the framework for both fields by means of examples. The main focus is on expert systems (AI) and electronic voting systems (security). Finally, I discuss consequences of the analysis for ethics in terms of (un)informed consent and dissent, and the associated division of responsibilities.

KW - EWI-18815

KW - SCS-Cybersecurity

KW - Trust

KW - explanation

KW - Systems theory

KW - Information Security

KW - confidence

KW - METIS-271135

KW - Actor-Network Theory

KW - Expert Systems

KW - IR-74620

KW - informed consent

U2 - 10.1007/s10676-010-9253-3

DO - 10.1007/s10676-010-9253-3

M3 - Article

VL - online pre

SP - 53

EP - 64

JO - Ethics and information technology

T2 - Ethics and information technology

JF - Ethics and information technology

SN - 1388-1957

IS - 1

ER -