What happens when decision support systems fail? — the importance of usability on performance in erroneous systems

Abstract

With the advent of artificial intelligence (AI) methods, smart decision support systems (DSSs) are becoming ubiquitous. Such systems help reduce complexity for operators by automating data integration tasks and recommending actions. However, these systems are sometimes flawed. It is not sufficiently understood whether, when and why operators comply with such systems in erroneous or correct cases. We empirically investigate compliance with correct and defective DSSs, the influence of correct and erroneous DSS’s on performance and subjective factors related to compliance. In the study, a business game was used as an experimental setting in which 40 users took part. The impact of system correctness on user acceptance, trust, compliance and overall performance was investigated. The results show that the defective system reduces trust in automation (−47%), reduces usefulness (−58%), reduces acceptance (−62%) and reduces overall performance (−32%). Overall, the defective system was less user-friendly (−27%). Nevertheless, users who rated the system’s usability higher, outperformed users who rated it lower. Usability is therefore an intermediary that compensates for the negative influence of erroneous DSSs.