Abstract

Artificial intelligence (AI) has shown its promise in assisting human decision-making. However, humans’ inappropriate decision to accept or reject suggestions from AI can lead to severe consequences in high-stakes AI-assisted decision-making scenarios. This problem persists due to insufficient understanding of human trust in AI. Therefore, this research studies how two types of human confidence that affect trust, their confidence in AI and confidence in themselves, evolve and affect humans’ decisions. A cognitive study and a quantitative model together examine how changing positive and negative experiences affect these confidences and ultimate decisions. Results show that human self-confidence, not their confidence in AI, directs the decision to accept or reject AI suggestions. Furthermore, this work finds that humans often misattribute blame to themselves and enter a vicious cycle of relying on a poorly performing AI. Findings reveal the need and provide insights to effectively calibrate human self-confidence for successful AI-assisted decision-making.