IPAB Workshop-11/04/19

Explainable artificial intelligence for autonomous and tele-operated robotic systems.

In this talk, I will present our work on the ORCA Hub project for the explanation of autonomous and tele-operated surveillance vehicles in an offshore drilling rig environment. In this setup, an explainable artificial intelligence (XAI) system learns to predict (from observational traces) local behaviour, medium-term goals and high-level commands that together give an explanation of the actual state of the system within a high-level plan. We present two approaches for behaviour prediction and explanation based on different information inputs. In the first setup, the XAI system learns a forward model of future behaviour and actual goals of the robot based on labelled data from global positioning and goal directions. In the second approach, the XAI system has limited access to the state of the agent, represented only by top-down images. The system is able to learn, in an unsupervised fashion the state representation, goals and PID control commands. We use deep convolutional networks and input sensitivity analysis to efficiently sample state predictions and goals with Monte Carlo approaches. 

As motivation for these approaches, I will also present a conceptual organisation of different techniques for XAI. Depending on the life cycle of the system, these techniques satisfy different requirements. For example, techniques used in the learning phase of the system might not be useful in a fully-deployed system. Similarly, the result of the studied behaviour is a factor that affects the XAI, e.g. an acceptable behaviour can be interpreted with a teleological explanation, while a failed/non-acceptable behaviour needs deeper introspection. XAI is meant to give understanding to outside reviewers, so the understanding and role of the human reviewers are factors to take into consideration for a successful XAI system where different levels of explanation satisfy different reviewers.