Toward Explainable Artificial Intelligence for Regression Models: A methodological perspective

In addition to the impressive predictive power of machine learning (ML) models, more recently, explanation methods have emerged that enable an interpretation of complex nonlinear learning models, such as deep neural networks. Gaining a better understanding is especially important, e.g., for safety-critical ML applications or medical diagnostics and so on. Although such explainable artificial intelligence (XAI) techniques have reached significant popularity for classifiers, thus far, little attention has been devoted to XAI for regression models (XAIR). In this review, we clarify the fundamental conceptual differences of XAI for regression and classification tasks, establish novel theoretical insights and analysis for XAIR, provide demonstrations of XAIR on genuine practical regression problems, and finally, discuss challenges remaining for the field.

Introduction. 

ML, in particular deep learning, has supplied a vast number of scientific and industrial applications with powerful predictive models. As ML models are being increasingly considered for high-stakes autonomous decisions, there has been a growing need for gaining trust in the model without giving up predictive power. XAI has developed as a response to the need of validating these highly powerful models [9], [64], [65]. Taking ML and XAI together, these technologies also offer a way of gaining new insights, e.g., nonlinear relations, into the complex data-generating processes under study.

Up to now, the main body of work in the field of XAI has revolved around explaining decisions of classification models, often in the context of image recognition tasks [10], [11], [56], [90], [92]. Regression, a major workhorse in ML and signal processing, has essentially received only little attention. In practice to date, XAI approaches designed for classification are applied for regression problems. Although such a naive application to regression can occasionally still yield useful results, in this article, we show that appropriate, theoretically well-founded explanation models are necessary and overdue. For example, when explaining classification models, we can rely on the implicit knowledge associated with the class and assume a decision boundary between the two or more classes. Additionally, the output itself can conveniently serve as a measure of model uncertainty, or even evidence against the respective class can be analyzed. In regression, on the other hand, we find none of these beneficial properties while we aim to explain a highly aggregated and application-specific model output that often corresponds to a physical entity with an attached unit.

In this article, we outline multiple challenges that emerge when explaining regression models, and we show how popular methods such as layerwise relevance propagation (LRP) [10], integrated gradients (IGs) [79], or Shapley value [48], [71], [78] can be applied or extended in a theoretically well-founded manner to properly address them. Our efforts are guided by how to formulate the question for which we would like an explanation in a way that addresses the user’s interpretation needs. In particular, we aim for an explanation that inherits the unit of measurement of the prediction (e.g., physical or monetary unit). The explanation should also be sufficiently contextualized, not only by being specific to each data point but also by localizing the explanation around a relevant range of predicted outputs. As an example, LRP and many other explanation methods assume (sometimes implicitly) a zero reference value relative to which they explain or expand [64]. Although in the classification setting one naturally explains relative to the decision boundary, i.e., f(x)=0, in regression settings, we consider the reference value to be a crucial parameter to integrate the desired context into the explanation and avoid the latter to be dominated by (uninteresting) coarse effects. Therefore, we generally discourage practitioners to apply XAI methods developed for classification problems in an out-of-the-box manner to regression models.