NeurIPS 2020 | An Explanation to What is Counterfactuals in Interpretable AI? (Tutorial)

325 views2758 WordsGrade 17 Readability5.0/5 RatingDownload TxT File
AI Pursuit x FinDL

Join the channel membership: https://www.youtube.com/c/AIPursuit/join Subscribe to the channel: https://www.youtube.com/c/AIPursuit?sub_confirmation=1 Support and Donation: Paypal ⇢ https://paypal.me/tayhengee Patreon ⇢ https://www.patreon.com/hengee BTC ⇢ bc1q2r7eymlf20576alvcmryn28tgrvxqw5r30cmpu ETH ⇢ 0x58c4bD4244686F3b4e636EfeBD159258A5513744 Doge ⇢ DSGNbzuS1s6x81ZSbSHHV5uGDxJXePeyKy Wanted to own BTC, ETH, or even Dogecoin? Kickstart your crypto portfolio with the largest crypto market Binance with my affiliate link: https://accounts.binance.com/en/register?ref=27700065 The video is reposted for educational purposes and encourages involvement in the field of research.

... Show More

Video Transcript:

All right now let's talk about local counterfactual explanations as machine learning models are being increasingly deployed to make high-stakes decisions it also becomes important to provide recourses to affected individuals and this is exactly what counterfactual explanations will help us do So counterfactual explanations uh give us the following information they tell us what features need to be changed and by how much to flip a model's prediction that is to reverse an unfavorable outcome to understand this better let us consider a concrete application scenario where an applicant sends his loan application to a bank and the bank might be using a predictive Model in this case f of x to determine which loan applications should be approved and which ones should not in this particular case let us consider the scenario where this particular applicant's loan application is rejected by this predictive model now instead of just communicating this rejection decision as it is without any other Input it can be extremely useful to the applicant if we provide more information about what he can change in order to increase the chance of his application getting accepted the next time he applies for a low so what we design or what we think of is to basically have these counter factual generation algorithms which communicate with the predictive models And provide recourses to the end users for example in this case a recourse might look like increase your salary by 50k and pay your credit card bills on time for the next three months to increase your chances of getting your loan application accepted okay all right now let's talk a little bit about the intuitions behind how to generate counterfactual Explanations what you see here is basically a simple diagram showing a decision boundary between the positive and the negative classes okay we have an instance x for which we need to find the counter factual and the intuitive way in which several algorithms try to do this is by basically making small Modifications to x until x crosses the decision boundary or this new version of x crosses the decision boundary and its prediction flips okay such instance is called as the counter factual of x so in this case you see two points c f one and c f two which are both counter factuals of x And c of one is obtained by moving x in the direction a and c of two is obtained by moving x in the direction b and as you notice both c f one and c f two are on the positive side of the decision boundary so both of them are actually valid counter factuals of x okay a lot of the different algorithms designed to generate counter factual explanations Pretty much differ on two key questions one is how to choose among the various possible candidate counterfactuals and the second is how much access is needed to the underlying predictive model when we are trying to determine what the counter factual would look like okay all right now let's go and look at some of the most important algorithms which were designed to generate Counter factual explanations the first algorithm for this problem was proposed by voctor at all and they use the simple objective function where the goal is to find a counter factual x prime for a given instance x in such a way that f of x prime that is the predictive model's prediction on x prime is actually the desired outcome And the distance between x and x prime is as small as it can get okay so that was their high level idea of what the objective function would look like okay and as you can see here the distance metric is a key component of this objective and the choice of the distance metric dictates what kinds of counterfactuals will actually end up being chosen and dr at all employed a normalized Manhattan distance as their distance metric all right so in fact wachter at all used a slightly different version of this objective that we just discussed uh which is shown on the right hand side of the slide uh this version of the objective is actually unconstrained as you can see and is also differentiable and they use adam optimization algorithm With random restarts to solve this objective so one of the key points to note here is that voctor what all's algorithm requires access to gradients of the underlying predictive model so it is not dealing with a complete black box all right so these are some of the counter factuals output by dr adolf's algorithm As you can see in case of person one the counterfactual says that if the lsat score was 34 uh then the a person would have obtained a desired score right and then other things to pay attention to here involve persons three and four what you actually see is that the counterfactual is asking these people to change their race in order to obtain A desired outcome and as we all know it is impossible to act upon these features now how to fix this problem and how to generate counterfactuals which are actually more feasible that is our next part of the discussion in fact austin at all exactly deal with this problem they rethink the objective function that we saw earlier a bit more and instead of thinking about minimizing The distance between x and x prime uh they take a different approach they basically introduce a two new pieces in the objective function one is instead of the distance between x and x prime they actually consider the cost that one needs to incur to change from x to x prime okay and they try to minimize that particular cost and also If you notice there is another variable introduced which is a a is basically the set of feasible counterfactuals input by the end user for example end users can explicitly say that changes to features like race and gender are not feasible therefore they cannot be valid counterfactuals okay and the cost function Uh that is chosen by us to netal's work is actually total log percentile shift and they use this cost function to capture the fact that changes become harder when starting off from higher percentile value that it is that is it is harder to move from 90th percentile to 95th percentile than it is to move from 50th percentile to 55th percentile So this logic is captured in uh the log percentile shift which they use as their cost function okay austin at all only consider the case where the underlying model is a linear classifier when solving this problem okay and that as we all know can be somewhat limiting furthermore their approach actually requires complete access to the linear classifier Which means they need to know the weight vector of the linear classifier so their approach is not designed to work with black box models now one might ask a question what if we have a black box or a non-linear classifier as our underlying model in that case there may be a workaround first we could generate a local linear model approximation for example using an approach like Lime and then apply ustonatol's framework on top of it to come up with these counterfactuals here is an example of a counterfactual produced for a particular loan applicant using the approach of wisdom at all as you can see there are different kinds of changes being recommended to reduce the number of credit cards from five to three and also to reduce the current debt from Three thousand to fifty dollars to thousand dollars one thing i would like to point out here is that in this particular context when we think of the current depth changing it could also potentially affect other features and in fact in a lot of real world settings changing one feature without affecting another is almost Impossible so how would this really affect counterfactuals or would it affect at all to understand let us again consider the scenario and study this more deeply okay now we gave this loan applicant a recourse which asked him to reduce his current debt from three thousand to fifty dollars two thousand dollars let's say this applicant comes back after an ear And says hey my current debt has gone down to thousand dollars now please give me a loan but then the predictive model might say that well that is true but your age has also increased by one year and the recourse is therefore no longer valid sorry right so these kinds of scenarios can definitely occur in practice so it is extremely important to account for feature interactions when generating Counterfactuals but how do we actually do it that will be the topic that we discuss now mahajan at all and kareemi at all proposed some solutions to tackle this problem for example mahajan at all proposed a new variant of the objective function where instead of the standard distance metric d that we were looking at or using so far they actually consider a new distance Metric denoted by d underscore console which leverages the structural causal model in the entire process of generating counterfactuals to try and understand this metric a little better and to develop an intuition for it let us consider this example where we have a data set with three features feature one two and three And what you see on the screen is a causal graph of that data set what we see here is that feature three has two parents feature one and feature two and neither feature one nor feature two actually have any parents in the castle graph so their approach divides all features into two groups a group u which is basically the set of Those features or notes with no parents in the causal graph and a set v which is basically the set of those features or notes with at least one parent in the castle graph and using this definition they define this new distance metric where for each feature u for which there is no parent in the causal graph they essentially use the same old standard l1 or l2 Distance between the corresponding feature values of the original instance and the counter factual that is they compute the distance between x u and x u prime on the other hand for those features v uh which basically have at least one parent in the causal graph uh they compute the distance between x v and the expected value of x prime v given the values of the parents of this particular feature V okay in doing so they are essentially incorporating the causal structure of the graph into the process of counter factual generation which means they're also incorporating the interactions between the features or considering them when generating counterfactuals so that the problems like the ones we just saw do not arise right okay So their approach of course requires knowledge of the full crossover craft which can often be impossible to obtain in practice but empirically they show that partial knowledge of the causal craft also seems to work fine in addition they propose a workaround where end users can provide inputs about feasibility constraints and the partial causal graph And in order to solve the proposed objective they leverage a variational auto encoder and one thing to note here is that their approach requires access to gradients of the underlying predictive model now other notions of feasible counterfactuals have also been proposed in literature for example one notion is based on the idea of data manifold closeness Where the goal is to generate a counter factual which is actually close to the original data distribution uh this basically uses the intuition that counter factuals which are actually very far off from the original data distribution may not even be valid data points and another take on what a feasible counterfactual should look like enforces the notion of sparsity by Ensuring that we only change a small number of features in the counter factual because requiring a user to act upon a large number of features can often be extremely hard in practice all right now we are going to talk about the next type of explanations that is global explanations global explanations are designed with the goal of Explaining the complete behavior of a given model which could potentially be a black box so these kinds of explanations provide a bird's eye view of mortal behavior and these explanations help detect big picture model biases which are persistent across larger subgroups of population and which are often harder to detect by examining local explanations of Several instances in that sense global explanations are actually complementary to local explanations just to rejog your memory while local explanations try to explain the individual predictions of a given model global explanations try to focus on explaining the complete behavior of the model and why local explanations help unearth biases in the local vicinity of a given Instance global explanations help shed light on big picture biases which affect larger subgroups in the population and while local explanations help us wet if individual predictions are being made for the right reasons global explanations help us with if a model at a high level is suitable for deployment now let's jump into some of the techniques which are proposed in literature to Construct global explanations let's start with the first approach where a global explanation is seen as a collection of local explanations all right the idea here is to basically first generate a local explanation for every instance in the data using one of the approaches that we discussed earlier in this talk and once we have that we then pick a subset of k local explanations to constitute the Global explanation and the choices that we really need to make here are what local explanation technique to use and how to choose the subset of k local explanations now let's see one of the first algorithms which actually uses this kind of idea or intuition the algorithm is sp line and it tries to construct global explanations using Local feature importances by now we are all aware that line explains a single prediction or local behavior of a model for a single instance right so in this case this black dot corresponds to a single explanation and it is quite impossible and extremely hard to examine all possible explanations to understand the global behavior of the model so instead sp line advocates to pick k Explanations to show to the user and how can we pick these k explanations these k explanations are picked in such a way that their representative that is these explanations summarize the models global behavior and they're also not redundant and diverse so let me explain a bit when i say what i mean by uh not redundant or diverse now let's Consider the three dots that you see on the slide here the red dot black dot and blue dot each of these correspond to an explanation let's say that we have already picked the explanation corresponding to the black dot in our into our subset of k local explanations once the black dot is picked as one of the explanations adding the red dot explanation to this set of k Local explanations is not going to add much value in fact it would be redundant because both the explanations corresponding to black dot and red dot are describing the same decision boundary roughly on the other hand if we choose the explanation corresponding to the blue dot that is not redundant and that provides diversity because it's capturing a Different decision boundary now sp line formulates this intuition as a submodular optimization problem and greedily picks a subset of k local explanations and as we can see this procedure is model agnostic because it does not require access to the internal details of the underlying predictive model now if we repeat the same procedure but replace line using anchors algorithm we will obtain a global explanation Which is a subset of k local rule sets because anker's algorithm actually outputs local rule sets and even this algorithm that is called as sp anchor is going to be model agnostic

Like it? Make YTScribe even better by leaving a review