[
  {
    "objectID": "index.html",
    "href": "index.html",
    "title": "Conformal Counterfactual Explanations",
    "section": "",
    "text": "Preface"
  },
  {
    "objectID": "notebooks/proposal.html#motivation",
    "href": "notebooks/proposal.html#motivation",
    "title": "1  High-Fidelity Counterfactual Explanations through Conformal Prediction",
    "section": "1.1 Motivation",
    "text": "1.1 Motivation\nCounterfactual Explanations are a powerful, flexible and intuitive way to not only explain black-box models but also enable affected individuals to challenge them through the means of Algorithmic Recourse.\n\n1.1.1 Counterfactual Explanations or Adversarial Examples?\nMost state-of-the-art approaches to generating Counterfactual Explanations (CE) rely on gradient descent in the feature space. The key idea is to perturb inputs \\(x\\in\\mathcal{X}\\) into a black-box model \\(f: \\mathcal{X} \\mapsto \\mathcal{Y}\\) in order to change the model output \\(f(x)\\) to some pre-specified target value \\(t\\in\\mathcal{Y}\\). Formally, this boils down to defining some loss function \\(\\ell(f(x),t)\\) and taking gradient steps in the minimizing direction. The so-generated counterfactuals are considered valid as soon as the predicted label matches the target label. A stripped-down counterfactual explanation is therefore little different from an adversarial example. In Figure 1.1, for example, generic counterfactual search as in Wachter, Mittelstadt, and Russell (2017) has been applied to MNIST data.\n\n\n\nFigure 1.1: You may not like it, but this is what stripped-down counterfactuals look like. Here we have used Wachter, Mittelstadt, and Russell (2017) to generate multiple counterfactuals for turning an 8 (eight) into a 3 (three).\n\n\nThe crucial difference between adversarial examples and counterfactuals is one of intent. While adversarial examples are typically intended to go unnoticed, counterfactuals in the context of Explainable AI are generally sought to be “plausible”, “realistic” or “feasible”. To fulfil this latter goal, researchers have come up with a myriad of ways. Joshi et al. (2019) were among the first to suggest that instead of searching counterfactuals in the feature space, we can instead traverse a latent embedding learned by a surrogate generative model. Similarly, Poyiadzi et al. (2020) use density … Finally, Karimi, Schölkopf, and Valera (2021) argues that counterfactuals should comply with the causal model that generates them [CHECK IF WE CAN PHASE THIS LIKE THIS]. Other related approaches include … All of these different approaches have a common goal: they aim to ensure that the generated counterfactuals comply with the (learned) data-generating process (DGB).\n\nDefinition 1.1 (Plausible Counterfactuals) Formally, if \\(x \\sim \\mathcal{X}\\) and for the corresponding counterfactual we have \\(x^{\\prime}\\sim\\mathcal{X}^{\\prime}\\), then for \\(x^{\\prime}\\) to be considered a plausible counterfactual, we need: \\(\\mathcal{X} \\approxeq \\mathcal{X}^{\\prime}\\).\n\nIn the context of Algorithmic Recourse, it makes sense to strive for plausible counterfactuals, since anything else would essentially require individuals to move to out-of-distribution states. But it is worth noting that our ambition to meet this goal, may have implications on our ability to faithfully explain the behaviour of the underlying black-box model (arguably our principal goal). By essentially decoupling the task of learning plausible representations of the data from the model itself, we open ourselves up to vulnerabilities. Using a separate generative model to learn \\(\\mathcal{X}\\), for example, has very serious implications for the generated counterfactuals. Figure 1.2 compares the results of applying REVISE (Joshi et al. 2019) to MNIST data using two different Variational Auto-Encoders: while the counterfactual generated using an expressive (strong) VAE is compelling, the result relying on a less expressive (weak) VAE is not even valid. In this latter case, the decoder step of the VAE fails to yield values in \\(\\mathcal{X}\\) and hence the counterfactual search in the learned latent space is doomed.\n\n\n\nFigure 1.2: Counterfactual explanations for MNIST using a Latent Space generator: turning a nine (9) into a four (4).\n\n\n\nHere it would be nice to have another example where we poison the data going into the generative model to hide biases present in the data (e.g. Boston housing).\n\n\nLatent can be manipulated:\n\ntrain biased model\ntrain VAE with biased variable removed/attacked (use Boston housing dataset)\nhypothesis: will generate bias-free explanations\n\n\n\n\n1.1.2 From Plausible to High-Fidelity Counterfactuals\nIn light of the findings, we propose to generally avoid using surrogate models to learn \\(\\mathcal{X}\\) in the context of Counterfactual Explanations.\n\nProposition 1.1 (Avoid Surrogates) Since we are in the business of explaining a black-box model, the task of learning realistic representations of the data should not be reallocated from the model itself to some surrogate model.\n\nIn cases where the use of surrogate models cannot be avoided, we propose to weigh the plausibility of counterfactuals against their fidelity to the black-box model. In the context of Explainable AI, fidelity is defined as describing how an explanation approximates the prediction of the black-box model (Molnar 2020). Fidelity has become the default metric for evaluating Local Model-Agnostic Models, since they often involve local surrogate models whose predictions need not always match those of the black-box model.\nIn the case of Counterfactual Explanations, the concept of fidelity has so far been ignored. This is not altogether surprising, since by construction and design, Counterfactual Explanations work with the predictions of the black-box model directly: as stated above, a counterfactual \\(x^{\\prime}\\) is considered valid if and only if \\(f(x^{\\prime})=t\\), where \\(t\\) denote some target outcome.\nDoes fidelity even make sense in the context of CE, and if so, how can we define it? In light of the examples in the previous section, we think it is urgent to introduce a notion of fidelity in this context, that relates to the distributional properties of the generated counterfactuals. In particular, we propose that a high-fidelity counterfactual \\(x^{\\prime}\\) complies with the class-conditional distribution \\(\\mathcal{X}_{\\theta} = p_{\\theta}(X|y)\\) where \\(\\theta\\) denote the black-box model parameters.\n\nDefinition 1.2 (High-Fidelity Counterfactuals) Let \\(\\mathcal{X}_{\\theta}|y = p_{\\theta}(X|y)\\) denote the class-conditional distribution of \\(X\\) defined by \\(\\theta\\). Then for \\(x^{\\prime}\\) to be considered a high-fidelity counterfactual, we need: \\(\\mathcal{X}_{\\theta}|t \\approxeq \\mathcal{X}^{\\prime}\\) where \\(t\\) denotes the target outcome.\n\nIn order to assess the fidelity of counterfactuals, we propose the following two-step procedure:\n\nGenerate samples \\(X_{\\theta}|y\\) and \\(X^{\\prime}\\) from \\(\\mathcal{X}_{\\theta}|t\\) and \\(\\mathcal{X}^{\\prime}\\), respectively.\nCompute the Maximum Mean Discrepancy (MMD) between \\(X_{\\theta}|y\\) and \\(X^{\\prime}\\).\n\nIf the computed value is different from zero, we can reject the null-hypothesis of fidelity.\n\nTwo challenges here: 1) implementing the sampling procedure in Grathwohl et al. (2020); 2) it is unclear if MMD is really the right way to measure this."
  },
  {
    "objectID": "notebooks/proposal.html#conformal-counterfactual-explanations",
    "href": "notebooks/proposal.html#conformal-counterfactual-explanations",
    "title": "1  High-Fidelity Counterfactual Explanations through Conformal Prediction",
    "section": "1.2 Conformal Counterfactual Explanations",
    "text": "1.2 Conformal Counterfactual Explanations\nIn Section 1.1.2, we have advocated for avoiding surrogate models in the context of Counterfactual Explanations. In this section, we introduce an alternative way to generate high-fidelity Counterfactual Explanations. In particular, we propose Conformal Counterfactual Explanations (CCE), that is Counterfactual Explanations that minimize the predictive uncertainty of conformal models.\n\n1.2.1 Minimizing Predictive Uncertainty\nSchut et al. (2021) demonstrated that the goal of generating realistic (plausible) counterfactuals can also be achieved by seeking counterfactuals that minimize the predictive uncertainty of the underlying black-box model. Similarly, Antorán et al. (2020) …\n\nProblem: restricted to Bayesian models.\nSolution: post-hoc predictive uncertainty quantification. In particular, Conformal Prediction.\n\n\n\n1.2.2 Background on Conformal Prediction\n\nDistribution-free, model-agnostic and scalable approach to predictive uncertainty quantification.\nConformal prediction is instance-based. So is CE.\nTake any fitted model and turn it into a conformal model using calibration data.\nOur approach, therefore, relaxes the restriction on the family of black-box models, at the cost of relying on a subset of the data. Arguably, data is often abundant and in most applications practitioners tend to hold out a test data set anyway.\n\n\nDoes the coverage guarantee carry over to counterfactuals?\n\n\n\n1.2.3 Generating Conformal Counterfactuals\nWhile Conformal Prediction has recently grown in popularity, it does introduce a challenge in the context of classification: the predictions of Conformal Classifiers are set-valued and therefore difficult to work with, since they are, for example, non-differentiable. Fortunately, Stutz et al. (2022) introduced carefully designed differentiable loss functions that make it possible to evaluate the performance of conformal predictions in training. We can leverage these recent advances in the context of gradient-based counterfactual search …\n\nChallenge: still need to implement these loss functions."
  },
  {
    "objectID": "notebooks/proposal.html#experiments",
    "href": "notebooks/proposal.html#experiments",
    "title": "1  High-Fidelity Counterfactual Explanations through Conformal Prediction",
    "section": "1.3 Experiments",
    "text": "1.3 Experiments\n\n1.3.1 Research Questions\n\nIs CP alone enough to ensure realistic counterfactuals?\nDo counterfactuals improve further as the models get better?\nDo counterfactuals get more realistic as coverage\nWhat happens as we vary coverage and setsize?\nWhat happens as we improve the model robustness?\nWhat happens as we improve the model’s ability to incorporate predictive uncertainty (deep ensemble, laplace)?\nWhat happens if we combine with DiCE, ClaPROAR, Gravitational?\nWhat about CE robustness to endogenous shifts (Altmeyer et al. 2023)?\nBenchmarking:\n\nadd PROBE (Pawelczyk et al. 2022) into the mix.\ncompare travel costs to domain shits.\n\n\n\nNice to have: What about using Laplace Approximation, then Conformal Prediction? What about using Conformalised Laplace?"
  },
  {
    "objectID": "notebooks/proposal.html#references",
    "href": "notebooks/proposal.html#references",
    "title": "1  High-Fidelity Counterfactual Explanations through Conformal Prediction",
    "section": "1.4 References",
    "text": "1.4 References\n\n\n\n\nAltmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek, Arie van Deursen, and Cynthia Liem. 2023. “Endogenous Macrodynamics in Algorithmic Recourse.” In First IEEE Conference on Secure and Trustworthy Machine Learning.\n\n\nAntorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José Miguel Hernández-Lobato. 2020. “Getting a Clue: A Method for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.\n\n\nGrathwohl, Will, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. 2020. “Your Classifier Is Secretly an Energy Based Model and You Should Treat It Like One.” In. https://openreview.net/forum?id=Hkxzx0NtDB.\n\n\nJoshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. “Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.\n\n\nKarimi, Amir-Hossein, Bernhard Schölkopf, and Isabel Valera. 2021. “Algorithmic Recourse: From Counterfactual Explanations to Interventions.” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 353–62.\n\n\nMolnar, Christoph. 2020. Interpretable Machine Learning. Lulu. com.\n\n\nPawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. 2022. “Probabilistically Robust Recourse: Navigating the Trade-Offs Between Costs and Robustness in Algorithmic Recourse.” arXiv Preprint arXiv:2203.06768.\n\n\nPoyiadzi, Rafael, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and Peter Flach. 2020. “FACE: Feasible and Actionable Counterfactual Explanations.” In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 344–50.\n\n\nSchut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.\n\n\nStutz, David, Krishnamurthy Dj Dvijotham, Ali Taylan Cemgil, and Arnaud Doucet. 2022. “Learning Optimal Conformal Classifiers.” In. https://openreview.net/forum?id=t8O-4LKFVx.\n\n\nWachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harv. JL & Tech. 31: 841."
  },
  {
    "objectID": "notebooks/intro.html#black-box-model",
    "href": "notebooks/intro.html#black-box-model",
    "title": "2  ConformalGenerator",
    "section": "2.1 Black-box Model",
    "text": "2.1 Black-box Model\nWe consider a simple binary classification problem. Let \\((X_i, Y_i), \\ i=1,...,n\\) denote our feature-label pairs and let \\(\\mu: \\mathcal{X} \\mapsto \\mathcal{Y}\\) denote the mapping from features to labels. For illustration purposes, we will use linearly separable data.\n\ncounterfactual_data = load_linearly_separable()\n\nWhile we could use a linear classifier in this case, let’s pretend we need a black-box model for this task and rely on a small Multi-Layer Perceptron (MLP):\n\nbuilder = MLJFlux.@builder Chain(\n    Dense(n_in, 32, relu),\n    Dense(32, n_out)\n)\nclf = NeuralNetworkClassifier(builder=builder, epochs=100)\n\nWe can fit this model to data to produce plug-in predictions."
  },
  {
    "objectID": "notebooks/intro.html#conformal-prediction",
    "href": "notebooks/intro.html#conformal-prediction",
    "title": "2  ConformalGenerator",
    "section": "2.2 Conformal Prediction",
    "text": "2.2 Conformal Prediction\nHere we will instead use a specific case of CP called split conformal prediction which can then be summarized as follows:1\n\nPartition the training into a proper training set and a separate calibration set: \\(\\mathcal{D}_n=\\mathcal{D}^{\\text{train}} \\cup \\mathcal{D}^{\\text{cali}}\\).\nTrain the machine learning model on the proper training set: \\(\\hat\\mu_{i \\in \\mathcal{D}^{\\text{train}}}(X_i,Y_i)\\).\n\nThe model \\(\\hat\\mu_{i \\in \\mathcal{D}^{\\text{train}}}\\) can now produce plug-in predictions.\n\n\n\n\n\n\nStarting Point\n\n\n\nNote that this represents the starting point in applications of Algorithmic Recourse: we have some pre-trained classifier \\(M\\) for which we would like to generate plausible Counterfactual Explanations. Next, we turn to the calibration step.\n\n\n\nCompute nonconformity scores, \\(\\mathcal{S}\\), using the calibration data \\(\\mathcal{D}^{\\text{cali}}\\) and the fitted model \\(\\hat\\mu_{i \\in \\mathcal{D}^{\\text{train}}}\\).\nFor a user-specified desired coverage ratio \\((1-\\alpha)\\) compute the corresponding quantile, \\(\\hat{q}\\), of the empirical distribution of nonconformity scores, \\(\\mathcal{S}\\).\nFor the given quantile and test sample \\(X_{\\text{test}}\\), form the corresponding conformal prediction set:\n\n\\[\nC(X_{\\text{test}})=\\{y:s(X_{\\text{test}},y) \\le \\hat{q}\\}\n\\tag{2.1}\\]\nThis is the default procedure used for classification and regression in ConformalPrediction.jl.\nUsing the package, we can apply Split Conformal Prediction as follows:\n\nX = table(permutedims(counterfactual_data.X))\ny =  counterfactual_data.output_encoder.labels\nconf_model = conformal_model(clf; method=:simple_inductive)\nmach = machine(conf_model, X, y)\nfit!(mach)\n\nTo be clear, all of the calibration steps (3 to 5) are post hoc, and yet none of them involved any changes to the model parameters. These are two important characteristics of Split Conformal Prediction (SCP) that make it particularly useful in the context of Algorithmic Recourse. Firstly, the fact that SCP involves posthoc calibration steps that happen after training, ensures that we need not place any restrictions on the black-box model itself. This stands in contrast to the approach proposed by Schut et al. (2021) in which they essentially restrict the class of models to Bayesian models. Secondly, the fact that the model itself is kept entirely intact ensures that the generated counterfactuals maintain fidelity to the model. Finally, note that we also have not resorted to a surrogate model to learn more about \\(X \\sim \\mathcal{X}\\). Instead, we have used the fitted model itself and a calibration data set to learn about the model’s predictive uncertainty."
  },
  {
    "objectID": "notebooks/intro.html#differentiable-cp",
    "href": "notebooks/intro.html#differentiable-cp",
    "title": "2  ConformalGenerator",
    "section": "2.3 Differentiable CP",
    "text": "2.3 Differentiable CP\nIn order to use CP in the context of gradient-based counterfactual search, we need it to be differentiable. Stutz et al. (2022) introduce a framework for training differentiable conformal predictors. They introduce a configurable loss function as well as smooth set size penalty.\n\n2.3.1 Smooth Set Size Penalty\nStarting with the former, Stutz et al. (2022) propose the following:\n\\[\n\\Omega(C_{\\theta}(x;\\tau)) = = \\max (0, \\sum_k C_{\\theta,k}(x;\\tau) - \\kappa)\n\\tag{2.2}\\]\nHere, \\(C_{\\theta,k}(x;\\tau)\\) is loosely defined as the probability that class \\(k\\) is assigned to the conformal prediction set \\(C\\). In the context of Conformal Training, this penalty reduces the inefficiency of the conformal predictor.\nIn our context, we are not interested in improving the model itself, but rather in producing plausible counterfactuals. Provided that our counterfactual \\(x^\\prime\\) is already inside the target domain (\\(\\mathbb{I}_{y^\\prime = t}=1\\)), penalizing \\(\\Omega(C_{\\theta}(x;\\tau))\\) corresponds to guiding counterfactuals into regions of the target domain that are characterized by low ambiguity: for \\(\\kappa=1\\) the conformal prediction set includes only the target label \\(t\\) as \\(\\Omega(C_{\\theta}(x;\\tau))\\). Arguably, less ambiguous counterfactuals are more plausible. Since the search is guided purely by properties of the model itself and (exchangeable) calibration data, counterfactuals also maintain high fidelity.\nThe left panel of Figure 2.1 shows the smooth size penalty in the two-dimensional feature space of our synthetic data.\n\n\n2.3.2 Configurable Classification Loss\nThe right panel of Figure 2.1 shows the configurable classification loss in the two-dimensional feature space of our synthetic data.\n\n\n\n\n\nFigure 2.1: Illustration of the smooth size loss and the configurable classification loss."
  },
  {
    "objectID": "notebooks/intro.html#fidelity-and-plausibility",
    "href": "notebooks/intro.html#fidelity-and-plausibility",
    "title": "2  ConformalGenerator",
    "section": "2.4 Fidelity and Plausibility",
    "text": "2.4 Fidelity and Plausibility\nThe main evaluation criteria we are interested in are fidelity and plausibility. Interestingly, we could also consider using these measures as penalties in the counterfactual search.\n\n2.4.1 Fidelity\nWe propose to define fidelity as follows:\n\nDefinition 2.1 (High-Fidelity Counterfactuals) Let \\(\\mathcal{X}_{\\theta}|y = p_{\\theta}(X|y)\\) denote the class-conditional distribution of \\(X\\) defined by \\(\\theta\\). Then for \\(x^{\\prime}\\) to be considered a high-fidelity counterfactual, we need: \\(\\mathcal{X}_{\\theta}|t \\approxeq \\mathcal{X}^{\\prime}\\) where \\(t\\) denotes the target outcome.\n\nWe can generate samples from \\(p_{\\theta}(X|y)\\) following Grathwohl et al. (2020). In Figure 2.2, I have applied the methodology to our synthetic data.\n\n\nM = CCE.ConformalModel(conf_model, mach.fitresult)\n\nniter = 100\nnsamples = 100\n\nplts = []\nfor (i,target) ∈ enumerate(counterfactual_data.y_levels)\n    sampler = CCE.EnergySampler(M, counterfactual_data, target; niter=niter, nsamples=100)\n    Xgen = rand(sampler, nsamples)\n    plt = plot(M, counterfactual_data; target=target, zoom=-3,cbar=false)\n    scatter!(Xgen[1,:],Xgen[2,:],alpha=0.5,color=i,shape=:star,label=\"X|y=$target\")\n    push!(plts, plt)\nend\nplot(plts..., layout=(1,length(plts)), size=(img_height*length(plts),img_height))\nFigure 2.2: ?(caption)\n\n\nAs an evaluation metric and penalty, we could use the average distance of the counterfactual \\(x^{\\prime}\\) from these generated samples, for example.\n\n\n2.4.2 Plausibility\nWe propose to define plausibility as follows:\n\nDefinition 2.2 (Plausible Counterfactuals) Formally, let \\(\\mathcal{X}|t\\) denote the conditional distribution of samples in the target class. As before, we have \\(x^{\\prime}\\sim\\mathcal{X}^{\\prime}\\), then for \\(x^{\\prime}\\) to be considered a plausible counterfactual, we need: \\(\\mathcal{X}|t \\approxeq \\mathcal{X}^{\\prime}\\).\n\nAs an evaluation metric and penalty, we could use the average distance of the counterfactual \\(x^{\\prime}\\) from (potentially bootstrapped) training samples in the target class, for example."
  },
  {
    "objectID": "notebooks/intro.html#counterfactual-explanations",
    "href": "notebooks/intro.html#counterfactual-explanations",
    "title": "2  ConformalGenerator",
    "section": "2.5 Counterfactual Explanations",
    "text": "2.5 Counterfactual Explanations\nNext, let’s generate counterfactual explanations for our synthetic data. We first wrap our model in a container that makes it compatible with CounterfactualExplanations.jl. Then we draw a random sample, determine its predicted label \\(\\hat{y}\\) and choose the opposite label as our target.\n\nx = select_factual(counterfactual_data,rand(1:size(counterfactual_data.X,2)))\ny_factual = predict_label(M, counterfactual_data, x)[1]\ntarget = counterfactual_data.y_levels[counterfactual_data.y_levels .!= y_factual][1]\n\nThe generic Conformal Counterfactual Generator penalises the only the set size only:\n\\[\nx^\\prime = \\arg \\min_{x^\\prime}  \\ell(M(x^\\prime),t) + \\lambda \\mathbb{I}_{y^\\prime = t} \\Omega(C_{\\theta}(x;\\tau))\n\\tag{2.3}\\]\n\n\n\n\n\nFigure 2.3: Comparison of counterfactuals produced using different generators.\n\n\n\n\n\n2.5.1 Benchmark\n\n# Data:\ndatasets = Dict(\n    :linearly_separable => load_linearly_separable(),\n    :overlapping => load_overlapping(),\n    :moons => load_moons(),\n    :circles => load_circles(),\n)\n\n# Models:\nmodels = Dict(\n    :MLP => M,\n)\n\nThen we can simply loop over the datasets and eventually concatenate the results like so:\n\nusing CounterfactualExplanations.Evaluation: benchmark\nbmks = []\nfor (dataname, dataset) in datasets\n    bmk = benchmark(dataset; models=models, generators=generators, suppress_training=true, dataname=dataname)\n    push!(bmks, bmk)\nend\nbmk = reduce(vcat, bmks)"
  },
  {
    "objectID": "notebooks/intro.html#multi-class",
    "href": "notebooks/intro.html#multi-class",
    "title": "2  ConformalGenerator",
    "section": "2.6 Multi-Class",
    "text": "2.6 Multi-Class\n\ncounterfactual_data = load_multi_class()\n\n\nX = table(permutedims(counterfactual_data.X))\ny =  counterfactual_data.output_encoder.labels\nconf_model = conformal_model(clf; method=:simple_inductive, coverage=0.99)\nmach = machine(conf_model, X, y)\nfit!(mach)\n\n\n\n\n\n\nFigure 2.4: Illustration of the smooth size loss and the configurable classification loss.\n\n\n\n\n\n\n\n\n\nFigure 2.5: Energy-based conditional samples.\n\n\n\n\n\nx = select_factual(counterfactual_data,rand(1:size(counterfactual_data.X,2)))\ny_factual = predict_label(M, counterfactual_data, x)[1]\ntarget = counterfactual_data.y_levels[counterfactual_data.y_levels .!= y_factual][1]\n\n\n\n\n\n\nFigure 2.6: Comparison of counterfactuals produced using different generators.\n\n\n\n\n\n\n\n\nGrathwohl, Will, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. 2020. “Your Classifier Is Secretly an Energy Based Model and You Should Treat It Like One.” In. https://openreview.net/forum?id=Hkxzx0NtDB.\n\n\nSchut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan Sacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable Counterfactual Explanations By Implicit Minimisation of Epistemic and Aleatoric Uncertainties.” In International Conference on Artificial Intelligence and Statistics, 1756–64. PMLR.\n\n\nStutz, David, Krishnamurthy Dj Dvijotham, Ali Taylan Cemgil, and Arnaud Doucet. 2022. “Learning Optimal Conformal Classifiers.” In. https://openreview.net/forum?id=t8O-4LKFVx."
  },
  {
    "objectID": "notebooks/intro.html#footnotes",
    "href": "notebooks/intro.html#footnotes",
    "title": "2  ConformalGenerator",
    "section": "",
    "text": "In other places split conformal prediction is sometimes referred to as inductive conformal prediction.↩︎"
  },
  {
    "objectID": "notebooks/references.html",
    "href": "notebooks/references.html",
    "title": "References",
    "section": "",
    "text": "Altmeyer, Patrick, Giovan Angela, Aleksander Buszydlik, Karol Dobiczek,\nArie van Deursen, and Cynthia Liem. 2023. “Endogenous\nMacrodynamics in Algorithmic\nRecourse.” In First IEEE\nConference on Secure and\nTrustworthy Machine\nLearning.\n\n\nAntorán, Javier, Umang Bhatt, Tameem Adel, Adrian Weller, and José\nMiguel Hernández-Lobato. 2020. “Getting a Clue: A\nMethod for Explaining Uncertainty Estimates.” https://arxiv.org/abs/2006.06848.\n\n\nGrathwohl, Will, Kuan-Chieh Wang, Joern-Henrik Jacobsen, David Duvenaud,\nMohammad Norouzi, and Kevin Swersky. 2020. “Your Classifier Is\nSecretly an Energy Based Model and You Should Treat It Like One.”\nIn. https://openreview.net/forum?id=Hkxzx0NtDB.\n\n\nJoshi, Shalmali, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and\nJoydeep Ghosh. 2019. “Towards Realistic Individual Recourse and\nActionable Explanations in Black-Box Decision Making Systems.” https://arxiv.org/abs/1907.09615.\n\n\nKarimi, Amir-Hossein, Bernhard Schölkopf, and Isabel Valera. 2021.\n“Algorithmic Recourse: From Counterfactual Explanations to\nInterventions.” In Proceedings of the 2021 ACM\nConference on Fairness, Accountability,\nand Transparency, 353–62.\n\n\nMolnar, Christoph. 2020. Interpretable Machine Learning.\nLulu. com.\n\n\nPawelczyk, Martin, Teresa Datta, Johannes van-den-Heuvel, Gjergji\nKasneci, and Himabindu Lakkaraju. 2022. “Probabilistically\nRobust Recourse: Navigating the\nTrade-Offs Between Costs and\nRobustness in Algorithmic\nRecourse.” arXiv Preprint arXiv:2203.06768.\n\n\nPoyiadzi, Rafael, Kacper Sokol, Raul Santos-Rodriguez, Tijl De Bie, and\nPeter Flach. 2020. “FACE: Feasible and\nActionable Counterfactual Explanations.” In Proceedings of\nthe AAAI/ACM Conference on AI,\nEthics, and Society, 344–50.\n\n\nSchut, Lisa, Oscar Key, Rory Mc Grath, Luca Costabello, Bogdan\nSacaleanu, Yarin Gal, et al. 2021. “Generating Interpretable\nCounterfactual Explanations By Implicit Minimisation of\nEpistemic and Aleatoric Uncertainties.”\nIn International Conference on Artificial\nIntelligence and Statistics, 1756–64.\nPMLR.\n\n\nStutz, David, Krishnamurthy Dj Dvijotham, Ali Taylan Cemgil, and Arnaud\nDoucet. 2022. “Learning Optimal\nConformal Classifiers.” In. https://openreview.net/forum?id=t8O-4LKFVx.\n\n\nWachter, Sandra, Brent Mittelstadt, and Chris Russell. 2017.\n“Counterfactual Explanations Without Opening the Black Box:\nAutomated Decisions and the GDPR.”\nHarv. JL & Tech. 31: 841."
  }
]