plot reviewer 1

dev/rebuttal/aaai/draft.md

@@ -3,9 +3,25 @@
 ## Weaknesses:
 
 1. Experiment results: need more comprehensive linguistic explanation of results
+
+Following the suggestion by the reviewer, we plan to add a the following linguistic explanation in a prominent place of Section 6: 
+
+"Overall, our findings demonstrate that \textit{ECCCo} produces plausible counterfactuals if and only if the black-box model itself has learned plausible explanations for the data. Thus, \textit{ECCCo} avoids the risk of generating plausible but potentially misleading explanations for models that are highly susceptible to implausible explanations. We therefore believe that \textit{ECCCo} can help researchers and practitioners to generate explanations they can trust and discern unreliable from trustworthy models."
+
+Elements of this explanation are already scattered across the paper, but we agree that it would be useful to highlight this notion also in Section 6.
+
 2. Core innovation: need more visualizations in 2D/3D space
+
+
+
 3. Structural clarity: add a flow chart
 
+Figure 2 shows our synthetic linearly separable data in the feature space, so the highlighted path corresponds to the actual counterfactual path of the sample. We will clarify this in the paper. 
+
+Adding a systematic flowchart is a great idea. Due to limited scope, may we suggest adding the following flowchart to the appendix? Alternatively, we may swap out Figure 2 for the flowchart. 
+
+
+
 # Reviewer 2
 
 ## Weaknesses:

experiments/notebooks/plots.qmd

+---
+format: pdf
+---
+
+# Plots
+
+```{julia}
+using Pkg; Pkg.activate("experiments")
+include("$(pwd())/experiments/setup_env.jl")
+```
+
+## Counterfactual Path - MNIST
+
+```{julia}
+outcome = Serialization.deserialize("results/linearly_separable_outcome.jls")
+data = outcome.exper.counterfactual_data
+```
+
+```{julia}
+factual = data.y_levels[2]
+target = data.y_levels[1]
+```
+
+```{julia}
+n_samp = 100
+plts = []
+for (mod_name, model) in outcome.model_dict
+    t = get_target_index(data.y_levels, target)
+    E(x) = -logits(model, x)[t, :]
+    x_samp = data.X[:,rand(findall(data.output_encoder.labels.==target),n_samp)]
+    x_rand = Float32.(randn(size(data.X,1), 1000))
+    dist = map(eachcol(x_rand)) do x
+        mean(map(y -> norm(x.-y),eachcol(x_samp)))
+    end
+    plt = scatter(E(x_rand), dist, alpha=0.5, label="", title=mod_name)
+    push!(plts, plt)
+end
+plot(plts..., layout=(1,length(outcome.model_dict)), size=(1000,250))
+```
\ No newline at end of file