Skip to content
Snippets Groups Projects
Commit 9b2b945b authored by Tom van Woudenberg's avatar Tom van Woudenberg
Browse files

Merge branch 'quizes-and-interactive' into 'main' 

Added interactive templates

See merge request !73
parents 90879d3b 96c46b49
No related branches found
No related tags found
2 merge requests!80check toc purge,!73Added interactive templates
Pipeline #207145 passed
Stage: test
Expand all Hide whitespace changes
Inline Side-by-side
Showing
with 372 additions and 779 deletions
book/_toc.yml
+ 5
− 2
View file @ 9b2b945b
......@@ -153,8 +153,11 @@ parts:
- file: cookbook/benchmarks/benchmark_load.ipynb
- file: cookbook/widgets.ipynb
- file: cookbook/exercise_checking.ipynb
- file: cookbook/example_gumbel.ipynb
- file: cookbook/example_quiz_interactive.ipynb
- file: cookbook/interactive_templates
sections:
- file: cookbook/example_gumbel.ipynb
- file: cookbook/example_quiz_interactive.ipynb
- file: cookbook/coding_theory_widgets.ipynb
- caption: Old Material
chapters:
- file: old/blank
......
book/cookbook/coding_theory_widgets.ipynb 0 → 100644
+ 329
− 0
View file @ 9b2b945b
This diff is collapsed.
book/cookbook/example_gumbel.ipynb
+ 13
− 2
View file @ 9b2b945b
%% Cell type:markdown id: tags:
# Gumbel Distribution Exercise
# Programming exercise
This page shows how you can convert the classical Jupyter Notebook assignment to a jupyter book format with live coding. A widget has been implemented to check the answers, of which the data is included in a separate python file.
%% Cell type:markdown id: tags:
## Gumbel Distribution Exercise
Imagine you are concerned with the concentration of an airborne contaminant, $X$, measured in ppm. A previous benchmark study estimates that there is a 10% and 1% chance, respectively, of exceeding 4 and 10 ppm, respectively. You have been asked by the regulatory agency to estimate the contaminant concentration with 0.1% probability of being exceeded. Prior studies suggest that a Gumbel distribution can be used to model contaminant concentration, given by the CDF:
$$
F_X(x) = \exp{\Bigg( -\exp{\Big(-\frac{x-\mu}{\beta}\Big)}\Bigg)}
$$
Using the cell blocks below as a guide (and also to check your analysis): Task 1) find the parameters of a Gumbel distribution that matches the information provided above, then, Tasks 2-3) use it to estimate the concentration with exceedance probability 0.1%.
To complete this assignment, you can use `numpy`, `matplotlib` and from the `math` library, `log` and `e` (these are imported for you when you inialize the notebook).
For this exercise, activate the live coding by clicking {fa}`rocket` --> {guilabel}`Live Code` on the top of this page. As soon as this is loaded, you can start the exercise.
%% Cell type:code id: tags:thebe-remove-input-init
``` python
%pip install ipywidgets
import matplotlib.pyplot as plt
from math import log, e
import numpy as np
from example_gumbel import check_example
```
%% Cell type:markdown id: tags:
**Task 0:** fill in the appropriate fitting points:
%% Cell type:code id: tags:
``` python
x_1 = _
p_1 = _
x_2 = _
p_2 = _
```
%% Cell type:markdown id: tags:
**Task 1:** derive the distribution parameters:
%% Cell type:code id: tags:
``` python
def gumbel_2_points(x_1, p_1, x_2, p_2):
"""Compute Gumbel distribution parameters from two points of the CDF.
Arguments:
x_1 (float): point one
p_1 (float): cumulative probability for point 1
x_2 (float): point two
p_2 (float): cumulative probability for point 2
"""
# YOUR CODE GOES HERE #
beta = _
mu = _
#######################
return mu, beta
```
%% Cell type:markdown id: tags:
The cells below will print your parameter values and create a plot to help confirm you have the right implementation.
%% Cell type:code id: tags:thebe-remove-input-init
``` python
def plot_distribution(mu, beta,
x_1, p_1, x_2, p_2):
plt.title("Gumbel Distribution, $1-F_X(x)$")
plt.xlabel("Contaminant Concentration, $X$ [ppm]")
plt.ylabel("Exceedance Probability [--]")
plt.grid(color='black', linestyle='-', linewidth=0.3)
x_axis = np.arange(0, 20, 0.1)
plt.plot(x_axis, np.vectorize(gumbel_distribution)(x_axis), linewidth=2)
plt.plot(x_1, 1 - p_1, 'ro')
plt.annotate("Point 1", (x_1 + 0.4, 1 - p_1 + 0.02))
plt.plot(x_2, 1 - p_2, 'ro')
plt.annotate("Point 2", (x_2 + 0.4, 1 - p_2 + 0.001))
plt.yscale("log")
plt.show()
```
%% Cell type:code id: tags:
``` python
mu, beta = gumbel_2_points(x_1, p_1, x_2, p_2)
print(f"Your mu: {mu:0.5f}\nYour beta: {beta:0.5f}")
gumbel_distribution = lambda x: 1 - e**(-e**(-(x - mu)/beta))
plot_distribution(mu, beta, x_1, p_1, x_2, p_2)
```
%% Cell type:markdown id: tags:
**Task 2:** write a function to solve for the random variable value (it will be tested for you with the Check answer button, along with the distribution parameters).
%% Cell type:code id: tags:
``` python
def find_x_with_probability_p(p):
""" Compute point in the gumbel distribution for which the CDF is p
Use the variables mu and beta defined above!
Hint: they have been defined globally, so you don't need to
include them as arguments.
"""
# YOUR CODE GOES HERE #
x = _
#######################
return x
```
%% Cell type:markdown id: tags:
**Task 3:** use the function `find_x_with_probability_p` to evaluate the random variable value with exceedance probabiltiy of 0.001.
%% Cell type:code id: tags:
``` python
# YOUR CODE GOES HERE #
x = _
#######################
print(f"The value of x with probability of exceedance 0.001 os {x:0.5f}")
```
%% Cell type:code id: tags:thebe-remove-input-init
``` python
check_example(globals())
```
%% Cell type:code id: tags:
``` python
```
......
book/cookbook/example_quiz_interactive.ipynb
+ 6
− 6
View file @ 9b2b945b
%% Cell type:markdown id: tags:
# Example theory-quiz-code page
# Combining theory with small interactive questions and in-depth coding exercise
This page shows an example of how to combine theory, quizes and code on one page. With headings and boxes, the interactive parts can be indicated (for now only if part of one cell)
Remark:
- The coding is not interactive when the {fa}`rocket` --> {guilabel}`Live Code` is not clicked, you have to make that clear to the reader. You can make sure all relevant cells are executed upon starting the interactive mode by adding `thebe-init` or `thebe-remove-input-init` as a cell tag.
- at this moment, the use of a jupyter-quiz combined with interactive code does not work.
## Weighted least-squares estimation
In ordinary least-squares estimation, we assume that all observations are equally important. In many cases this is not realistic, as observations may be obtained by different measurement systems, or under different circumstances. We want our methodology for least-squares estimation to be able to take this into account.
We achieve this goal by introducing a weight matrix in the minimization problem:
$$
\underset{\mathrm{x}}{\min} {\mathrm{(y-Ax)^T W(y-Ax)}}
$$
In the unweighted least-squares approach, we arrive at the normal equation by pre-multiplying both sides of $\mathrm{y=Ax}$ with the transpose of the design matrix $\mathrm{A^T}$:
$$
\mathrm{y=Ax} \; \rightarrow \; \mathrm{A^T\; y = A^T\; A x }
$$
In the weighted least-squares approach, we now need to add weight matrix $W$ to this pre-multiplication factor, i.e., $ \mathrm{A^T W}$, to obtain the normal equation:
$$
\mathrm{y=Ax} \; \rightarrow \; \mathrm{A^T W \; y = A^TW \; A x}
$$
The normal matrix is now defined as $\mathrm{N=A^T W A}$. From this, assuming that the normal matrix $N$ is invertible (non-singular) we find the weighted least-squares estimate $ \mathrm{\hat{x}} $,
$$
\begin{align*}
\mathrm{\hat{x}} &= \mathrm{(A^T W A)^{-1} A^T W y} \\
&= \arg \underset{\mathrm{x}}{\min} {\mathrm{(y-Ax)^T W(y-Ax)}}
\end{align*}
$$
We also find the derived estimate $ \mathrm{\hat{y}} $ and $ \mathrm{\hat{\epsilon}} $:
$$
\mathrm{\hat{y} = A \hat{x} = A (A^T W A )^{-1} A^T W y}
$$
$$
\mathrm{\hat{\epsilon} = y - \hat{y}= y - A \hat{x} = y-A (A^T W A )^{-1} A^T W y = (I- A(A^T W A )^{-1} A^T W) y}
$$
## Quiz question
### Quiz question
:::{card}
<iframe src="https://tudelft.h5p.com/content/1292046737060674407/embed" aria-label="WLS_1" width="1088" height="637" frameborder="0" allowfullscreen="allowfullscreen" allow="autoplay *; geolocation *; microphone *; camera *; midi *; encrypted-media *"></iframe><script src="https://tudelft.h5p.com/js/h5p-resizer.js" charset="UTF-8"></script>
:::
## Video
### Video
```{eval-rst}
.. raw:: html
<iframe width="560" height="315" src="https://www.youtube.com/embed/iJmkkz37EuU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
```
## Discussion on the weight matrix
### Discussion on the weight matrix
The weight matrix $\mathrm{W}$ expresses the (relative) weights between the observations. It is always a square matrix. The size of the weight matrix depends on the number of observations, $m$. The size of the weight matrix is $m\times m$.
If it is a unit matrix ($\mathrm{W=I}$), this implies that all observations have equal weight. Note that in this case the equations are equal to the ordinary least-squares solution.
If it is a diagonal matrix, with different values on the diagonal, this implies that entries with a higher value correspond to the observations which are considered to be of more importance. If the weight matrix has non-zero elements on the off-diagonal positions, this implies that (some) observations are correlated.
## Weighted least-squares estimator: properties
### Weighted least-squares estimator: properties
Until now, we have looked at the weighted least-squares solution of a single *realization* of the observations, where generally we assume that it is a realization of the *random* observable vector $Y$, since measurements are affected by random errors. As such it follows the the weighted least-squares *estimator* is given by:
$$
\hat{X} = \mathrm{(A^T W A )^{-1} A^T W} Y
$$
This estimator has two important properties: it is *linear* and *unbiased*.
The linearity property is due to the fact that $\hat{X}$ is a linear function of the observables $Y$.
The unbiased property means that the expectation of $\hat{X}$ is equal to the true (but unknown) $\mathrm{x}$. This can be shown as follows:
$$
\mathbb{E}(\hat{X}) = \mathrm{(A^T W A )^{-1} A^T W} \mathbb{E}(Y) = \mathrm{(A^T W A )^{-1} A^T W Ax = x}
$$
This a very desirable property. It applies that if we would repeat the measurements many times to obtain a new estimate, the *average of the estimated* values would be equal to the truy values.
%% Cell type:markdown id: tags:
## Exercise
### Exercise
:::{card}
For this exercise, activate the live coding by clicking {fa}`rocket` --> {guilabel}`Live Code` on the top of this page. As soon as this is loaded, you can start coding yourself.
You have a time series of 8 measurements and fit a model assuming a linear trend (constant velocity).
Times of observations, observed values and number of observations are given.
Remark: It would be nice if the box in which this exercise is given extends to all cells below.
%% Cell type:code id: tags:thebe-remove-input-init
``` python
# this will print all float arrays with 3 decimal places
import numpy as np
float_formatter = "{:.2f}".format
np.set_printoptions(formatter={'float_kind':float_formatter})
%pip install ipywidgets
import ipywidgets as widgets
from IPython.display import display
import operator
def check_answer(variable_name, expected, comparison = operator.eq):
output = widgets.Output()
button = widgets.Button(description="Check answer")
def _inner_check(button):
with output:
if comparison(globals()[variable_name], expected):
output.outputs = [{'name': 'stdout', 'text': 'Correct!', 'output_type': 'stream'}]
else:
output.outputs = [{'name': 'stdout', 'text': 'Incorrect!', 'output_type': 'stream'}]
button.on_click(_inner_check)
display(button, output)
```
%% Cell type:code id: tags:thebe-init
``` python
import numpy as np
import matplotlib.pyplot as plt
```
%% Cell type:code id: tags:thebe-init
``` python
# times of observation [months]
t = np.arange(8)
# observed heights [m]
y = [1.39, 1.26, 1.48, 4.03, 5.89, 5.14, 6.47, 7.64]
# number of observations
m = len(t)
```
%% Cell type:markdown id: tags:
Fill in the correct $\mathrm{A}$-matrix.
%% Cell type:code id: tags:
``` python
# design matrix
A = np.column_stack((np.ones(m), t))
```
%% Cell type:code id: tags:thebe-remove-input-init
``` python
check_answer("A", np.column_stack((np.ones(m), t)), np.array_equiv)
```
%% Cell type:markdown id: tags:
Define the weight matrix for $\mathrm{W}=I_m$
%% Cell type:code id: tags:thebe-remove-input-init
``` python
W_1 = 1 #added to have initial input
```
%% Cell type:code id: tags:
``` python
# Weight matrix for case 1
W_1 = ?
```
%% Cell type:code id: tags:thebe-remove-input-init
``` python
check_answer("W_1",np.eye(m), np.array_equiv)
```
%% Cell type:markdown id: tags:
Define the weight matrix with the weight of first 3 observations is five times as large as the rest
%% Cell type:code id: tags:thebe-remove-input-init
``` python
W_2 = 1
```
%% Cell type:code id: tags:
``` python
# Weight matrix for case 2
W_2 = ?
```
%% Cell type:code id: tags:thebe-remove-input-init
``` python
# Weight matrix for case 2
w=5
W_2_ans = np.eye(m)
W_2_ans[0,0] = w
W_2_ans[1,1] = w
W_2_ans[2,2] = w
check_answer("W_2",W_2_ans, np.array_equiv)
```
%% Cell type:markdown id: tags:
Define the weight matrix with the weight of 4th and 5th observation is five times as large as the rest
%% Cell type:code id: tags:
``` python
# Weight matrix for case 3
W_3 = ?
```
%% Cell type:markdown id: tags:
:::
......
book/cookbook/exercise_checking.ipynb
+ 1
− 1
View file @ 9b2b945b
%% Cell type:markdown id: tags:
# Exercise checking with interactive elements
# Exercise checking with widgets
Exercise checking can be intuitively incorporated into Python using tags. These examples show different approaches to achieving this goal.
%% Cell type:code id: tags:thebe-remove-input-init
``` python
%pip install ipywidgets
import ipywidgets as widgets
from IPython.display import display
import operator
def check_answer(variable_name, expected, comparison = operator.eq):
output = widgets.Output()
button = widgets.Button(description="Check answer")
def _inner_check(button):
with output:
if comparison(globals()[variable_name], expected):
output.outputs = [{'name': 'stdout', 'text': 'Correct!', 'output_type': 'stream'}]
else:
output.outputs = [{'name': 'stdout', 'text': 'Incorrect!', 'output_type': 'stream'}]
button.on_click(_inner_check)
display(button, output)
```
%% Cell type:code id: tags:thebe-init
``` python
# This example has the user type in the answer as a Python variable, but they need to run the cell to update the answer checked!
pi = 3.14
```
%% Cell type:code id: tags:thebe-remove-input-init
``` python
import math
check_answer("pi", 3.14, math.isclose)
```
......
book/cookbook/interactive_element.ipynb
+ 1
− 767
View file @ 9b2b945b
This diff is collapsed.
book/cookbook/interactive_templates.md 0 → 100644
+ 16
− 0
View file @ 9b2b945b
# Templates for interactive coding / non-coding elements
These pages show examples of how to use interactive coding / non-coding elements in a Jupyter Book.
You can make use of:
- Admonitions
- JupyterQuiz, for now this shouldn't be combined with thebe on a single page
- H5p
- Grasple-exercises
- Interactive widgets
- Live coding exercise checking with widgets
- Any combination of the above
The following templates are given:
```{tableofcontents}
```
\ No newline at end of file
book/cookbook/widgets.ipynb
+ 1
− 1
View file @ 9b2b945b
......@@ -200,7 +200,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
"version": "3.9.17"
},
"orig_nbformat": 4
},
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment