Skip to content
Snippets Groups Projects
Commit 5ccb7a39 authored by Kwangjin Lee's avatar Kwangjin Lee
Browse files

Merge branch 'main' into publish

parents 083dc60a 299dbce8
No related branches found
No related tags found
2 merge requests!4Update 2 files,!3Main
Pipeline #244998 passed
......@@ -18,7 +18,15 @@ before_script:
build-page:
stage: build
script:
- JEKYLL_ENV=production bundle exec jekyll build --config _config.yml --destination public
- |
if [[ "$CI_COMMIT_BRANCH" == "main" ]]; then
JEKYLL_ENV=production bundle exec jekyll build --config _config_draft.yml --destination public
elif [[ "$CI_COMMIT_BRANCH" == "publish" ]]; then
JEKYLL_ENV=production bundle exec jekyll build --config _config.yml --destination public
else
echo "Branch is neither main nor publish. Skipping build."
exit 1
fi
- echo "Listing contents of public directory:"
- ls -laR public
artifacts:
......
......@@ -17,7 +17,7 @@ title: MUDE 2024 website
tagline: CEGM1000 Modelling, Uncertainty, and Data Analysis for Engineers
description: Course website for TU Delft CEGM1000, or Modelling, Uncertainty, and Data Analysis for Engineers
author: By MUDE Teachers and the Student Army
baseurl: /2024-website # the subpath of your site, e.g. /blog
baseurl: /2024 # the subpath of your site, e.g. /blog
url: https://mude.citg.tudelft.nl # the base hostname & protocol for your site, e.g. http://example.com
exclude:
[
......@@ -41,14 +41,14 @@ heading_anchors: true
permalink: pretty
aux_links:
Brightspace:
- "https://brightspace.tudelft.nl/d2l/home/594596"
- "https://brightspace.tudelft.nl/d2l/home/680254"
Answers:
- "https://answers.citg.tudelft.nl"
Files:
- "https://mude.citg.tudelft.nl/course-files/"
- "https://mude.citg.tudelft.nl/2024/files"
Textbook:
- "https://mude.citg.tudelft.nl/book"
Main MUDE Site:
- "https://mude.citg.tudelft.nl/2024/book"
Home:
- "https://mude.citg.tudelft.nl"
aux_links_new_tab: true
footer_content: 'The MUDE Team &copy; 2024 <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY 4.0</a>'
......@@ -98,3 +98,5 @@ compress_html:
startings: []
blanklines: false
profile: false
draft_banner: "This is the DRAFT version of the MUDE 2024 website. Content may change frequently."
# Welcome to Jekyll!
#
# This config file is meant for settings that affect your whole site, values
# which you are expected to set up once and rarely edit after that. If you find
# yourself editing these this file very often, consider using Jekyll's data files
# feature for the data you need to update frequently.
#
# For technical reasons, this file is *NOT* reloaded automatically when you use
# 'jekyll serve'. If you change this file, please restart the server process.
# Site settings
# These are used to personalize your new site. If you look in the HTML files,
# you will see them accessed via {{ site.title }}, {{ site.github_repo }}, and so on.
# You can create any custom variable you would like, and they will be accessible
# in the templates via {{ site.myvariable }}.
title: MUDE 2024 website
tagline: CEGM1000 Modelling, Uncertainty, and Data Analysis for Engineers
description: Course website for TU Delft CEGM1000, or Modelling, Uncertainty, and Data Analysis for Engineers
author: By MUDE Teachers and the Student Army
baseurl: /2024/draft # the subpath of your site, e.g. /blog
url: https://mude.citg.tudelft.nl # the base hostname & protocol for your site, e.g. http://example.com
exclude:
[
"Gemfile",
"Gemfile.lock",
"LICENSE",
vendor,
"README.md",
"build-local.sh",
files,
"build-page.sh",
"_config.yml.bak",
]
logo: "/assets/logo/MUDE Logo.png"
# Theme settings
theme: just-the-docs
color_scheme: wider
search_enabled: true
heading_anchors: true
permalink: pretty
aux_links:
Brightspace:
- "https://brightspace.tudelft.nl/d2l/home/680254"
Answers:
- "https://answers.citg.tudelft.nl"
Files:
- "https://mude.citg.tudelft.nl/2024/files"
Textbook:
- "https://mude.citg.tudelft.nl/2024/book"
Home:
- "https://mude.citg.tudelft.nl"
aux_links_new_tab: true
footer_content: 'The MUDE Team &copy; 2024 <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY 4.0</a>'
# Collections for website data
collections:
staffers:
modules:
schedules:
announcements:
questions:
# Default layouts for each collection type
defaults:
- scope:
path: ""
type: staffers
values:
layout: staffer
height: 300
subpath: "/assets/images/"
width: 300
- scope:
path: ""
type: modules
values:
layout: module
- scope:
path: ""
type: schedules
values:
layout: schedule
- scope:
path: ""
type: announcements
values:
layout: announcement
- scope:
path: ""
type: questions
values:
layout: question
compress_html:
clippings: all
comments: all
endings: all
startings: []
blanklines: false
profile: false
draft_banner: "This is the DRAFT version of the MUDE 2024 website. Content may change frequently."
......@@ -14,4 +14,18 @@ seo:
MUDE is a module at the Civil Engineering and Geosciences faculty of Delft University of Technology: {{ site.tagline }}, taken by all first year students in the MSc degree programs Applied Earth Sciences (AES), Environmental Engineering (EE) and Civil Engineering (CE).
test.
\ No newline at end of file
We hope you are looking forward to kicking off your MSc program with the first MUDE lecture on Tuesday, September 3rd!
More information about the module will be provided in that session. For now you can see general information about the module [here](https://studiegids.tudelft.nl/a101_displayCourse.do?course_id=69347){:target="_blank"} as well as the course schedule via [MyTimetable](https://mytimetable.tudelft.nl/schedule){:target="_blank"}.
We know you are busy getting ready for the start of the year, but if you have any free time we encourage you to take a look at [this page for incoming students](https://mude.citg.tudelft.nl/incoming/){:target="_blank"} and try reviewing some of the material, depending on your prior experience.
If you have personal questions, you may send a request to MUDE-CEG@tudelft.nl. Please reserve general questions about the module until the end of the first lecture.
If you would like more information about the course contents before the first lecture, you can take a look at the [contents from last year](https://mude.citg.tudelft.nl/2023/){:target="_blank"}, or our [interactive online textbook](https://mude.citg.tudelft.nl/2023/book/intro.html){:target="_blank"}, but please note there will be significant changes, so only use it to get a general idea for what to expect!
We look forward to meeting you on Tuesday!
Kind regards,
The MUDE Team
\ No newline at end of file
---
layout: page
title: Webservers
description: Information about the setup and configuration of various webservers.
nav_order: 1
nav_exclude: false
permalink: /webservers/
---
This document describes the setup and configuration of various webservers. At the moment, the servers have very different setups. We will try to align them over time, but initially, work will be limited to `mude` and `interactivetextbooks`.
List of active servers with Ubuntu is here, for reference:
```
edu01.citg.tudelft.nl
hub01.citg.tudelft.nl
interactivetextbooks.citg.tudelft.nl
mude-utilities.citg.tudelft.nl
mude.citg.tudelft.nl
pybox.citg.tudelft.nl
```
# Webserver File System
The webserver hosts websites via `nginx` and the files are located in `/var/web_server/htdocs/`.
# GitLab Runner
GitLab Runner is used to execute the CI/CD (Continuous Integration and Continuous Deployment) jobs defined in a GitLab project's `.gitlab-ci.yml` file. It can be installed on various platforms, including Linux, macOS, Windows, and can also run in Docker containers. In our case, we have GitLab Runner installed on the mude-utilities Linux server.
Currently, most of the mude repositories are using the runner #1020 (rU9Qxsz9J).
### Runner Information
| **System ID** | **Status** | **Version** | **IP Address** | **Executor** | **Arch/Platform** |
|------------------|------------|-----------------------|--------------------|--------------|-------------------|
| s_1b6e44856200 | Online | 16.9.1 (782c6ecb) | 131.180.146.33 | docker | amd64/linux |
You can check this information via the [GitLab Runner Dashboard](https://gitlab.tudelft.nl/mude) -> **Build** -> **Runners**.
This runner is located on the `mude-utilities.citg.tudelft.nl` Ubuntu server. To check the status of the runner in the Linux terminal, use the following command:
```sh
sudo systemctl status gitlab-runner
```
# GitLab CI/CD
Note that the TU Delft GitLab (`gitlab.tudelft.nl`) has a limited set of features compared to EWI GitLab (`gitlab.ewi.tudelft.nl`).
There is a different setup on the `mude` and `interactivetextbooks` servers.
Here is the explanation for example `https://mude.citg.tudelft.nl/archive`
## GitLab CI/CD Pipeline Configuration
This configuration defines a CI/CD pipeline that deploys files to an archive directory on a remote server. The pipeline uses the latest Alpine Linux image and includes the following key sections:
### Stages
```yaml
stages:
- deploy
```
- Stages: Currently, only a deploy stage is defined. Typically, pipelines include test, build, and deploy stages.
Rules: The job triggers only on pushes to the main branch
```yaml
deploy-to-archive:
stage: deploy
#this part should be the same as the pre-defined ones.
rules:
- if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"
```
- Rules: The job is triggered only when a push is made to the main branch.
```yaml
before_script:
- apk update && apk add --no-cache rsync openssh
- eval $(ssh-agent -s)
- echo "$MUDE_DEPLOYER_SSH_KEY" | tr -d '\r' | ssh-add -
- mkdir -p ~/.ssh
- chmod 700 ~/.ssh
- echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
```
- Install Dependencies: Updates package list and installs rsync and openssh.
- SSH Setup: Initializes SSH agent, adds SSH key, and configures SSH to skip host key checking.
```yaml
script:
- 'echo "Current directory: $PWD"'
- ls -la
- if [ ! -d "src" ]; then echo "src directory does not exist. Creating it..."; mkdir src; echo "Moving necessary files to src directory..."; mv *.html *.js *.css src/ 2>/dev/null || true; fi
- echo "Contents of src directory:"
- ls -la src
- rsync -avz --delete-after --exclude=".git/" --exclude=".cache/" --exclude="venv/" src/ $MUDE_SERVER_USER@$MUDE_SERVER_ADDRESS:/tmp/archive_update/
- |
ssh $MUDE_SERVER_USER@$MUDE_SERVER_ADDRESS "
sudo rsync -avz --delete-after /tmp/archive_update/ /var/web_server/htdocs/archive/ &&
sudo chown -R root:root /var/web_server/htdocs/archive &&
sudo rm -rf /tmp/archive_update &&
sudo chmod 755 /var/web_server/htdocs/archive
"
```
- Debug Information: Displays current directory and lists files.
- Directory Preparation: Checks for src directory, creates it if missing, and moves necessary files into it.
File Syncing: Uses rsync to sync files to a temporary directory on the remote server.
- Remote Commands: Executes SSH commands to update the archive directory on the remote server:
Syncs files to the target directory.
Changes ownership and permissions.
Cleans up temporary files.
```yaml
environment:
name: production
url: https://mude.citg.tudelft.nl/archive
```
- Environment URL: Specifies the URL for the production environment.
Basically, this configuration ensures that only changes pushed to the main branch trigger the deployment, securely transfers files, and updates the remote server's archive directory.
This pipeline can be used for other server setup as well, but you need to define the CI/CD variables first.
## Setting Up CI/CD Variables
Navigate to your repository -> Settings -> CI/CD -> Variables
Define these variables:
1. MUDE_SERVER_ADDRESS: Current value = mude.citg.tudelft.nl
2. MUDE_SERVER_USER: Current value = kwangjinlee
3. MUDE_DEPLOYER_SSH_KEY: SSH private key for authentication
- This is already done for `mude.citg.tudelft.nl`
- First of all, create a ssh key using the following ssh-keygen command
```sh
ssh-keygen -t rsa -b 4096
```
- When you run this command:
- It will prompt you to choose a file location to save the key
- You'll have the option to set a passphrase (recommended for security)
- This generates two files:
- A private key (e.g., id_rsa)
- Define this as the MUDE_DEPLOYER_SSH_KEY CI/CD variable
- A public key (e.g., id_rsa.pub)
- Add this to ~/.ssh/authorized_keys on the server
This process enables SSH connection without specifying a key.
## Recommendations for Future Development
- Update the MUDE_SERVER_USER to ensure an authorized username is used.
- Consider adding additional stages (e.g., test, build) to the pipeline for a more comprehensive CI/CD process.
- Regularly review and update the SSH keys and server access permissions for security.
# `mude` Setup
To modify the server setup, SSH into `mude.citg.tudelft.nl`. You can use the command below:
```bash
sudo nano /var/web_server/website_docker_configuration/default.conf
```
```
location /archive {
alias /var/www/html/archive;
index index.html index.htm intro.html;
try_files $uri $uri/ =404;
}
location /incoming {
alias /var/www/html/incoming;
index index.html index.htm intro.html;
try_files $uri $uri/ =404;
}
location /teacher {
alias /var/www/html/teacher;
index index.html index.htm inro.html;
try_files $uri $uri/ =404;
}
```
This is the example of the subdomain:
- mude.citg.tudelft.nl/archive
- mude.citg.tudelft.nl/incoming
- mude.citg.tudelft.nl/teacher
All you need to do to set up the new subdomain is to write exactly as the example shows.
### Configuration Details
#### Alias
```nginx
alias /var/www/html/{domain name};
```
This sets the alias for the `/ {domain name}` location to the directory `/var/www/html/{domain name}`. When a request matches `/ {domain name}`, Nginx serves the files from this directory.
If you `cd /var/www/html`, you could see nothing except the default Nginx `index.html` file on the `mude.citg.tudelft.nl` Linux server. This is because the server is using a proxy. `/var/web_server/htdocs/` is where the actual contents stay.
#### Index Files
```nginx
index index.html index.htm intro.html;
```
This specifies the default files to serve if a directory is requested. Nginx will look for `index.html`, `index.htm`, or `intro.html` in that order.
To add a new subdomain and `index.html` for the new website, use the command:
```bash
cd /var/web_server/htdocs/
sudo mkdir {domain name}
sudo nano {domain name}/index.html
```
Write down anything in the `index.html` file. Press `(Ctrl+X -> Y)` to save and exit from the nano text editor. In this way, your `index.html` file will be created successfully.
#### Try Files
```nginx
try_files $uri $uri/ =404;
```
This directive attempts to serve the requested URI as a file or a directory. If neither exists, it returns a 404 error.
### Restarting Docker
After you finish the setup above, you need to restart the Docker container with the following commands:
1. Check if the configuration is correct:
```bash
sudo docker exec website_docker_configuration-nginx-1 nginx -t
```
```
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
```
If it shows this, then it is well done.
2. If the test is successful, reload Nginx:
```bash
sudo docker exec website_docker_configuration-nginx-1 nginx -s reload
```
```
[notice] 36#36: signal process started
```
This message means the server is successfully reloaded.
Now you can navigate to `mude.citg.tudelft.nl/{domain name}` to test your new website!
# `interactivetextbooks` setup
_kwangjin_
# `SSL Certificate Setup` Guide
### Before You Begin
- **Authentication:** Use your NetID login credentials for all server access.
- **File Handling:** Commands assume copying, pasting, or transferring files to the current working directory.
### Generating CSR (Certificate Signing Request) on Webserver
- **Create CSR File on Webserver:**
- Follow [this guide](https://tu-delft-dcc.github.io/infrastructure/VPS_SSL_Certs.html) to generate `.csr` and `.key` files on your webserver.
### Transferring CSR from Server to Local Machine
1. **Secure Copy (SCP) from Server to Local:**
- Access the intermediary server:
```
ssh {NetID}@student-linux.tudelft.nl or {NetID}@linux-bastion.tudelft.nl
```
- Copy the `.csr` file from server to intermediary server:
```
scp mude.citg.tudelft.nl:mude.citg.tudelft.nl.csr /home/nfs/{NetId}/
```
- Copy the `.csr` file from intermediary server to your local machine:
```
scp {NetID}@student-linux.tudelft.nl:/home/nfs/{NetID}/mude.citg.tudelft.nl.csr .
```
### Requesting SSL Certificate
1. **Submit CSR to TUDelft Top Desk:**
- Visit [TUDelft Top Desk](https://www.tudelft.topdesk.net) and submit the `.csr` file to request your SSL certificate.
### Uploading PEM File to Webserver
- To figure out where the current cerficate exists, use the command below:
```
sudo nano /etc/nginx/sites-available/mude
```
```
server {
# SSL configuration
#
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/certs/mude_citg_tudelft_nl.pem;
ssl_certificate_key /etc/ssl/private/mude.citg.tudelft.nl.key;
```
In this example, ssl_certificate and key are staying in (This key file should be created with the .csr)
- /etc/ssl/certs
- /etc/ssl/private
1. **SCP from Local Machine to Intermediary Server:**
- Upload `.pem` file to intermediary server:
```
scp mude.citg.tudelft.nl.pem {NetID}@student-linux.tudelft.nl:/home/nfs/{NetID}/
```
2. **SCP from Intermediary Server to Webserver:**
- Access intermediary server:
```
ssh {NetID}@student-linux.tudelft.nl
```
- Transfer `.pem` file to webserver:
```
scp /home/nfs/{NetID}/mude.citg.tudelft.nl.pem {NetID}@mude.citg.tudelft.nl:
```
### Updating SSL Certificate
- Backup existing SSL files:
```
sudo cp /etc/ssl/certs/mude_citg_tudelft_nl.pem /etc/ssl/certs/mude_citg_tudelft_nl.pem.bak
sudo cp /etc/ssl/private/mude.citg.tudelft.nl.key /etc/ssl/private/mude.citg.tudelft.nl.key.bak
```
- Replace with new `.pem` and `.key` files:
```
sudo cp /path/to/new/mude.citg.tudelft.nl.pem /etc/ssl/certs/mude_citg_tudelft_nl.pem
sudo cp /path/to/new/mude.citg.tudelft.nl.key /etc/ssl/private/mude.citg.tudelft.nl.key
```
- Set permissions:
```
sudo chmod 777 /etc/ssl/certs/mude_citg_tudelft_nl.pem
sudo chmod 600 /etc/ssl/private/mude.citg.tudelft.nl.key
sudo chown root:root /etc/ssl/certs/mude_citg_tudelft_nl.pem
sudo chown root:root /etc/ssl/private/mude.citg.tudelft.nl.key
```
- Update nginx configuration:
```
sudo nano /etc/nginx/sites-available/mude
```
Update `ssl_certificate` and `ssl_certificate_key` paths to point to new files. (It will be the same if you follow the instructions)
- Verify and restart nginx:
```
sudo nginx -t
sudo systemctl restart nginx
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment