Prepare your Jupyter Lab environment#

This notebook HPC-setup.ipynb is used to prepare your HPC Jupyter Lab environment on the first day of the training.

Execute he following cells to:

  • Install custom Python kernels with all packages used in the training sessions

  • Copy all notebooks and data used for training to your user accounts home directory

1. Copy notebooks and data#

This only needs to be done once!

The following cell copies the Jupyter notebooks and data used in the training sessions to your home directory.

It will create a directory ~/iom_notebooks with subdirectories for the sessions.

  1. Execute the cell with ctrl-enter or shift-enter

  2. When prompted to “Select Kernel”, leave the default Python 3 kernel and click “Select”.

  3. Wait until the copy process has finished

  4. In the file browser on the left side, click the refresh icon

  5. A directory iom_notebooks should appear

!bash /projects/p_scads_trainings/iom-ai/scripts/copy_data.sh

2. Install the Python Kernels#

This only needs to be done once!

For our practical sessions we will use different clusters on the TUD HPC resources:

  • Cluster romeo (CPU only) on Day 1 and Day 2

  • Cluster alpha (CPU+GPU) on Day 3 and Day 4

We need to install separate Python kernels for each cluster:

  • “iom-venv-torch-rom” with packages including pytorch for cluster romeo

  • “iom-venv-torch-alp” with packages including pytorch for cluster alpha

  • “iom-venv-tensor-alp” with packages including tensorflow for cluster alpha

Required steps:

  1. Execute the following cells with ctrl-enter or shift-enter

  2. When prompted to “Select Kernel”, leave the default Python 3 kernel and click “Select”.

  3. Wait for the according outputs

  4. Reload the page

  5. You can now select the mentioned Python kernels from the available kernels

Run this cell on Day 1 on cluster romeo only!#

Wait for this output:

  • Installed kernelspec iom-venv-torch-rom in [...]

!bash /projects/p_scads_trainings/iom-ai/scripts/install_python_kernel_romeo.sh

Run this cell on Day 3 on cluster alpha only!#

Wait for this outputs:

  • Installed kernelspec iom-venv-torch-alp in [...]

  • Installed kernelspec iom-venv-tensor-alp in [...]

!bash /projects/p_scads_trainings/iom-ai/scripts/install_python_kernel_alpha.sh