Create your own Jupyter Kernel


1. Start a Jupyter Application with Jupyter@JSC on the Login Node of the HPC System of your choice.
2. Open a new Terminal
3. Load these modules (you can paste into the terminal with SHIFT+Right Mouse Button):

JURON:

$ module purge
$ module load python jupyterlab/1.2.1

JURECA & JUWELS:

$ module purge
$ module use $OTHERSTAGES
$ module load Stages/Devel-2019a GCCcore/.8.3.0 Jupyter/2019a-Python-3.6.8

4. Create a virtual environment named at a path of your choice
python3 -m venv <your_path>/<name>

5. Activate your environment and update PYTHONPATH
$ source <your_path>/<name>/bin/activate
(<name>) $ export PYTHONPATH=<your_path>/<name>/lib/python3.6/site-packages:${PYTHONPATH}

6. Install python libraries of your choice
(<name>) $ pip install --ignore-installed ipykernel
(<name>) $ <install whatever else you need in your Python virtual environment using pip>

7. Create your Kernel
(<name>) $ python3 -m ipykernel install --user --name=<name>

8. Now you have to write a launch script for your kernel, to load other modules
(<name>) $ vi <your_path>/<name>/kernel.sh

Write into it:

#!/bin/bash
<the commands from bullet point 3>

<load your modules here, you can even load modules from other Stages>

source <your_path>/<name>/bin/activate
export PYTHONPATH=<your_path>/<name>/lib/python3.6/site-packages:${PYTHONPATH}
exec python -m ipykernel $@

Make the file executable:

(<name>) $ chmod +x <your_path>/<name>/kernel.sh

Here is an example of a kernel.sh file:



9. Update your kernel file to use the kernel.sh script

(<name>) $ vi ~/.local/share/jupyter/kernels/<name>/kernel.json

Change the argv part to this:

"argv": [
  "<your_path>/<name>/kernel.sh",
  "-f",
  "{connection_file}"
 ],

Here is an example of a kernel.json file:



10. Now you can use your new kernel! You have to restart your JupyterLab in Jupyter@JSC to see it in the launcher. But it can be directly used on the top right of each of your notebooks.

11. If other users should use your kernel they have to run these commands:

$ <your_path>/<name>/bin/python3 -m ipykernel install --user --name=<name>
$ vi ~/.local/share/jupyter/kernels/<name>/kernel.json
$ # Change the file like it is described in point 9