Based on documentation from Yale Research Computing
...
| Code Block |
|---|
| language | bash |
|---|
| title | Sample SLURM submit script |
|---|
|
#!/bin/bash
#SBATCH --partition=lts
#SBATCH --nodes=1
#SBATCH --ntasks-per-node 1
#SBATCH --time=1:00:00
#SBATCH --job-name=jupyter-notebook
#SBATCH --output=jupyter-notebook-%J.log
# Load Python module
module load anaconda/python3
# get tunneling info
export XDG_RUNTIME_DIR=""
ipnport=$(shuf -i8000-9999 -n1)
ipnip=$(hostname -s)
## print tunneling instructions to jupyter-log-{jobid}.txt
echo -e "
Copy/Paste this in your local terminal to ssh tunnel with remote
-----------------------------------------------------------------
ssh -N -L $ipnport:$ipnip:$ipnport ${USER}@sol.cc.lehigh.edu
-----------------------------------------------------------------
Then open a browser on your local machine to the following address
------------------------------------------------------------------
localhost:$ipnport
------------------------------------------------------------------
and use the token that appears below to login.
OR replace "$ipnip" in the address below with "localhost" and copy
to your local browser.
"
jupyter notebook --no-browser --port=$ipnport --ip=$ipnip |
Modify the above script for resources requested and submit using sbatch. A copy of this script is available at /share/Apps/examples/jupyter/jupyter.slurm. Submit it without modification using the command
| Code Block |
|---|
|
sbatch /share/Apps/examples/jupyter/jupyter.slurm |
Running Jupyter Notebook interactively on the compute node
If you need to run Jupyter Notebooks interactively, then you should request an interactive session on the partition of your choice.
| Code Block |
|---|
|
srun -p //partitionname// --ntasks-per-node=//number of cores per node// -N //number of nodes -t //time requested in HH:MM:SS// --pty /bin/bash |
Note that resources may not be available readily and you will need to wait until resources become available. Once resources are available, the scheduler will log you into a computer node. Load the appropriate Anaconda Python module (see available python modules for more information) and run the commands as listed in the above slurm script. For your convenience, the required commands are available in a script file, /share/Apps/examples/jupyter/notebook.sh.
| Code Block |
|---|
|
sh /share/Apps/examples/jupyter/notebook.sh |
The ssh command required to tunnel connections from your local system to the sol compute node and the jupyter notebook token will be printed on the screen.
Starting the Tunnel
Once you have submitted your job and it starts, your notebook server will be ready for you to connect. You can run squeue -u $(whoami) to check. You will see an “R” in the ST or status column for your notebook job if it is running. If you see a “PD” in the status column, you will have to wait for your job to start running to connect. The log file with information about how to connect will be in the directory you submitted the script from, and be named jupyter-notebook-[jobid].log where jobid is the slurm id for your job.
MacOS and Linux
On a Mac or Linux machine, you can start the tunnel with an SSH command. You can check the output from the job you started to get the specific info you need.
...
| Code Block |
|---|
|
Copy/Paste this in your local terminal to ssh tunnel with remote
-----------------------------------------------------------------
ssh -N -L 8854:sol-a102:8854 alp514@sol.cc.lehigh.edu
-----------------------------------------------------------------
Then open a browser on your local machine to the following address
------------------------------------------------------------------
localhost:8854
------------------------------------------------------------------
and use the token that appears below to login.
OR replace sol-a102 in the address below with localhost and copy
to your local browser.
[I 09:40:19.466 NotebookApp] [nb_conda_kernels] enabled, 6 kernels found
[I 09:40:22.584 NotebookApp] [nb_anacondacloud] enabled
[I 09:40:22.591 NotebookApp] [nb_conda] enabled
[I 09:40:22.678 NotebookApp] ✓ nbpresent HTML export ENABLED
[W 09:40:22.678 NotebookApp] ✗ nbpresent PDF export DISABLED: No module named 'nbbrowserpdf'
[I 09:40:22.678 NotebookApp] Serving notebooks from local directory: /home/alp514
[I 09:40:22.678 NotebookApp] The Jupyter Notebook is running at:
[I 09:40:22.679 NotebookApp] http://sol-a102:8854/?token=a693ea95fc6b239b33c466fc67cdc3a1a3b78198810abec5
[I 09:40:22.679 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 09:40:22.680 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
http://sol-a102:8854/?token=a693ea95fc6b239b33c466fc67cdc3a1a3b78198810abec5
[I 09:41:21.640 NotebookApp] 302 GET /?token=a693ea95fc6b239b33c466fc67cdc3a1a3b78198810abec5 (128.180.1.26) 1.73ms
[I 09:41:30.524 NotebookApp] Shutting down on /api/shutdown request.
[I 09:41:30.527 NotebookApp] Shutting down 0 kernels |
Create a ssh tunnel from your local system to the sol compute node. While running this example, the command used to tunnel into sol-a102 was
| Code Block |
|---|
|
ssh -N -L 8854:sol-a102:8854 alp514@sol.cc.lehigh.edu |
Once the tunnel was established, open a browser on your local system and use the link provided and replace node name with localhost. For example, the following link was used to access Jupyter Notebook wasfor the above example.
| Code Block |
|---|
|
http://localhost:8854/?token=a693ea95fc6b239b33c466fc67cdc3a1a3b78198810abec5 |
If you have trouble or need help, please feel free to contact Research Computing Staff.