Page Comparison

...

Users can select the number of cores and time limit (up to 4 hours) using the usual SLURM flags. It is also possible to use other partitions for interactive jobs with longer limits, but we cannot predict your wait times in advance.

Anchor
python python

Python

We have started to add popular python packages directly to the Lmod modules system, so that quick calculations require zero extra installation steps. For example:

$ module load python py-numpy
$ python
Python 3.13.0 (main, Jan  6 2025, 19:35:53) [GCC 12.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.random.randint(100)
97

If you need to install your own environments, we recommend the following procedure.

Note that we are using a trick to write a file from the command line using cat…EOF. This command should be copied through the second EOF and executed directly in the terminal. You could just as easily write the text into a new file with your favorite text editor.

module load python
# name your ceph project
CEPH_PROJECT=hpctraining_proj
# go to the shared forlder
cd $CEPH_PROJECT/shared
# the cat command below writes a spec file
# you should copy the entire multi-line command through the second EOF
cat > env-myenv-spec.txt <<EOF
scipy
seaborn
EOF
python -m venv ./venv-projectA
source ./venv-projectA/bin/activate
pip install -r env-myenv-spec.txt

Later, you can use this environment with:

source $CEPH_PROJECT/shared/venv-projectA/bin/activate

This procedure is the best way to add software to your Python-based software environment if it is not available when you search for module spider <some_package_name>.

Anchor
anaconda anaconda

Anaconda

We have installed a miniconda3 module so that users can build their own Anaconda environments. Please not that a standard Python virtual enviroment is the preferred way to build virtual environments unless you need access to the broader set of packages provided by conda.

NOTE: We have customized the miniconda3 module so that you should NOT need to modify your ~/.bashrc file to use conda environments. These customizations are extremely confusing on HPC systems that use Lmod to manage software. To avoid troubleshooting headaches later, we strongly recommend that you use module load miniconda3 instead of modifying your ~/.bashrc.

We recommend building your environments in the shared folder in your research group’s Ceph project. The following procedure will allow you to build a shared environment from a single specification (yaml) file.

Note that we are using a trick to write a file from the command line using cat…EOF. This command should be copied through the second EOF and executed directly in the terminal. You could just as easily write the text into a new file with your favorite text editor.

# instructions for maintaining a shared conda env
module load miniconda3
# name your ceph project
CEPH_PROJECT=hpctraining_proj
# go to the shared forlder
cd $CEPH_PROJECT/shared
# the cat command below writes a spec file
# you should copy the entire multi-line command through the second EOF
cat > env-conda-myenv-spec.yaml <<EOF
name: stats2
channels:
  - javascript
dependencies:
  - python=3.9
  - bokeh=2.4.2
  - conda-forge::numpy=1.21.*
  - nodejs=16.13.*
  - flask
  - pip
  - pip:
    - Flask-Testing
EOF
conda env update -f env-conda-myenv-spec.yaml -p ./env-conda-myenv
# the following is the path to this environment
echo $PWD/env-conda-myenv
# activate any arbitrary conda environment by using this path
module load miniconda3
conda activate ~/$CEPH_PROJECT/shared/env-conda-myenv

You can add any conda or pip packages to the Anaconda environment file we used above. The conda env update procedure above ensures that you can easily update or reproduce this environment on other systems.

Anchor
pending-software-requests pending-software-requests

Pending software requests

The following items are under construction as of January 9, 2025. We expect them to be added soon:

...

1. Many pieces of research software were requested for newer architectures, namely the Ice Lake and Sapphire rapids architectures provided by the arch/ice24v2 module.
2. Several other pieces of software requested by individual research groups. These are still in-progress. Users will be notified via a response to their tickets when the software has been added.

...