/
Software

Software

Jul 08, 2022

 

Several free and open source software and libraries are installed and accessible using the module command. A few licensed software from https://software.lehigh.edu are also available. Since, all Sol nodes have infiniband interconnect while Hawk nodes do not, the default MPI implementation is mvapich2 on Sol and mpich on Hawk. The default compiler for building applications is Intel 20.0.3 while the default compiler for GPU applications is 19.0.3. All software is compiled with both mvapich2 and mpich for AVX, AVX2 and AVX512 instruction set. All libraries are built with multiple compilers - gcc 8.3.1 and 9.3.0, and intel 19.0.3 and 20.0.3 

Module Environment

The Module package provides for the dynamic modification of a users's environment via module files. Module can be used:

  • to manage necessary changes to the environment, such as changing the default path or defining environment variables

  • to manage multiple versions of applications, tools and libraries

  • to manage software where name conflicts with other software would cause problems

Modules have been created for many of the software packages installed on Sol. They make your job easier by defining environment variables and adding directories to your path which are necessary when using a given package. 

Some useful module commands are:

Command

Description

Command

Description

module avail

lists all the available modules

module help foo

displays help on module foo, for some applications usage information is also displayed

module show foo

indicates what changes would be made to the environment by loading module foo without actually loading it

module load foo

loads module foo

module list

displays your currently loaded modules

module swap foo1 foo2

switches loaded module foo1 with module foo2

module unload foo

reverses all changes to the environment made by previously loading module foo

module spider foo

Query all modules in MODULEPATH and any module hierarchy

module use [-a] [path]

Prepend or Append path to MODULEPATH

module unuse [path]

Remove path from MODULEPATH

Since Feb 1 2021, LMOD is used to provide the module environment and not all modules are available. Only modules for applications and libraries built with the loaded compiler and mpi combination are visible. On the Sol head node, intel 20.0.3 and mvapich2 2.3.4 are loaded by default. To see GPU enabled packages, load the intel 19.0.3 module. To use applications and libraries built with the system default compiler, gcc 8.3.1, unload the intel compiler.

Installed Software

For software with multiple version installed, default modules are listed in bold face.

Please feel free to build your own software stack. If you need to fine tune or rebuild (for e.g. different compiler/mpi combination) some of the installed packages, please feel free to use the installation scripts (not all are available) from https://gogs.cc.lehigh.edu/alp514/solsoftware (Lehigh IP or VPN required). SPACK is used to built the HPC software stack and the installation scripts are not updated for the current compiler and module names, nor have they been tested for the current versions of software.

Available Software on Sol (replace mvapich2 with mpich for Hawk)
[2021-08-06 07:08.25] ~ [alp514.sol](1001): module av ------------------------------------------------------------- /share/Apps/share/Modules/lmod/linux-centos8-x86_64/mvapich2/2.3.4/intel/20.0.3 -------------------------------------------------------------- crystal17/1.0.2 dl_poly/4.09 lammps/20200303-anj6 namd/2.14 nwchem/7.0.2 openmolcas/19.11 opensees/3.3.0 (D) vasp/5.4.4.pl2-neb vasp/5.4.4.pl2 vasp/5.4.4 vasp/6.1.2 (D) ---------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/mvapich2/2.3.4-wguydha/intel/20.0.3 ----------------------------------------------------- adios2/2.6.0 globalarrays/5.7 hdf5/1.10.7 (D) lammps/20201029 (D) netcdf-cxx/4.2 (D) petsc/3.14.1 quantum-espresso/6.5 shengbte/1.1.1-8a63749 almabte/1.3.2 gromacs/2020.4-plumed hypre/2.20.0 mumps/5.3.3 netcdf-fortran/4.5.3 (D) plumed/2.6.2 quantum-espresso/6.6 (D) superlu-dist/6.3.1 boost/1.74.0 (D) gromacs/2020.4 (D) lammps/20200303 netcdf-c/4.7.4 (D) parmetis/4.0.3 py-espresso/4.0.2 relion/3.1.1 ---------------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/intel/20.0.3 ---------------------------------------------------------------- arpack-ng/3.7.0 (D) gsl/2.5 (D) metis/5.1.0 (D) netcdf-c/4.7.4 openblas/0.3.10 (D) py-pyparsing/2.4.2 (D) suite-sparse/5.7.2 (D) boost/1.74.0 hdf5/1.10.7 mpich/3.3.2 (D) netcdf-cxx/4.2 openmpi/4.0.5 (D) qhull/2020.1 (D) superlu-mt/3.1 (D) fftw/3.3.8 (D) intel-mkl/2020.3.279 (D) mvapich2/2.3.4 (L,D) netcdf-fortran/4.5.3 opensees/3.3.0 qrupdate/1.1.2 (D) superlu/5.2.1 (D) -------------------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/Core -------------------------------------------------------------------- angsd/0.933 canu/1.8 gatk/4.1.0.0 libarchive/3.4.3 ngstools/2019-06-24 picard/2.20.8 snpeff/5.0 arpack-ng/3.7.0 cuda/10.2.89 gcc/9.3.0 magma/2.5.4 nlopt/2.6.2 pilon/1.22 stacks/2.55 bamtools/2.5.1 cuda/11.0.2 gmap-gsnap/2020-06-01 masurca/3.4.2 nvhpc/20.9 py-pyparsing/2.4.2 star/2.7.0e bartender/1.1 cuda/11.1.0 (D) gnuplot/5.2.8 metis/5.1.0 octave/5.2.0 pythia8/8303 subread/2.0.0 bayescan/2.1 cudnn/8.0.4.30-11.0-linux-x64 gsl/2.5 miniasm/2018-3-30 openbabel/3.0.0 python/3.8.6 (L) suite-sparse/5.7.2 bayestraits/3.0.2 eigen/3.3.8 hal/2.1 minimap2/2.14 openblas/0.3.10 qhull/2020.1 superlu-mt/3.1 bgc/1.03 exonerate/2.4.0 hdf5/1.10.7 mpich/3.3.2 openjdk/1.8.0_222-b10 qrupdate/1.1.2 superlu/5.2.1 blast-plus/2.9.0 express/1.5.2 hisat2/2.2.0 mvapich2/2.3.4 openjdk/11.0.2 (D) r/4.0.3 swig/4.0.2 (D) blat/35 fastjet/3.3.3 intel-mkl/2020.3.279 nccl/2.7.8-1 openmpi/4.0.5 rmcprofile/6.7.9 tabix/2013-12-16 boost/1.74.0 fastqc/0.11.7 intel/19.0.3 netcdf-c/4.7.4 paml/4.9j root/6.20.08 trimmomatic/0.39 bowtie/1.3.0 fastx-toolkit/0.0.14 intel/20.0.3 (L,D) netcdf-cxx/4.2 pear/0.9.11 rsem/1.3.1 trinity/2.12.0 bowtie2/2.4.1 fftw/3.3.8 jellyfish/2.2.7 netcdf-fortran/4.5.3 perl/5.32.0 salmon/0.14.1 vcftools/0.1.15 bwa/0.7.17 freebayes/1.1.0 kallisto/0.46.2 ngmlr/0.2.7 phast/1.4 samtools/1.10 vesta/3.4.6 ------------------------------------------------------------------ /share/Apps/lusoft/share/spack/lmod/default/linux-centos8-x86_64/Core ------------------------------------------------------------------- anaconda3/2019.10 axel/2.16.1 graphviz/2.44.1 libxp/1.0.3 ncdu/1.13 pwgui/6.1 scons/3.1.2 tcl/8.6.10 valgrind/3.15.0 anaconda3/2020.07 (D) cmake/3.18.4 julia/1.5.2 motif/2.3.8 parallel/20200822 qcachegrind/20.08.0 screen/4.8.0 tk/8.6.10 vmd/1.9.3 aria2/1.35.0 ddd/3.3.12 krb5/1.18.2 nano/5.6.1 povray/3.7.0.10 rclone/1.53.3 swig/4.0.2 tmux/3.1b (L) --------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/applications/licensed --------------------------------------------------------------------------- abaqus/2019 cactus/1.2.3 comsol/5.4 guppy/3.1.5-gpu guppy/4.0.14 (D) mathematica/12.0 simmodeler/8.0 xds/feb2021 (D) ansys/20.2 cactus/1.3.0 (D) gaussian/g09 (D) guppy/3.1.5 gurobi/8.1.1 matlab/R2020a stata/15 xds/jan2020 ansys/21.1 (D) ccp4/7.0 gaussian/g16 guppy/4.0.14-gpu knitro/10.3.0 phenix/1.16 sysweld/2020.5 ---------------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/default ---------------------------------------------------------------------------------- cent7 gpu hawk hawkgpu sol (L) solgpu soltools (L) --------------------------------------------------------------- /share/Apps/share/Modules/lmod/linux-centos8-x86_64/py_venv/pythonapps/avx2 ---------------------------------------------------------------- all htseq/0.11.2-m5yqkbo matplotlib/3.3.3-dttvy5f (D) pandas/1.1.4-sdbbcxk pytorch/1.7.0-cuda seaborn/0.9.0-7xrppb4 biopython/1.73-aoor5vo jupyter/1.0.0-7ew3qb6 mdanalysis/1.0.0-tfngrft phonopy/1.10.0-iuaj2xc scikit-build/0.10.0-vww4qdm six/1.15.0-obed3ik cclib/1.5.post1-wv3arnb jupyterlab/2.2.7-ln6bc44 moltemplate/2.5.8-iclz2ic pip/20.2-ig2u7ky scikit-image/0.17.2-qhu4bnf tensorflow/2.4.1-cuda cython/0.29.21-trlkbz5 keras/2.2.4-cuda-rihsv3p multiqc/1.7-xi6ehki porechop/0.2.4-tgrse4w scikit-learn/0.23.2-nnpi22y theano/1.0.4-cuda-srh7rqj dask/2.16.0-hcixvbx lxml/4.5.2-2d2z266 numba/0.50.1-orbwfra pyparsing/2.4.2-yktitpw scikit-optimize/0.5.2-xngjbiq virtualenv/16.7.6-yh3ujrd horovod/0.21.3-cuda matplotlib/2.2.5-fimnl7l numpy/1.19.4-hyxetpw pyspark/3.0.1-f3qh2hw scipy/1.5.4-fbs3ffk virtualenvwrapper/4.8.2-vozt6md Where: L: Module is loaded D: Default Module Use "module spider" to find all possible modules and extensions. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

 

Available GPU enabled software on Hawk (replace mpich with mvapich2 for Sol)
[2021-08-06 07:11.02] ~ [alp514.sol](1003): module av --------------------------------------------------------------- /share/Apps/share/Modules/lmod/linux-centos8-x86_64/mpich/3.3.2/intel/19.0.3 --------------------------------------------------------------- namd/2.14 ------------------------------------------------------ /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/mpich/3.3.2-hj3gqoq/intel/19.0.3 ------------------------------------------------------ boost/1.74.0 (D) gromacs/2020.4 (D) hypre/2.20.0 lammps/20201029 (D) netcdf-c/4.7.4 (D) netcdf-fortran/4.5.3 (D) petsc/3.14.1 py-espresso/4.0.2 superlu-dist/6.3.1 gromacs/2020.4-plumed hdf5/1.10.7 (D) lammps/20200303 mumps/5.3.3 netcdf-cxx/4.2 (D) parmetis/4.0.3 plumed/2.6.2 relion/3.1.1 ---------------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/intel/19.0.3 ---------------------------------------------------------------- arpack-ng/3.7.0 (D) gsl/2.5 (D) metis/5.1.0 (D) netcdf-c/4.7.4 openblas/0.3.10 (D) qrupdate/1.1.2 (D) superlu/5.2.1 (D) boost/1.74.0 hdf5/1.10.7 mpich/3.3.2 (L,D) netcdf-cxx/4.2 openmpi/4.0.5 (D) suite-sparse/5.7.2 (D) fftw/3.3.8 (D) intel-mkl/2020.3.279 (D) mvapich2/2.3.4 (D) netcdf-fortran/4.5.3 qhull/2020.1 (D) superlu-mt/3.1 (D) -------------------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/Core -------------------------------------------------------------------- angsd/0.933 canu/1.8 gatk/4.1.0.0 libarchive/3.4.3 ngstools/2019-06-24 picard/2.20.8 snpeff/5.0 arpack-ng/3.7.0 cuda/10.2.89 (L) gcc/9.3.0 magma/2.5.4 nlopt/2.6.2 pilon/1.22 stacks/2.55 bamtools/2.5.1 cuda/11.0.2 gmap-gsnap/2020-06-01 masurca/3.4.2 nvhpc/20.9 py-pyparsing/2.4.2 star/2.7.0e bartender/1.1 cuda/11.1.0 (D) gnuplot/5.2.8 metis/5.1.0 octave/5.2.0 pythia8/8303 subread/2.0.0 bayescan/2.1 cudnn/8.0.4.30-11.0-linux-x64 gsl/2.5 miniasm/2018-3-30 openbabel/3.0.0 python/3.8.6 (L) suite-sparse/5.7.2 bayestraits/3.0.2 eigen/3.3.8 hal/2.1 minimap2/2.14 openblas/0.3.10 qhull/2020.1 superlu-mt/3.1 bgc/1.03 exonerate/2.4.0 hdf5/1.10.7 mpich/3.3.2 openjdk/1.8.0_222-b10 qrupdate/1.1.2 superlu/5.2.1 blast-plus/2.9.0 express/1.5.2 hisat2/2.2.0 mvapich2/2.3.4 openjdk/11.0.2 (D) r/4.0.3 swig/4.0.2 (D) blat/35 fastjet/3.3.3 intel-mkl/2020.3.279 nccl/2.7.8-1 openmpi/4.0.5 rmcprofile/6.7.9 tabix/2013-12-16 boost/1.74.0 fastqc/0.11.7 intel/19.0.3 (L) netcdf-c/4.7.4 paml/4.9j root/6.20.08 trimmomatic/0.39 bowtie/1.3.0 fastx-toolkit/0.0.14 intel/20.0.3 (D) netcdf-cxx/4.2 pear/0.9.11 rsem/1.3.1 trinity/2.12.0 bowtie2/2.4.1 fftw/3.3.8 jellyfish/2.2.7 netcdf-fortran/4.5.3 perl/5.32.0 salmon/0.14.1 vcftools/0.1.15 bwa/0.7.17 freebayes/1.1.0 kallisto/0.46.2 ngmlr/0.2.7 phast/1.4 samtools/1.10 vesta/3.4.6 ------------------------------------------------------------------ /share/Apps/lusoft/share/spack/lmod/default/linux-centos8-x86_64/Core ------------------------------------------------------------------- anaconda3/2019.10 axel/2.16.1 graphviz/2.44.1 libxp/1.0.3 ncdu/1.13 pwgui/6.1 scons/3.1.2 tcl/8.6.10 valgrind/3.15.0 anaconda3/2020.07 (D) cmake/3.18.4 julia/1.5.2 motif/2.3.8 parallel/20200822 qcachegrind/20.08.0 screen/4.8.0 tk/8.6.10 vmd/1.9.3 aria2/1.35.0 ddd/3.3.12 krb5/1.18.2 nano/5.6.1 povray/3.7.0.10 rclone/1.53.3 swig/4.0.2 tmux/3.1b (L) --------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/applications/licensed --------------------------------------------------------------------------- abaqus/2019 cactus/1.2.3 comsol/5.4 guppy/3.1.5-gpu guppy/4.0.14 (D) mathematica/12.0 simmodeler/8.0 xds/feb2021 (D) ansys/20.2 cactus/1.3.0 (D) gaussian/g09 (D) guppy/3.1.5 gurobi/8.1.1 matlab/R2020a stata/15 xds/jan2020 ansys/21.1 (D) ccp4/7.0 gaussian/g16 guppy/4.0.14-gpu knitro/10.3.0 phenix/1.16 sysweld/2020.5 ---------------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/default ---------------------------------------------------------------------------------- cent7 gpu hawk hawkgpu (L) sol solgpu soltools (L) --------------------------------------------------------------- /share/Apps/share/Modules/lmod/linux-centos8-x86_64/py_venv/pythonapps/avx2 ---------------------------------------------------------------- all htseq/0.11.2-m5yqkbo matplotlib/3.3.3-dttvy5f (D) pandas/1.1.4-sdbbcxk pytorch/1.7.0-cuda seaborn/0.9.0-7xrppb4 biopython/1.73-aoor5vo jupyter/1.0.0-7ew3qb6 mdanalysis/1.0.0-tfngrft phonopy/1.10.0-iuaj2xc scikit-build/0.10.0-vww4qdm six/1.15.0-obed3ik cclib/1.5.post1-wv3arnb jupyterlab/2.2.7-ln6bc44 moltemplate/2.5.8-iclz2ic pip/20.2-ig2u7ky scikit-image/0.17.2-qhu4bnf tensorflow/2.4.1-cuda cython/0.29.21-trlkbz5 keras/2.2.4-cuda-rihsv3p multiqc/1.7-xi6ehki porechop/0.2.4-tgrse4w scikit-learn/0.23.2-nnpi22y theano/1.0.4-cuda-srh7rqj dask/2.16.0-hcixvbx lxml/4.5.2-2d2z266 numba/0.50.1-orbwfra pyparsing/2.4.2-yktitpw scikit-optimize/0.5.2-xngjbiq virtualenv/16.7.6-yh3ujrd horovod/0.21.3-cuda matplotlib/2.2.5-fimnl7l numpy/1.19.4-hyxetpw pyspark/3.0.1-f3qh2hw scipy/1.5.4-fbs3ffk virtualenvwrapper/4.8.2-vozt6md Where: L: Module is loaded D: Default Module Use "module spider" to find all possible modules and extensions. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

 

Most applications are optimized for AVX, AVX2 and AVX512 instruction set. The AVX2 compiled applications are loaded on the Sol head node and will not work on AVX based cpus i.e. debug partition. To use optimized applications, your should add the command source /etc/profile.d/zlmod.sh to your submit script before loading any modules. This script ensures that the MODULEPATH variable corresponds to the highest instruction set supported by the CPU rather than the default AVX2 compiled applications.

Software

Version

Type

Subject

module

GPU Support 

Software

Version

Type

Subject

module

GPU Support 

Abaqus

2019

Application

CFD

abaqus/2019

 

ADIOS2

2.6.0

Dev Tools

Library

adios2/2.6.0

 

almaBTE

1.3.2

Application

Materials

almabte/1.3.2

 

Anaconda3

2019.10 (Python 3.7.4)

Application

Various

anaconda3/2019.10

 

Anaconda3

2020.07 (Python 3.8.6)

Application

Various

anaconda3/2020.07

 

ANGSD

0.933

Application

Bioinformatics

angsd/0.933

 

Ansys

20.2

Application

CFD

ansys/20.2

 

aria2

1.35.0

Dev Tools

Utility

aria2/1.35.0

 

ARPACK

3.7.0

Dev Tools

Library

arpack-ng/3.7.0

 

AutoDock

4.2.6

Application

Biochemistry

autodock/4.2.6

 

Avogadro

 

Visualization

Chemistry

Open OnDemand App

 

Axel

2.16.1

Dev Tools

Utility

axel/2.16.1

 

BamTools

2.5.1

Application

Bioinformatics

bamtools/2.5.1

 

Bartender

1.1

Application

Bioinformatics

bartender/1.1

 

BayeScan

2.1

Application

Bioinformatics

bayescan/2.1

 

BayesTraits

3.0.2

Application

Bioinformatics

bayestraits/3.0.2

 

BerkeleyGW

3.0.1

Application

Materials

berkeleygw/3.0.1

 

bgc

1.03

Application

Bioinformatics

bgc/1.03

 

BLAST

2.9.0

Application

Bioinformatics

blast-plus/2.9.0

 

BLAT

35

Application

Bioinformatics

blat/35

 

Blender

 

Visualization

 

Open Ondemand App

 

BOOST

1.64.0

Dev Tools

Library

boost/1.74.0

 

Bowtie

1.3.0

Application

Bioinformatics

bowtie/1.3.0

 

Bowtie2

2.4.1

Application

Bioinformatics

bowtie2/2.4.1

 

BWA

0.7.17

Application

Bioinformatics

bwa/0.7.15

 

Cactus

1.3.0

Application

Bioinformatics

cactus/1.3.0

 

Canu

1.8

Application

Bioinformatics

canu/1.8

 

CCP4

7.0

Application

Chemistry

ccp4/7.0

 

CMake

3.18.4

Dev Tools

Utilities

cmake/3.18.4

 

COMSOL

5.4

Application

CFD

comsol/5.4

 

COMSOL

6.0

Application

CFD

comsol/6.0

 

CRYSTAL17

1.0.2

Application

Chemistry

crystal17/1.0.2

 

CUDA

10.2.89

Dev Tools

Programming

cuda/10.2.89

 

CUDA

11.0.2

Dev Tools

Programming

cuda/11.0.2

 

CUDA

11.1.0

Dev Tools

Programming

cuda/11.1.0

 

CUDNN

8.0.4.30

Dev Tools

Library

cudnn/8.0.4.30-11.0-linux-x64

 

DDD

3.3.12

Dev Tools

Utilities

ddd/3.3.12

 

DL_POLY_4

4.09

Application

Materials

dl_poly/4.09

 

DL_POLY_4

4.10

Application

Materials

dl_poly/4.10

 

DMTCP

2.6.0

Dev Tools

Utilities

dmtcp/2.6.0

 

Eigen

3.3.8

Dev Tools

Library

eigen/3.3.8

 

ESPResSo

4.0.2

Application

Materials

py-espresso/4.0.2

 

Exonerate

2.4.0

Application

Bioinformatics

exonerate/2.4.0

 

eXpress

1.5.2

Application

Bioinformatics

express/1.5.2

 

FastJet

3.3.3

Application

High Energy Physics

fastjet/3.3.3

 

Fastqc

0.11.7

Application

Bioinformatics

fastqc/0.11.7

 

FASTX-Toolkit

0.0.14

Application

Bioinformatics

fastx-toolkit/0.0.14

 

FFTW

3.3.8

Dev Tools