Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 40 Next »

What is Python?

Python is an interpreted, high-level and general-purpose programming language. Python's design philosophy emphasizes code readability with its notable use of significant indentation.

What is Anaconda?

Anaconda is a freemium open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment. Its package management system is conda.

Which Python versions are available?

Versionmodule nameNotes
3.8.6python/3.8.6optimized for avx, avx2 and avx512
3.8.6anaconda3/2020.07
3.7.4anaconda3/2019.10same as cent7 + anaconda/python_3.6
List of Python modules on Sol (switch to mpich for Hawk)
[2021-08-06 07:15.02] ~
[alp514.sol](1009): module av

-------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/mvapich2/2.3.4-ksbev3r/Core ---------------------------------------------------------
   adios2/2.6.0        hdf5/1.10.7         (D)    kahip/2.00            netcdf-cxx/4.2       (D)    openfoam/2006     petsc/3.14.1              scotch/6.0.10
   boost/1.74.0 (D)    horovod/0.21.3-cuda (D)    mumps/5.3.3           netcdf-fortran/4.5.3 (D)    paraview/5.8.1    py-mpi4py/3.0.3           superlu-dist/6.3.1
   fftw/3.3.8   (D)    hypre/2.20.0               netcdf-c/4.7.4 (D)    opencv/4.2.0                parmetis/4.0.3    pytorch/1.7.0-cuda (D)    tensorflow/2.4.1-cuda (D)

-------------------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/Core --------------------------------------------------------------------
   angsd/0.933          canu/1.8                             gatk/4.1.0.0                 libarchive/3.4.3            ngstools/2019-06-24          picard/2.20.8             snpeff/5.0
   arpack-ng/3.7.0      cuda/10.2.89                         gcc/9.3.0                    magma/2.5.4                 nlopt/2.6.2                  pilon/1.22                stacks/2.55
   bamtools/2.5.1       cuda/11.0.2                          gmap-gsnap/2020-06-01        masurca/3.4.2               nvhpc/20.9                   py-pyparsing/2.4.2        star/2.7.0e
   bartender/1.1        cuda/11.1.0                   (D)    gnuplot/5.2.8                metis/5.1.0                 octave/5.2.0                 pythia8/8303              subread/2.0.0
   bayescan/2.1         cudnn/8.0.4.30-11.0-linux-x64        gsl/2.5                      miniasm/2018-3-30           openbabel/3.0.0              python/3.8.6       (L)    suite-sparse/5.7.2
   bayestraits/3.0.2    eigen/3.3.8                          hal/2.1                      minimap2/2.14               openblas/0.3.10              qhull/2020.1              superlu-mt/3.1
   bgc/1.03             exonerate/2.4.0                      hdf5/1.10.7                  mpich/3.3.2                 openjdk/1.8.0_222-b10        qrupdate/1.1.2            superlu/5.2.1
   blast-plus/2.9.0     express/1.5.2                        hisat2/2.2.0                 mvapich2/2.3.4       (L)    openjdk/11.0.2        (D)    r/4.0.3                   swig/4.0.2         (D)
   blat/35              fastjet/3.3.3                        intel-mkl/2020.3.279         nccl/2.7.8-1                openmpi/4.0.5                rmcprofile/6.7.9          tabix/2013-12-16
   boost/1.74.0         fastqc/0.11.7                        intel/19.0.3                 netcdf-c/4.7.4              paml/4.9j                    root/6.20.08              trimmomatic/0.39
   bowtie/1.3.0         fastx-toolkit/0.0.14                 intel/20.0.3          (D)    netcdf-cxx/4.2              pear/0.9.11                  rsem/1.3.1                trinity/2.12.0
   bowtie2/2.4.1        fftw/3.3.8                           jellyfish/2.2.7              netcdf-fortran/4.5.3        perl/5.32.0                  salmon/0.14.1             vcftools/0.1.15
   bwa/0.7.17           freebayes/1.1.0                      kallisto/0.46.2              ngmlr/0.2.7                 phast/1.4                    samtools/1.10             vesta/3.4.6

------------------------------------------------------------------ /share/Apps/lusoft/share/spack/lmod/default/linux-centos8-x86_64/Core -------------------------------------------------------------------
   anaconda3/2019.10        axel/2.16.1     graphviz/2.44.1    libxp/1.0.3    ncdu/1.13            pwgui/6.1              scons/3.1.2     tcl/8.6.10    valgrind/3.15.0
   anaconda3/2020.07 (D)    cmake/3.18.4    julia/1.5.2        motif/2.3.8    parallel/20200822    qcachegrind/20.08.0    screen/4.8.0    tk/8.6.10     vmd/1.9.3
   aria2/1.35.0             ddd/3.3.12      krb5/1.18.2        nano/5.6.1     povray/3.7.0.10      rclone/1.53.3          swig/4.0.2      tmux/3.1b

--------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/applications/licensed ---------------------------------------------------------------------------
   abaqus/2019        cactus/1.2.3        comsol/5.4          guppy/3.1.5-gpu     guppy/4.0.14  (D)    mathematica/12.0    simmodeler/8.0    xds/feb2021 (D)
   ansys/20.2         cactus/1.3.0 (D)    gaussian/g09 (D)    guppy/3.1.5         gurobi/8.1.1         matlab/R2020a       stata/15          xds/jan2020
   ansys/21.1  (D)    ccp4/7.0            gaussian/g16        guppy/4.0.14-gpu    knitro/10.3.0        phenix/1.16         sysweld/2020.5

---------------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/default ----------------------------------------------------------------------------------
   cent7    gpu    hawk    hawkgpu    sol    solgpu    soltools

--------------------------------------------------------------- /share/Apps/share/Modules/lmod/linux-centos8-x86_64/py_venv/pythonapps/avx2 ----------------------------------------------------------------
   all                        htseq/0.11.2-m5yqkbo        matplotlib/3.3.3-dttvy5f  (D)    pandas/1.1.4-sdbbcxk       pytorch/1.7.0-cuda               seaborn/0.9.0-7xrppb4
   biopython/1.73-aoor5vo     jupyter/1.0.0-7ew3qb6       mdanalysis/1.0.0-tfngrft         phonopy/1.10.0-iuaj2xc     scikit-build/0.10.0-vww4qdm      six/1.15.0-obed3ik
   cclib/1.5.post1-wv3arnb    jupyterlab/2.2.7-ln6bc44    moltemplate/2.5.8-iclz2ic        pip/20.2-ig2u7ky           scikit-image/0.17.2-qhu4bnf      tensorflow/2.4.1-cuda
   cython/0.29.21-trlkbz5     keras/2.2.4-cuda-rihsv3p    multiqc/1.7-xi6ehki              porechop/0.2.4-tgrse4w     scikit-learn/0.23.2-nnpi22y      theano/1.0.4-cuda-srh7rqj
   dask/2.16.0-hcixvbx        lxml/4.5.2-2d2z266          numba/0.50.1-orbwfra             pyparsing/2.4.2-yktitpw    scikit-optimize/0.5.2-xngjbiq    virtualenv/16.7.6-yh3ujrd
   horovod/0.21.3-cuda        matplotlib/2.2.5-fimnl7l    numpy/1.19.4-hyxetpw             pyspark/3.0.1-f3qh2hw      scipy/1.5.4-fbs3ffk              virtualenvwrapper/4.8.2-vozt6md

  Where:
   L:  Module is loaded
   D:  Default Module

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


List of Conda environments
[2021-08-06 07:18.07] ~
[alp514.sol](1012): module av

-------------------------------------------------------------------- /share/Apps/lusoft/share/spack/lmod/avx2/linux-centos8-x86_64/Core --------------------------------------------------------------------
   angsd/0.933          canu/1.8                             gatk/4.1.0.0                 libarchive/3.4.3        ngstools/2019-06-24          picard/2.20.8         snpeff/5.0
   arpack-ng/3.7.0      cuda/10.2.89                         gcc/9.3.0                    magma/2.5.4             nlopt/2.6.2                  pilon/1.22            stacks/2.55
   bamtools/2.5.1       cuda/11.0.2                          gmap-gsnap/2020-06-01        masurca/3.4.2           nvhpc/20.9                   py-pyparsing/2.4.2    star/2.7.0e
   bartender/1.1        cuda/11.1.0                   (D)    gnuplot/5.2.8                metis/5.1.0             octave/5.2.0                 pythia8/8303          subread/2.0.0
   bayescan/2.1         cudnn/8.0.4.30-11.0-linux-x64        gsl/2.5                      miniasm/2018-3-30       openbabel/3.0.0              python/3.8.6          suite-sparse/5.7.2
   bayestraits/3.0.2    eigen/3.3.8                          hal/2.1                      minimap2/2.14           openblas/0.3.10              qhull/2020.1          superlu-mt/3.1
   bgc/1.03             exonerate/2.4.0                      hdf5/1.10.7                  mpich/3.3.2             openjdk/1.8.0_222-b10        qrupdate/1.1.2        superlu/5.2.1
   blast-plus/2.9.0     express/1.5.2                        hisat2/2.2.0                 mvapich2/2.3.4          openjdk/11.0.2        (D)    r/4.0.3               swig/4.0.2         (D)
   blat/35              fastjet/3.3.3                        intel-mkl/2020.3.279         nccl/2.7.8-1            openmpi/4.0.5                rmcprofile/6.7.9      tabix/2013-12-16
   boost/1.74.0         fastqc/0.11.7                        intel/19.0.3                 netcdf-c/4.7.4          paml/4.9j                    root/6.20.08          trimmomatic/0.39
   bowtie/1.3.0         fastx-toolkit/0.0.14                 intel/20.0.3          (D)    netcdf-cxx/4.2          pear/0.9.11                  rsem/1.3.1            trinity/2.12.0
   bowtie2/2.4.1        fftw/3.3.8                           jellyfish/2.2.7              netcdf-fortran/4.5.3    perl/5.32.0                  salmon/0.14.1         vcftools/0.1.15
   bwa/0.7.17           freebayes/1.1.0                      kallisto/0.46.2              ngmlr/0.2.7             phast/1.4                    samtools/1.10         vesta/3.4.6

------------------------------------------------------------------ /share/Apps/lusoft/share/spack/lmod/default/linux-centos8-x86_64/Core -------------------------------------------------------------------
   anaconda3/2019.10          axel/2.16.1     graphviz/2.44.1    libxp/1.0.3    ncdu/1.13            pwgui/6.1              scons/3.1.2     tcl/8.6.10    valgrind/3.15.0
   anaconda3/2020.07 (L,D)    cmake/3.18.4    julia/1.5.2        motif/2.3.8    parallel/20200822    qcachegrind/20.08.0    screen/4.8.0    tk/8.6.10     vmd/1.9.3
   aria2/1.35.0               ddd/3.3.12      krb5/1.18.2        nano/5.6.1     povray/3.7.0.10      rclone/1.53.3          swig/4.0.2      tmux/3.1b

--------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/applications/licensed ---------------------------------------------------------------------------
   abaqus/2019        cactus/1.2.3        comsol/5.4          guppy/3.1.5-gpu     guppy/4.0.14  (D)    mathematica/12.0    simmodeler/8.0    xds/feb2021 (D)
   ansys/20.2         cactus/1.3.0 (D)    gaussian/g09 (D)    guppy/3.1.5         gurobi/8.1.1         matlab/R2020a       stata/15          xds/jan2020
   ansys/21.1  (D)    ccp4/7.0            gaussian/g16        guppy/4.0.14-gpu    knitro/10.3.0        phenix/1.16         sysweld/2020.5

---------------------------------------------------------------------------------- /share/Apps/share/Modules/lmod/default ----------------------------------------------------------------------------------
   cent7    gpu    hawk    hawkgpu    sol    solgpu    soltools

------------------------------------------------------------------- /share/Apps/share/Modules/lmod/linux-centos8-x86_64/py_venv/2020.07 --------------------------------------------------------------------
   conda/biofluids    conda/bioinformatics    conda/biostats    conda/chem    conda/mldl    conda/nlp    conda/r (D)

  Where:
   L:  Module is loaded
   D:  Default Module

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".


Seminar Slides

What's the difference between python and anaconda3 modules?

Python can be built by compiling the source code or by installing binaries from the Anaconda Python Distribution. Compiled python i.e. python/3.8.6 is built using GNU compiler and optimized for the underlying CPU architecture. Anaconda Python is most likely unoptimized or built for generic x86_64  architecture and can be slower than compiled python.

If you use tensorflow from one of the anaconda modules, you might see the following warning "Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA". SSE4/AVX provide double/quadruple the FLOPs compared to generic x86_64 while AVX2/AVX512 double/quadruples the AVX FLOPS at the expense of cpu speed. If you use python packages from the anaconda3 modules, you are most likely using unoptimized binaries. To use MPI enabled versions of Tensorflow, PyTorch, Py4MPI and Horovod, unload the intel modules and reload mvapich2/mpich/openmpi.


 List of Python packages (Updated on 12/10/21)

The python/3.8.6 module is loaded by default. To use packages listed under that module, run the command module load all. To use MPI enabled packages, mpi4py, horovod, pytorch and tensorflow, you need to unload the intel module if loaded and load one of the mpi modules - mpich, mvapich2 or openmpi.

For others, first load the anaconda3/2020.07 module using module load anaconda3/2020.07 followed by any one of the conda modules for e.g. module load conda/chem.

You can load both at the same time for eg. module load python/3.8.6 all OR module load anaconda3/2020.07 conda/bioinformatics. If you get an error, load them sequentially.

The conda/spark module was built for use with Apache Spark and provides R packages such as sparklyr

Package

anaconda3/2020.07

conda/biofluids

conda/bioinformatics

conda/biostats

conda/chem

conda/mldl

conda/nlp

conda/spark

python/3.8.6+all

absl-py




0.13.0


0.10.0



0.10.0

aiohttp




3.7.4.post0


3.7.3




alabaster

0.7.12







0.7.12


amset





0.3.0





anaconda-client

1.7.2









anaconda-navigator

1.9.12









anaconda-project

0.8.3









anyio


3.1.0

3.1.0

3.3.0

3.1.0

3.1.0

3.1.0

3.4.0


APScheduler





3.6.3





argh

0.26.2







0.26.2


argon2-cffi


20.1.0

20.1.0

20.1.0

20.1.0

20.1.0

20.1.0

21.1.0

20.1.0

arrow








1.2.1


arviz




0.11.2






asciinema









2.0.2

ase





3.20.1





asn1crypto

1.3.0









astor






0.8.1




astroid

2.4.2







2.9.0


astropy

4.0.1.post1









astunparse





1.6.3

1.6.3



1.6.3

async-generator


1.10

1.10

1.10

1.10

1.10

1.10

1.10

1.10

async-timeout




3.0.1


3.0.1




atomicwrites

1.4.0







1.4.0


attrs

19.3.0

20.3.0

20.3.0

21.2.0

20.3.0

20.3.0

20.3.0

21.2.0

20.3.0

autopep8

1.5.3







1.6.0


Babel

2.8.0

2.9.1

2.9.1

2.9.1

2.9.1

2.9.1

2.9.1

2.9.1

2.7.0

backcall

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.1.0

backports.functools-lru-cache

1.6.4

1.6.1

1.6.1

1.6.4

1.6.1

1.6.1

1.6.1

1.6.4


backports.shutil-get-terminal-size

1.0.0









backports.tempfile

1.0









backports.weakref

1.0.post1









barcode-splitter



0.18.6







beautifulsoup4

4.9.1




4.9.3





binaryornot








0.4.4


biopython





1.78




1.73

bitarray

1.4.0









bkcharts

0.2









black








21.11b1


bleach

3.1.5

3.2.1

3.2.1

4.0.0

3.2.1

3.2.1

3.2.1

4.1.0

3.1.0

blinker






1.4




bokeh

2.1.1









boltons





21.0.0





BoltzTraP2





20.7.1





boto

2.49.0









Bottleneck

1.3.2








1.2.1

brotlipy

0.7.0

0.7.0

0.7.0

0.7.0

0.7.0

0.7.0

0.7.0

0.7.0


cached-property





1.5.1




1.5.2

cachetools




4.2.2


4.1.1




cclib





1.6.4




1.5.post1

certifi

2020.6.20

2021.5.30

2021.5.30

2021.5.30

2021.5.30

2021.10.8

2021.5.30

2021.10.8

2020.6.20

cffi

1.14.0

1.14.3

1.14.3

1.14.6

1.14.3

1.14.3

1.14.3

1.15.0

1.14.3

cftime




1.5.0

1.3.0





chardet

3.0.4

3.0.4

3.0.4

4.0.0

3.0.4

3.0.4

3.0.4

4.0.0

3.0.4

charset-normalizer




2.0.0




2.0.9


click

7.1.2


7.1.2

8.0.1

7.1.2

7.1.2


8.0.3

7.1.2

click-plugins




1.1.1






cligj




0.7.2






cloudpickle

1.5.0




1.6.0

1.6.0


2.0.0

1.2.1

clyent

1.2.2









colorama

0.4.3

0.4.4

0.4.4

0.4.4

0.4.4

0.4.4

0.4.4

0.4.4

0.4.4

coloredlogs



15.0







colormath



3.0.0







commonmark



0.9.1







conda

4.10.3









conda-build

3.18.11









conda-package-handling

1.7.3









conda-verify

3.4.2









configobj









5.0.6

contextlib2

0.6.0.post1









cookiecutter








1.6.0


covidcast




0.1.5






cryptography

2.9.2

3.2.1

3.2.1

3.4.7

3.2.1

3.2.1

3.2.1

36.0.0

3.3.2

cycler

0.10.0

0.10.0

0.10.0

0.10.0

0.10.0

0.10.0

0.10.0

0.11.0

0.10.0

Cython

0.29.21



0.29.17

0.29.21




0.29.21

cytoolz

0.10.1





0.11.0




dask

2.20.0




2021.6.2

2.30.0



2.16.0

dataclasses








0.8


debugpy




1.4.1




1.5.1


decorator

4.4.2

4.4.2

4.4.2

5.0.9

4.4.2

4.4.2

4.4.2

5.1.0

4.4.2

defusedxml

0.6.0

0.6.0

0.6.0

0.7.1

0.6.0

0.6.0

0.6.0

0.7.1

0.6.0

delphi-epidata




0.2.6






descartes




1.1.0






diff-match-patch

20200713







20200713


dill




0.3.4

0.3.4




0.3.1.dev0

diskcache









4.1.0

distributed

2.20.0








2.10.0

docutils

0.16







0.16


eliot





1.13.0





entrypoints

0.3

0.3

0.3

0.3

0.3

0.3

0.3

0.3

0.3

epiweeks




2.1.2






et-xmlfile

1.0.1









fastcache

1.1.0









fastprogress




1.0.0






filelock

3.0.12



3.0.12






findspark








1.4.2


Fiona




1.8.20






flake8

3.8.3







4.0.1


Flask

1.1.2




1.1.2





flatbuffers




2.0





20210514195623

fonttools








4.28.3


fsspec

0.7.4




2021.6.0




0.7.3

future

0.18.2


0.18.2


0.18.2



0.18.2

0.18.2

gast






0.3.3



0.3.3

geomm





0.2.0





geopandas




0.9.0






gevent

20.6.2









gitdb

4.0.7

4.0.7

4.0.7

4.0.7

4.0.7

4.0.7

4.0.7

4.0.9

4.0.7

GitPython

3.1.18

3.1.17

3.1.17

3.1.18

3.1.17

3.1.17

3.1.17

3.1.24

3.1.18

glob2

0.7









globus-cli









2.1.0

globus-sdk









2.0.1

gmpy2

2.0.8




2.1.0b1





google-auth






1.23.0




google-auth-oauthlib






0.4.1




google-pasta






0.2.0



0.2.0

greenlet

0.4.16









GridDataFormats





0.5.0




0.5.0

grpcio






1.33.2



1.32.0

gsd





2.4.0




1.9.3

h5py

2.10.0




2.10.0

2.10.0



2.10.0

HeapDict

1.0.1








1.0.1

horovod









0.21.3

html5lib

1.1









HTSeq









0.11.2

humanfriendly



9.1







idna

2.10

2.10

2.10

3.1

2.10

2.10

2.10

3.1

2.8

imagecodecs






2020.5.30




imageio

2.9.0



2.9.0


2.9.0



2.5.0

imageio-ffmpeg




0.4.4






imagesize

1.2.0







1.3.0


importlib-metadata

1.7.0

3.0.0

3.0.0

4.6.3

3.0.0

3.0.0

3.0.0

4.8.2


importlib-resources








5.4.0


inflection








0.5.1


interpolation





2.1.6





intervaltree

3.0.2







3.0.2


ipykernel

5.3.2

5.3.4

5.3.4

6.0.3

5.3.4

5.3.4

5.3.4

6.6.0

5.3.4

ipython

7.16.1

7.19.0

7.19.0

7.26.0

7.19.0

7.19.0

7.19.0

7.30.1

7.18.1

ipython-genutils

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

0.2.0

ipywidgets

7.5.1

7.5.1

7.5.1

7.6.3

7.5.1

7.5.1

7.5.1

7.6.5

7.5.1

isort

4.3.21







5.10.1


itsdangerous

1.1.0




1.1.0





jax




0.2.17






jaxlib




0.1.67






jdcal

1.4.1









jedi

0.17.1

0.17.2

0.17.2

0.18.0

0.17.2

0.17.2

0.17.2

0.18.1

0.13.3

jeepney

0.4.3







0.7.1


jellyfish








0.8.9


Jinja2

2.11.2

2.11.2

2.11.2

3.0.1

2.11.2

2.11.2

2.11.2

3.0.3

2.10.3

jinja2-time








0.2.0


jmespath









0.10.0

joblib

0.16.0



1.0.1

0.17.0

0.17.0



0.14.0

json5

0.9.5

0.9.5

0.9.5

0.9.5

0.9.5

0.9.5

0.9.5

0.9.5

0.9.4

jsonschema

3.2.0

3.2.0

3.2.0

3.2.0

3.2.0

3.2.0

3.2.0

4.2.1

3.2.0

jupyter

1.0.0



1.0.0




1.0.0

1.0.0

jupyter-client

6.1.6

6.1.7

6.1.7

6.1.12

6.1.7

6.1.7

6.1.7

7.1.0

6.1.7

jupyter-console

6.1.0

6.2.0

6.2.0

6.4.0

6.2.0

6.2.0

6.2.0

6.4.0

6.1.0

jupyter-core

4.6.3

4.7.0

4.7.0

4.7.1

4.7.0

4.7.0

4.7.0

4.9.1

4.6.3

jupyter-server


1.8.0

1.8.0

1.10.2

1.8.0

1.8.0

1.8.0

1.13.1


jupyter-server-mathjax


0.2.3

0.2.3

0.2.3

0.2.3

0.2.3

0.2.3

0.2.3


jupyterlab

2.1.5

3.0.16

3.0.16

3.1.1

3.0.16

3.0.16

3.0.16

3.2.4

2.2.7

jupyterlab-git

0.24.0

0.30.1

0.30.1

0.31.0

0.30.1

0.30.1

0.30.1

0.34.0

0.24.0

jupyterlab-pygments


0.1.2

0.1.2

0.1.2

0.1.2

0.1.2

0.1.2

0.1.2

0.1.1

jupyterlab-server

1.2.0

2.6.0

2.6.0

2.6.2

2.6.0

2.6.0

2.6.0

2.8.2

1.2.0

jupyterlab-widgets




1.0.0




1.0.2


Keras






2.4.3



2.2.4

Keras-Applications









1.0.8

Keras-Preprocessing






1.1.0



1.1.2

keyring

21.2.1







23.4.0


kiwisolver

1.2.0

1.3.1

1.3.1

1.3.1

1.3.1

1.3.1

1.3.1

1.3.2

1.1.0

klepto




0.2.1






latexcodec





2.0.1





lazy-object-proxy

1.4.3







1.6.0


libarchive-c

2.9









llvmlite

0.33.0+1.g022ab0f

0.34.0



0.35.0




0.34.0

locket

0.2.0




0.2.1




0.2.0

loguru





0.5.3





lxml

4.5.2








4.5.2

lzstring



1.0.4







Mako




1.1.4


1.1.3



1.0.4

Markdown



3.3.4



3.3.3




MarkupSafe

1.1.1

1.1.1

1.1.1

2.0.1

1.1.1

1.1.1

1.1.1

2.0.1

1.1.1

matplotlib

3.2.2

3.3.3

3.3.3

3.4.2

3.3.2

3.3.3

3.3.3

3.5.0

3.3.3

matplotlib-inline




0.1.2




0.1.3


mccabe

0.6.1







0.6.1


MDAnalysis





1.0.0




1.0.0

mdtraj





1.9.6





memory-profiler





0.58.0





mistune

0.8.4

0.8.4

0.8.4

0.8.4

0.8.4

0.8.4

0.8.4

0.8.4

0.8.4

mkl-fft

1.1.0









mkl-random

1.1.1









mkl-service

2.3.0



2.4.0


2.3.0




mmtf-python





1.1.2




1.1.2

mock

4.0.2








3.0.5

molmod





1.4.8





moltemplate









2.5.8

monty





4.0.2





more-itertools

8.4.0









mpi4py









3.0.3

mpmath

1.1.0



1.2.1

1.1.0





msgpack

1.0.0




1.0.0




1.0.0

multidict




5.1.0


4.7.5




multipledispatch

0.6.0









multiprocessing-logging





0.3.1





multiqc



1.10







munch




2.5.0






munkres








1.1.4


mypy-extensions








0.4.3


mystic




0.3.8






navigator-updater

0.2.1









nbclassic


0.3.1

0.3.1

0.3.1

0.3.1

0.3.1

0.3.1

0.3.4


nbclient


0.5.1

0.5.1

0.5.3

0.5.1

0.5.1

0.5.1

0.5.9

0.5.0

nbconvert

5.6.1

6.0.7

6.0.7

6.1.0

6.0.7

6.0.7

6.0.7

6.3.0

6.0.1

nbdime

2.1.0

3.1.0

3.1.0

3.1.0

3.1.0

3.1.0

3.1.0

3.1.1

2.1.0

nbformat

5.0.7

5.0.8

5.0.8

5.1.3

5.0.8

5.0.8

5.0.8

5.1.3

5.0.7

ndim





0.1.4





nest-asyncio


1.4.3

1.4.3

1.5.1

1.4.3

1.4.3

1.4.3

1.5.4

1.4.0

netCDF4




1.5.7

1.5.4





networkx

2.4


2.5


2.5.1

2.5



2.4

nltk

3.5






3.4.4



nose

1.3.7








1.3.7

notebook

6.0.3

6.1.5

6.1.5

6.4.0

6.1.5

6.1.5

6.1.5

6.4.6

6.1.4

numba

0.50.1

0.51.2



0.52.0




0.50.1

numexpr

2.7.1








2.7.0

numpy

1.18.5

1.19.4

1.19.4

1.19.5

1.19.4

1.19.4

1.19.4

1.21.4

1.19.4

numpydoc

1.1.0







1.1.0


oauthlib






3.0.1




olefile

0.46

0.46

0.46

0.46

0.46

0.46

0.46

0.46


OpenMM





7.5.1





openmm-systems





0.0.0





openpyxl

3.0.4









opt-einsum




3.3.0


0+untagged.61.gd905544.dirty



3.3.0

orthopy





0.8.4





packaging

20.4

20.4

20.4

21.0

20.4

20.4

20.4

21.3

21.0

palettable





3.3.0





pandas

1.0.5

1.1.4

1.1.4

1.3.1

1.1.4

1.1.4

1.1.4

1.3.4

1.1.4

pandocfilters

1.4.2

1.4.2

1.4.2

1.4.2

1.4.2

1.4.2

1.4.2

1.5.0

1.4.2

parso

0.7.0

0.7.1

0.7.1

0.8.2

0.7.1

0.7.1

0.7.1

0.8.3

0.6.1

partd

1.1.0




1.2.0




1.1.0

path

13.1.0









pathlib2

2.3.5









pathspec








0.9.0


pathtools

0.1.2









patsy

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.2


pbr









2.0.0

pep8

1.7.1









periodictable





1.5.2





petastorm









0.9.8

pexpect

4.8.0

4.8.0

4.8.0

4.8.0

4.8.0

4.8.0

4.8.0

4.8.0

4.7.0

phono3py





1.21.0





phonopy





2.8.1




1.10.0

pickleshare

0.7.5

0.7.5

0.7.5

0.7.5

0.7.5

0.7.5

0.7.5

0.7.5

0.7.5

Pillow

7.2.0

8.0.1

8.0.1

8.3.1

8.0.1

8.0.1

8.0.1

8.4.0

7.2.0

Pint





0.17





pip

20.1.1

20.2.4

20.2.4

21.2.2

20.2.4

20.2.4

20.2.4

21.3.1

20.2

pkginfo

1.5.0.1









platformdirs








2.3.0


plotly





4.14.1





pluggy

0.13.1







1.0.0


ply

3.11









pmdarima




1.8.1






porechop



0.2.4






0.2.4

pox




0.3.0






poyo








0.5.0


prometheus-client

0.8.0

0.9.0

0.9.0

0.11.0

0.9.0

0.9.0

0.9.0

0.12.0

0.7.1

prompt-toolkit

3.0.5

3.0.8

3.0.8

3.0.19

3.0.8

3.0.8

3.0.8

3.0.24

2.0.9

protobuf






3.13.0



3.12.2

psutil

5.7.0




5.8.0



5.8.0

5.7.2

ptyprocess

0.6.0

0.6.0

0.6.0

0.7.0

0.6.0

0.6.0

0.6.0

0.7.0

0.6.0

py

1.9.0








1.8.0

py4j






0.10.9.2


0.10.9.2

0.10.9

pyarrow






2.0.0


5.0.0

0.17.1

pyasn1






0.4.8




pyasn1-modules






0.2.7




pybind11









2.5.0

pybtex





0.23.0





pycairo





1.20.0





pycodestyle

2.6.0







2.8.0


pycosat

0.6.3









pycparser

2.20

2.20

2.20

2.20

2.20

2.20

2.20

2.21

2.20

pycurl

7.43.0.5









pydocstyle

5.0.2







6.1.1


pyFFTW





0.12.0





pyflakes

2.2.0







2.4.0


Pygments

2.6.1

2.7.2

2.7.2

2.9.0

2.7.2

2.7.2

2.7.2

2.10.0

2.6.1

pygpu




0.7.6


0.7.6



0.7.6

PyJWT






1.7.1



1.7.1

pylint

2.5.3







2.12.2


pyls-spyder








0.4.0


pymatgen





2020.10.20





pymc3




3.11.2






Pympler





0.9





pyodbc

4.0.0-unsupported









pyOpenSSL

19.1.0

19.1.0

19.1.0

20.0.1

19.1.0

19.1.0

19.1.0

21.0.0


pyparsing

2.4.7

2.4.7

2.4.7

2.4.7

2.4.7

2.4.7

2.4.7

3.0.6

2.4.2

pyproj




3.1.0






PyQt5


5.12.3

5.12.3

5.12.3

5.12.3

5.12.3

5.12.3

5.12.3

5.13.1

PyQt5_sip




4.19.18




4.19.18


PyQt5-sip


4.19.18

4.19.18


4.19.18

4.19.18

4.19.18


4.19.21

PyQtChart


5.12

5.12

5.12

5.12

5.12

5.12

5.12


PyQtWebEngine


5.12.1

5.12.1

5.12.1

5.12.1

5.12.1

5.12.1

5.12.1


pyrsistent

0.16.0

0.17.3

0.17.3

0.17.3

0.17.3

0.17.3

0.17.3

0.18.0

0.15.7

pysam









0.15.2

PySocks

1.7.1

1.7.1

1.7.1

1.7.1

1.7.1

1.7.1

1.7.1

1.7.1


pyspark






3.2.0


3.2.0

3.0.1

pytest

5.4.3









python-dateutil

2.8.1

2.8.1

2.8.1

2.8.2

2.8.1

2.8.1

2.8.1

2.8.2

2.8.0

python-jsonrpc-server

0.3.4









python-language-server

0.34.1









python-lsp-black








1.0.1


python-lsp-jsonrpc








1.0.0


python-lsp-server








1.3.2


pytz

2020.1

2020.4

2020.4

2021.1

2020.4

2020.4

2020.4

2021.3

2020.1

PyWavelets

1.1.1





1.1.1



1.1.1

pyxdg

0.26







0.27


PyYAML

5.3.1


5.4.1


5.3.1

5.3.1


6.0

5.3.1

pyzmq

19.0.1

20.0.0

20.0.0

22.1.0

20.0.0

20.0.0

20.0.0

22.3.0

18.1.0

QDarkStyle

2.8.1







3.0.2


qstylizer








0.2.1


QtAwesome

0.7.2







1.1.1


qtconsole

4.7.5

4.7.7

4.7.7

5.1.1

4.7.7

4.7.7

4.7.7

5.2.1


QtPy

1.9.0

1.9.0

1.9.0

1.9.0

1.9.0

1.9.0

1.9.0

1.11.3


quadpy





0.16.2





regex

2020.6.8







2021.11.10


requests

2.24.0

2.25.0

2.25.0

2.26.0

2.25.0

2.25.0

2.25.0

2.26.0

2.24.0

requests-oauthlib






1.3.0




requests-unixsocket




0.2.0






retrying





1.3.3





rich



9.13.0







rope

0.17.0







0.22.0


rsa






4.6




Rtree

0.9.4







0.9.7


ruamel-yaml

0.15.87









ruamel.yaml





0.16.12





ruamel.yaml.clib





0.2.2





scikit-build









0.10.0

scikit-image

0.16.2





0.17.2




scikit-learn

0.23.1



0.24.2

0.23.2

0.23.2



0.23.2

scikit-optimize









0.5.2

scipy

1.5.0

1.5.3

1.5.3

1.7.0

1.5.3

1.5.3

1.5.3

1.7.3

1.5.4

seaborn

0.10.1

0.11.0

0.11.0

0.11.1

0.11.0

0.11.0

0.11.0

0.11.2

0.9.0

SecretStorage

3.1.2







3.3.1


seekpath





1.9.5





semver




2.13.0






Send2Trash

1.5.0

1.5.0

1.5.0

1.7.1

1.5.0

1.5.0

1.5.0

1.8.0

1.5.0

setuptools

49.2.0.post20200714

49.6.0.post20201009

49.6.0.post20201009

49.6.0.post20210108

49.6.0.post20201009

49.6.0.post20201009

49.6.0.post20201009

59.4.0

50.3.2

Shapely




1.7.1






simplegeneric

0.8.1









simplejson



3.17.2







singledispatch

3.4.0.3









sip

4.19.13









six

1.15.0

1.15.0

1.15.0

1.16.0

1.15.0

1.15.0

1.15.0

1.16.0

1.15.0

smmap

3.0.5

3.0.5

3.0.5

3.0.5

3.0.5

3.0.5

3.0.5

3.0.5

4.0.0

sniffio


1.2.0

1.2.0

1.2.0

1.2.0

1.2.0

1.2.0

1.2.0


snowballstemmer

2.0.0







2.2.0


sortedcollections

1.2.1









sortedcontainers

2.2.2







2.4.0

2.1.0

soupsieve

2.0.1




2.0.1





spectra



0.0.11







spglib





1.16.0





Sphinx

3.1.2







4.3.1


sphinxcontrib-applehelp

1.0.2







1.0.2


sphinxcontrib-devhelp

1.0.2







1.0.2


sphinxcontrib-htmlhelp

1.0.3







2.0.0


sphinxcontrib-jsmath

1.0.1







1.0.1


sphinxcontrib-qthelp

1.0.3







1.0.3


sphinxcontrib-serializinghtml

1.1.4







1.1.5


sphinxcontrib-websupport

1.2.3









spyder

4.1.4







5.2.0


spyder-kernels

1.9.2







2.2.0


SQLAlchemy

1.3.18









statsmodels

0.11.1

0.12.1

0.12.1

0.11.1

0.12.1

0.12.1

0.12.1

0.13.1


stevedore









1.28.0

sumo





2.1.0





sympy

1.6.1



1.9

1.7.1





tables

3.6.1









tabulate





0.8.7





tamkin





1.2.6





tblib

1.6.0








1.6.0

Tempita





0.5.3.dev0





tenacity




8.0.1






tensorboard






2.4.0




tensorboard-plugin-wit






1.7.0




tensorflow






2.2.0



2.4.1

tensorflow-estimator






2.2.0




termcolor






1.1.0



1.1.0

terminado

0.8.3

0.9.1

0.9.1

0.10.1

0.9.1

0.9.1

0.9.1

0.12.1

0.8.3

testpath

0.4.4

0.4.4

0.4.4

0.5.0

0.4.4

0.4.4

0.4.4

0.5.0

0.4.2

textdistance








4.2.2


Theano






1.0.5



1.0.4

Theano-PyMC




1.1.2






threadpoolctl

2.1.0



2.2.0

2.1.0

2.1.0



2.0.0

three-merge








0.1.1


tifffile






2020.11.18



2020.10.1

tinycss2








1.1.1


toml

0.10.1







0.10.2


tomli








1.2.2


toolz

0.10.0




0.11.1

0.11.1



0.9.0

torch






1.4.0



1.7.0

tornado

6.0.4

6.1

6.1

6.1

6.1

6.1

6.1

6.1

6.0.3

tqdm

4.47.0



4.62.0

4.51.0




4.45.0

traitlets

4.3.3

5.0.5

5.0.5

5.0.5

5.0.5

5.0.5

5.0.5

5.1.1

5.0.4

typed-ast








1.5.1


typing_extensions








4.0.1


typing-extensions

3.7.4.2

3.10.0.0

3.7.4.3

3.10.0.0

3.10.0.0

3.7.4.3

3.10.0.0


3.7.4

tzlocal





2.1





ujson

1.35







4.2.0


uncertainties





3.1.5





unicodecsv

0.14.1









unicodedata2








13.0.0.post2


urllib3

1.25.9

1.25.11

1.25.11

1.26.6

1.25.11

1.25.11

1.25.11

1.26.7

1.25.6

virtualenv









16.7.6

virtualenv-clone









0.2.6

virtualenvwrapper









4.8.2

vtk





9.0.1





watchdog

0.10.3







2.1.6


wcwidth

0.2.5

0.2.5

0.2.5

0.2.5

0.2.5

0.2.5

0.2.5

0.2.5

0.1.7

webencodings

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

0.5.1

websocket-client


0.57.0

0.57.0

0.57.0

0.57.0

0.57.0

0.57.0

1.2.3


wepy





1.1.0





Werkzeug

1.0.1




1.0.1

1.0.1




wheel

0.34.2

0.35.1

0.35.1

0.36.2

0.35.1

0.35.1

0.35.1

0.37.0

0.36.2

whichcraft








0.6.1


widgetsnbextension

3.5.1

3.5.1

3.5.1

3.5.1

3.5.1

3.5.1

3.5.1

3.5.2

3.5.1

wrapt

1.11.2





1.12.1


1.13.3

1.12.1

wurlitzer

2.0.1







3.0.2


xarray




0.19.0






xlrd

1.2.0









XlsxWriter

1.2.9









xlwt

1.3.0









xmltodict

0.12.0









yapf

0.30.0







0.31.0


yarl




1.6.3


1.6.3




zict

2.0.0








1.0.0

zipp

3.1.0

3.4.0

3.4.0

3.5.0

3.4.0

3.4.0

3.4.0

3.6.0


zope.event

4.4









zope.interface

4.7.1




5.4.0






Which module should I use?

Please check if the package you want to use is available by running the command conda search <packagename> before emailing Research Computing staff to install a package. You can always create a new conda environment where you can install and maintain a set of python packages that are most useful for your research.

How to search for conda packages

conda list: see list of packages installed. 
conda search modulename: see if modulename is available for conda to install


How to install python packages in my own directories

Let's suppose that you want to install a python package named test123 locally. You need to create a conda environment and install test123 (and any other packages) to that environment.

conda create --name mycondaenv test123

If you omit test123, then an empty environment is created.

Useful commands:

conda env list : To see list of different conda environments. Default environment has a star (*) in the second field 
conda activate mycondaenv: To switch to the mycondaenv environment and use the test123 package 
conda deactivate mycondaenv: To switch to the root environment


If you need to install additional python packages within this environment, then use conda to install the packages within that environment

conda activate mycondaenv
conda install mypythonpackage

For more information, see the Conda User Guide

How do I use Jupyter Notebooks on Sol

Jupyter Notebooks are available using the Open OnDemand portal as well as the command line in interactive sessions .

How can I built my own python software stack using SPACK

Download  SPACK to your home directory (If using another directory, replace $HOME)

Download and install from github
git clone https://github.com/spack/spack
To use system spack as an upstream install
cd ${HOME}/spack
git checkout releases/v0.16
cp /share/Apps/usr/etc/spack/* ${HOME}/spack/etc/spack

Default target is x86_64. To build applications optimized for underlying cpu replace x86_64 on line 97 of packages.yaml with

  • ivybridge for Ivybridge nodes i.e. debug partition (will run on all partitions)
  • haswell for Haswell and Broadwell nodes i.e lts, im1080, eng, and engc partitions (will not run on debug partition)
  • skylake_avx512 for Skylake and Cascade lake nodes only i.e. enge, im2080, health, chem, hawcpu, hawkgpu, hawkmem and infolab partitions


Add spack to your environment by sourcing the setup-env.sh (add to your .bashrc to always have spack in your path)
source ${HOME}/spack/share/spack/setup-env.sh


Add compilers to spack
spack compiler add

  

Find an installed package e.g. numpy
spack find py-numpy

All python (r) packages have py- (r-) prefix

Install a package, for e.g. JupyterLab
spack install py-jupyterlab
If you want the spack built packages to be available for use with module command, modify the module path as follows
export MODULEPATH=${HOME}/spack/share/spack/modules/linux-centos8-x86_64:${MODULEPATH}


Visit SPACK Website for list of packages that can be built with SPACK.


  • No labels