Skip to navigation
Site Primary Navigation:
- About SDSC
- Services
- Support
- Research & Development
- Education & Training
- News & Events
Search The Site:
SDSC User Support staff Marty Kandes was kind enough to build several Singularity containers designed specifically to run on SDSC's Comet supercomputer. Here is a brief tutorial on how to run Singularity containers on Comet.
Containers are located on Comet in the following path:
/oasis/scratch/comet/mkandes/temp_project/singularity/images/
If you want to build your own containers based on those already built for Comet, use the filesizes in the table below as a guide for minimum container size.
You can access the def files used to build the containers at:
https://github.com/mkandes/naked-singularity
| Container Name | Size (GB) |
| gmtsar.img | 17.18 |
| julia-dep-only.img | 17.18 |
| keras-tensorflow-cpu.img | 17.18 |
| keras-tensorflow-gpu.img | 17.18 |
| kinc.img | 4.27 |
| meep.img | 4.3 |
| mxnet-cpu.img | 8.59 |
| mxnet-gpu.img | 17.18 |
| neon-cpu.img | 8.59 |
| neon-gpu.img | 17.18 |
| pytorch-cpu.img | 17.65 |
| pytorch-gpu-dev.img | 34.36 |
| pytorch-gpu.img | 34.36 |
| tensorflow-cpu.img | 17.18 |
| tensorflow-gpu.img | 17.18 |
| theano-cpu.img | 8.59 |
| theano-gpu.img | 17.65 |
| torch-gpu.img | 8.59 |
| ubuntu-cuda.img | 8.59 |
| ubuntu-cuda-opencl.img | 17.18 |
| ubuntu-cuda-openmpi.img | 8.59 |
| ubuntu.img | 2.15 |
| ubuntu-mvapich2.img | 2.15 |
| ubuntu-openmpi.img | 2.15 |
Here are some brief descriptions of their content and purpose.
"ubuntu.img"
This is a general purpose Singularity container using the most recent Ubuntu from the default mirror at http://us.archive.ubuntu.com/ubuntu and includes miniconda 2 and 3, gfortran, python 3, zip utilities, and more.
"ubuntu-openmpi.img"
This container is similar to "ubuntu.img" above but also includes OpenMPI and basic drivers for user space access to Ethernet, ROMA, and Infiniband.
"ubuntu-cuda.img"
This container is similar to "ubuntu.img" above but also includes NVIDIA CUDA drivers, libraries, and packages.
"ubuntu-cuda-opencl.img"
TBD
"ubuntu-cuda-openmpi.img"
This container is similar to "ubuntu-cuda.img" above but also includes a CUDA-aware version of OpenMPI.
"ubuntu-mvapich2.img"
This container is similar to "ubuntu.img" above but also includes MVAPICH2 and basic drivers for user space access to Ethernet, ROMA, and Infiniband.
"gmtsar.img":
This container is designed to make it easy for Comet users to run GMTSAR (http://topex.ucsd.edu/gmtsar/downloads/).
"julia-dep-only.img"
TBD
"keras-tensorflow-cpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the Tensorflow deep learning application who are running jobs using Comet's GPU nodes. Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.
"keras-tensorflow-gpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the Tensorflow deep learning application who are running jobs using Comet's GPU nodes. Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.
"kinc.img"
TBD
"meep.img"
This container is similar to "ubuntu.img" above but also includes the Meep* application and basic drivers for user space access to Ethernet, ROMA, and Infiniband.
*Meep is a free and open-source software package for simulating electromagnetic systems via the finite-difference time-domain (FDTD) method. Meep is an acronym for MIT Electromagnetic Equation Propagation.
"mxnet-cpu.img"
TBD
"mxnet-gpu.img"
TBD
"neon-cpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite software to support users of the neon deep learning application who wish to run on Comet's compute nodes. neon is Intel Nervana ‘s reference deep learning framework committed to best performance on all hardware. Designed for ease-of-use and extensibility.
"neon-gpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the neon deep learning application who wish to run on Comet's GPU nodes. neon is Intel Nervana ‘s reference deep learning framework committed to best performance on all hardware. Designed for ease-of-use and extensibility.
"pytorch-cpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite software to support users of the PyTorch machine learning application who wish to run on Comet's compute nodes.
"pytorch-gpu-dev.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the PyTorch machine learning application who wish to run on Comet's GPU nodes. PyTorch is an open source machine learning library for Python, based on Torch, used for applications such as natural language processing. This container differs from the above PyTorch container by including additional development tools for more advanced customization.
"pytorch-gpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the PyTorch machine learning application who wish to run on Comet's compute nodes. PyTorch is an open source machine learning library for Python, based on Torch, used for applications such as natural language processing.
"tensorflow-cpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite software to support users of the Tensorflow deep learning application who are running jobs using Comet's compute nodes.
"tensorflow-gpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the Tensorflow deep learning application who are running jobs using Comet's GPU nodes.
"theano-cpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite software to support users of the Theano deep learning application who wish to run on Comet's compute nodes.
"theano-gpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the Theano deep learning application who are running jobs using Comet's GPU nodes.
"torch-gpu.img"
This container is similar to the basic ubuntu containers listed above and also include the requisite NVIDIA libraries, drivers, and packages needed to support users of the Torch machine learning application. Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first. It is easy to use and efficient, thanks to an easy and fast scripting language, LuaJIT, and an underlying C/CUDA implementation.