Introduction to Singularity: Creating and Running Containers for High-Performance Computing

Singularity is a container platform which allows you as an HPC user to pack the applications and libraries into a single container (as a Singularity image). With the Singularity image, you can share, move and reproduce the application you build in the container. You can also build the image outside of the supercomputer cluster, which means with your laptop, you will have full control to choose any major Linux distributions, system libraries, and can decide how to install the application you need. What is more important is, unlike other container platforms such as Docker, Singularity is dedicated to running on clusters! Therefore, once the Singularity image is created, you can easily run it on many of the supercomputer clusters including TSCC at SDSC. You don’t have to worry about how to install all the software you need on each different cluster.

In this webinar, we will first show how to build Singularity images on your local resource such as your laptop. Some commonly used repositories will be introduced, e.g.: Docker Hub, so you will learn where to pull other containers to create the Singularity images. For more complicated software requests, we will introduce the proper workflow to build a Singularity sandbox for test purposes, and how to record them into a Singularity recipe so a final custom container can be created accordingly.

Once the Singularity image is created, it can be uploaded and run on a cluster like TSCC. We will show three ways to run the applications within the container. Based on the cluster file system structure, we will introduce the binding point technique from Singularity to locate the proper storage, and how to add binding points into the recipe. Also, we will let you know how to make a short command to replace the long Singularity run command.

Prerequisites: Basic knowledge of using HPC environment is assumed but not required.