SDSC to Double ‘Comet’ Supercomputer’s Graphic Processor Count

Published May 2, 2017

The San Diego Supercomputer Center (SDSC) at the University of California San Diego has been granted a supplemental award from the National Science Foundation (NSF) to double the number of graphic processing units, or GPUs, on its petascale-level Comet supercomputer in direct response to growing demand for GPU computing across a wide range of research domains.

Under the supplemental NSF award, valued at just over $900,000, SDSC is expanding the high-performance computing resource with the addition of 36 GPU nodes, each with four NVIDIA P100s, for a total of 144 GPUs. This will double the number of GPUs on Comet from the current 144 to 288. The nodes are being provided by Dell, the vendor and co-design partner for Comet. They are expected to be in production by early July.

The expansion will make Comet the largest provider of GPU resources available to the NSF-funded Extreme Science and Engineering Discovery Environment (XSEDE), a national partnership of institutions that provides academic researchers with the most advanced collection of digital resources and services in the world. Prior to this award, the NSF granted SDSC a total of $24 million to develop and operate Comet, which went into production in mid-2015.

Once used primarily for video game display graphics, today’s much more powerful GPUs have been developed that have more accuracy, speed, and accessible memory for more scientific applications that range from phylogenetics and molecular dynamics to creating some the most detailed seismic simulations ever made to better predict ground motions to save lives and minimize property damage.

“This expansion is reflective of a wider adoption of GPUs throughout the scientific community, which is being driven in large part by the availability of community-developed applications that have been ported to and optimized for GPUs,” said SDSC Director Michael Norman, who is also the principal investigator for the Comet program.

Applications include but are not limited to GPU-memory management systems such as VAST, analysis of data from large scientific instruments, and molecular dynamics software packages such as AMBER, LAMMPS, and BEAST – the latter used extensively by SDSC’s Cyberinfrastructure for Phylogenetic Research (CIPRES) science gateway, which receives the majority of its computing resources from Comet. 

Francis Halzen, principal investigator of the IceCube Neutrino Observatory, the first detector of its kind designed to observe the cosmos from deep within the South Pole ice, welcomes the new GPU addition.

“The IceCube neutrino detector transforms natural Antarctic ice at the South Pole into a particle detector,” said Halzen, also a physics professor at the University of Wisconsin-Madison. “Progress in understanding the precise optical properties of the ice leads to increasing complexity in simulating the propagation of photons in the instrument and to a better overall performance of the detector.”

“This expansion will help the XSEDE organization meet increased demand for GPU resources from these areas, as well as prepare for research in new areas, such as machine learning, which has become increasingly important for a wide range of research in areas including image processing, bioinformatics, linguistics, and others,” said SDSC Director of Scientific Applications and Comet Co-PI Bob Sinkovits.

The P100 is NVIDIA’s newest GPU. SDSC benchmarking tests show that for some applications the GPUs achieve speed-ups of two times over the K80 GPUs already in Comet. The new GPU nodes will be added to the existing Comet GPU nodes and become a separately allocable resource.

The amended NSF award number for Comet, including the GPU additions, is FAIN 1341698. The award is estimated to run until March 30, 2020.

About SDSC

As an Organized Research Unit of UC San Diego, SDSC is considered a leader in data-intensive computing and cyberinfrastructure, providing resources, services, and expertise to the national research community, including industry and academia. Cyberinfrastructure refers to an accessible, integrated network of computer-based resources and expertise, focused on accelerating scientific inquiry and discovery. SDSC supports hundreds of multidisciplinary programs spanning a wide variety of domains, from earth sciences and biology to astrophysics, bioinformatics, and health IT. SDSC’s petascale Comet supercomputer continues to be a key resource within the National Science Foundation’s XSEDE (Extreme Science and Engineering Discovery Environment) program.