SDSC’s ‘Comet’ Supercomputer Assists in Latest LIGO Discovery

Published October 18, 2017

This week’s landmark discovery of gravitational and light waves generated by the collision of two neutron stars eons ago was made possible by a signal verification and analysis performed by Comet, an advanced supercomputer based at the San Diego Supercomputer Center (SDSC) at UC San Diego.

The discovery was made using the National Science Foundation’s Laser Interferometer Gravitational Wave Observatory (LIGO), which recently earned three researchers the 2017 Nobel Prize in Physics for their detection of gravitational waves in the universe, as hypothesized by Albert Einstein in 1915.

SDSC’s Comet was one of several high-performance (HPC) systems used to confirm the newest finding, as well as the initial discovery. As before, LIGO researchers benefited from high-throughput computing via Comet and the Open Science Grid, a multi-disciplinary research partnership specializing in large-scale computing services funded by the U.S. Department of Energy and the NSF.

Comet is also funded by the NSF, and reflects a commitment by SDSC to support high-throughput computing (HTC) on national systems for large experimental facilities that started with SDSC’s support of large-scale data analysis for the Large Hadron Collider using the Gordon supercomputer several years ago.

Two Million Hours

“LIGO researchers use many HPC resources of course, but among those accessed via OSG and the NSF’s Extreme Science and Engineering Discovery Environment (XSEDE), Comet was the top resource used, based on hours of computational time,” said Frank Würthwein, a UC San Diego physics professor who leads SDSC’s distributed high-throughput computing activities, and is OSG’s executive director.

LIGO researchers have so far consumed more than 2 million hours of computational time on Comet through OSG – including about 630,000 hours each to help verify LIGO’s findings in 2015 and the current neutron star collision – using Comet’s Virtual Clusters for rapid, user-friendly analysis of extreme volumes of data, according to Würthwein.

The latest observations of a catastrophic collision of two neutron stars more than 130 light years (one light year equals almost 6 trillion miles) from Earth was initially detected last August via LIGO’s instruments. LIGO researchers subsequently alerted colleagues with the Europe-based Virgo detector and some 70 other ground- and space-based observatories about the detection of the faint signal.

Among other things, astronomers with the U.S. Gemini Observatory, the European Very Large Telescope, and the Hubble Space Telescope discovered that the cosmic fireball, called a kilonova, spewed forth a cloud of gold dust many times more massive than the Earth – in addition to platinum, uranium and other heavy metals – solving a decades-long mystery of where about half of all elements heavier than iron are produced.

“We’re especially proud that Comet is assisting LIGO researchers, and we look forward to offering this system and future ones, along with SDSC’s high-throughput computing expertise, to the LIGO project,” said SDSC Director Michael Norman, who also is the Principal Investigator for the Comet program. “As an astrophysicist myself, it is gratifying to see this extremely collaborative research community tackle science’s grand challenges related to extreme cosmic events.”

Duncan Brown, a LIGO collaborator and The Charles Brightman Professor of Physics at Syracuse University’s Department of Physics who studies gravitational waveforms for black hole and neutron star binaries, said Comet helped speed signal verification and analysis of the event, allowing researchers to confirm their findings in a matter of days rather than weeks or months.

“Comet’s contribution through the OSG and XSEDE allowed us to rapidly turn around the offline analysis in about a day,” said Brown. “That in turn allowed us to do several one-day runs, as opposed to having to spend several weeks before publishing our findings.”

About SDSC

As an Organized Research Unit of UC San Diego, SDSC is considered a leader in data-intensive computing and cyberinfrastructure, providing resources, services, and expertise to the national research community, including industry and academia. Cyberinfrastructure refers to an accessible, integrated network of computer-based resources and expertise, focused on accelerating scientific inquiry and discovery. SDSC supports hundreds of multidisciplinary programs spanning a wide variety of domains, from earth sciences and biology to astrophysics, bioinformatics, and health IT. SDSC’s petascale Comet supercomputer continues to be a key resource within the National Science Foundation’s XSEDE (Extreme Science and Engineering Discovery Environment) program.

About LIGO

The NSF funded LIGO project is operated by Caltech and MIT, which conceived of LIGO and led the Initial and Advanced LIGO projects. Financial support for the Advanced LIGO project was led by the NSF with Germany (Max Planck Society), the U.K. (Science and Technology Facilities Council) and Australia (Australian Research Council) making significant commitments and contributions to the project. More than 1,200 scientists and some 100 institutions from around the world participate in the effort through the LIGO Scientific Collaboration, which includes the GEO Collaboration and the Australian collaboration OzGrav. Additional partners are listed at http://ligo.org/partners.php

About Virgo

The Virgo collaboration consists of some 280 physicists and engineers belonging to 20 European research groups: six from Centre National de la Recherche Scientifique (CNRS) in France; eight from the Istituto Nazionale di Fisica Nucleare (INFN) in Italy; two in the Netherlands with Nikhef; the MTA Wigner RCP in Hungary; the POLGRAW group in Poland; Spain with the University of Valencia; and the European Gravitational Observatory, EGO, the laboratory hosting the Virgo detector near Pisa in Italy, funded by CNRS, INFN, and Nikhef.