Diving into Data-Enabled Scientific Computing at SDSC

Published April 19, 2022

By Cynthia Dillon, SDSC Communications

The San Diego Supercomputer Center (SDSC)—established in 1985 as one of the nation’s first supercomputer centers—architects, designs, builds and operates innovative supercomputers such as Gordon, Comet, Expanse and Voyager. In recent years such projects have been funded by competitive National Science Foundation (NSF) grants, and the Data-Enabled Scientific Computing (DESC) Division at SDSC has played an important role in acquiring these grants.

In addition to the NSF-funded supercomputers, grants acquired by DESC principal investigators (PIs) and Co-PIs span a range of areas—high-performance computing (HPC), molecular dynamics, quantum chemistry, genomics, cosmology, neuroscience, data provenance, parallel math libraries, commercial cloud access, CI software, HPC network, science gateways and education. This diversity in grant domains parallels the breadth of the DESC Division, which is one of seven at SDSC. Headed by Amit Majumdar, DESC is organized into multiple groups which have specific expertise to lead HPC and computational sciences innovation and to serve and collaborate with thousands of SDSC’s national, University of California-wide and UC San Diego researchers, as well as industry partners.

“We provide full support to the user community, and provide training to thousands of users on various topics that allow them to use these machines effectively to do science and education, including classroom teaching,” said Majumdar. “We also develop and improve performance of user applications covering the complete set of science domains such that the codes make optimal use of our compute resources in terms of scalability of compute algorithms with massive parallelism, data movement and data storage. The science gateways, used by thousands of researchers, use these supercomputers for computing.”  

In recent years SDSC supercomputers have enabled the “long tail of science” and “computing without boundaries” for about 100,000 users for research and education. “We strongly contribute toward democratization of access to large-scale computing, including the commercial cloud, by researchers and educators nationally,” said Shava Smallen who leads the Cloud Software Development Group.

According to Majumdar, DESC has about 30 staff members and about two-thirds of them have master’s degrees or doctoral degrees in computational/computer science, domain science areas of biochemistry, bioinformatics, data science, physics and many engineering disciplines. DESC researchers have published their research results in prestigious journals such as Nature and Science and often present at reputed national and international conferences, including winning best papers and being co-recipients of awards such as the Gordon Bell Prize.

“Many of the staff members wear a dual hat, where, in addition to working on supercomputers or projects funded by NSF, UC San Diego, UC or industry, they are PIs and Co-PIs on research grants funded by NSF, NIH and other funding agencies,” said Subhashini Sivagnanam, who leads the Cyberinfrastructure Services and Solutions Group.

According to Andrea Zonca, who leads the Scientific Computing Applications Group, the DESC division functions in such a way that allows it to combine the knowledge, expertise and experience of DESC members from different groups. “This enables us to provide optimal solutions to SDSC’s large user communities, be it by developing and operating supercomputers and clusters or via funded research projects led by PIs and Co-PIs from DESC,” said Zonca.

DESC also utilizes expertise and resources from other SDSC divisions and groups such as Research Data Services (RDS), Cyberinfrastructure and Convergence Research and Education (CICORE), Sustainable Scientific Software (S3), High Throughput Computing (HTC), Sherlock Cloud Solutions and Services (SCSS), Business Services and External Relations. “For architecting the supercomputers, we work very closely with SDSC’s Deputy Director Shawn Strande,” noted Majumdar.

“The depth and breadth of expertise and knowledge of DESC staff members, and the research carried out by many of them, allow us to serve the broad community of researchers in academia and industry,” said Christopher Irving, leader of the High-Performance Computing Systems Group. “All of our groups serve them by operating and supporting the supercomputers and by providing CI solutions for research and education.”

DESC team members and others pictured at a division event, March 2021. Photo courtesy of Amit Majumdar.

Supercomputers and clusters, which DESC designs, builds and operates, include the Expanse supercomputer; Comet, previously funded by the NSF and currently funded by the Center for Western Weather and Water Extremes (CW3E) at Scripps Institution of Oceanography; the Triton Shared Compute Cluster (TSCC), an agile, medium-scale, high-performance and efficient computing cluster primarily for campus researchers and students to explore and begin innovative research and the Popeye machine, operated at SDSC on behalf of the Flatiron Institute of the Simons Foundation.

“We have two upcoming innovative machines funded by the NSF—Voyager, which will provide AI-focused hardware to explore AI in science and engineering, and the National Research Platform (NRP), which is a distributed infrastructure with compute hardware on the West coast, Midwest and East coast, and a content delivery system with caches in the national network backbone of Internet2 in five additional locations,” said Mahidhar Tatineni, manager of the User Services group. “Across all of our supercomputers there are about 200,000 cores and about 1,500 accelerators, which deliver nearly 2 billion core-hours and over 12 million accelerator hours per year, and provide 46 petabytes of usable storage. About 100,000 users have been using these resources in recent years.”

DESC members provide systems expertise in innovative systems management and significant domain and computational science expertise in using these machines. “In addition, DESC staff collaborate with many researchers in various capacities and run their own research programs,” said Andreas Goetz, who leads the Computational Chemistry Group.

Additionally, throughout each year, DESC experts host numerous training sessions and workshops, including hackathons—basically a design race in which computer programmers and others involved in software development work in small teams to create and/or optimize functioning software within a limited timeframe.

DESC also hosts SDSC’s long-running Summer Institute featuring its members and others as speakers during the week-long supercomputing and data science training session. Other experiential learning projects include the Supercomputing Student Cluster Competition (SCCC) for UC San Diego students, which is an international competition to build a cluster and run optimized applications to achieve the highest performance possible on an architecture. “SDSC also hosts a 14-week HPC User Training series, which is in its fourth year, with focus on undergraduates, graduates, and the general research communities,” said Mary Thomas, who leads the HPC Training programs and directs the SCCC.  

“We have NSF grants related to CI training focusing on AI and neuroscience. Another program, running over a decade and managed by Ange Mason of DESC, is the Research Experience for High School Students (REHS) program, which allows high school students to do an eight-week long summer internship with mentors from various divisions of SDSC,” said Bob Sinkovits, SDSC’s HPC Education, Outreach and Training lead. “This is a tremendous program that allows high school students to gain important research experience and an environment to collaborate with others. These high school students have gone to top universities in the U.S., including UC San Diego.”

DESC’s HPC@MSI program is specifically targeted for Minority Serving Institutions (MSI)  and focused on introducing them to national cyberinfrastructure resources.

“In addition to topics of supercomputing and data science, we host training sessions, on topics of science gateways for faculties from Hispanic Serving Institutions (HSIs) and collaborate with MSI faculties in various capacities,” said Nicole Wolter, who manages the HPC@MSI and HPC@UC programs among others. “Many of our NSF-funded projects include strong efforts in Broadening Participation in Computing.”

According to Majumdar, it is the people of the DESC division who make him the proudest and who are the most enjoyable part of his work. “They are tremendously talented, knowledgeable, dedicated, hard-working and smart—with a sense of humor, too! They take pride in their work, and many of them are scientists and researchers who carry out novel research funded by their own grants.”

To learn more about the DESC Division at SDSC, please use the SDSC Directory to contact the following group and program leaders:

Cloud Software Development: Shava Smallen

Cyberinfrastructure Services and Solutions: Subhashini Sivagnanam

HPC Systems: Christopher Irving

Scientific Computing Applications: Andrea Zonca

User Services: Mahidhar Tatineni

SDSC's XSEDE and XSEDE ECSS Programs: Bob Sinkovits 

Data-Driven and High-Performance Computational Chemistry: Andreas Goetz

Bioinformatics and High-Performance Computing: Wayne Pfeiffer

HPC Education, Outreach and Training: Mary Thomas and Bob Sinkovits

HPC@MSI and HPC@UC Programs: Nicole Wolter

HPC Network and Architecture: Tom Hutton