NSF Pledges $35 Million for a Distributed Terascale Facility

Published 08/09/2001

SAN DIEGO, CA - August 9, 2001 - The National Science Foundation (NSF) has awarded $53 million to four U.S. research institutions to build and deploy a distributed terascale facility (DTF). The DTF will be the largest, most comprehensive infrastructure ever deployed for scientific research-with more than 13.6 teraflops (trillions of calculations per second) of computing power and facilities capable of managing and storing more than 450 terabytes (trillion of bytes) of data.

The four research institutions in the DTF project are the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign, the San Diego Supercomputer Center (SDSC) at the University of California at San Diego, Argonne National Laboratory, and the California Institute of Technology. Each institution plays a key role in the NSF's Partnerships for Advanced Computational Infrastructure (PACI) program, which is building the 21st century's information infrastructure. NCSA leads the National Computational Science Alliance (Alliance) and SDSC leads the National Partnership for Advanced Computational Infrastructure (NPACI). Argonne is a major Alliance partner and Caltech is a key NPACI partner. The partnership expects to work primarily with IBM, Intel Corporation, and Qwest Communications to build the facility, along with Myricom, Sun Microsystems and Oracle Corporation.

Linux clusters purchased through the DTF award and distributed across the four DTF sites will total 11.6 teraflops of computing power. In addition, two 1-teraflops Linux cluster systems already in use at NCSA will be integrated into the DTF system, creating the 13.6-teraflops system-the most powerful distributed computing system ever. Besides the world's fastest unclassified supercomputers, The DTF's hardware and software will include ultra high-speed networks, high-resolution visualization environments, and toolkits for grid computing. All of these components will be tightly integrated into an information infrastructure dubbed the "TeraGrid." Scientists and industry researchers across the country will be able to tap into this infrastructure to solve scientific problems.

"Nothing like the DTF has ever been attempted before. This will be the largest, most comprehensive infrastructure ever deployed for open scientific research," said Dan Reed, director of NCSA and the Alliance and a principal investigator of the TeraGrid award. "Unprecedented amounts of data are being generated by new observatories and sensors, and groups of scientists are conducting new simulations of increasingly complex phenomena. This new age of science requires a sustainable national infrastructure that can bring together new tools, powerful computers, and the best minds in the country. This is the infrastructure that will allow us to solve the most pressing scientific problems of our time."

The DTF will consist primarily of clustered IBM servers based on Intel® Itanium™ family processors interconnected with Myricom's Myrinet. It will build upon two existing clusters of 1,300-plus Itanium and IA-32 processors already deployed at NCSA. The clusters will operate as a single distributed facility, linked via a dedicated optical network that will initially operate at 40 gigabits per second and later be upgraded to 50-80 gigabits per second. The DTF network, developed in partnership with Qwest, will transport data 16 times faster than the fastest research networks now in operation. It will connect to Abilene, the high-performance network that links more than 180 research institutions across the country, STAR TAP, an interconnect point in Chicago that provides access to and from international research networks, and CENIC's CalREN-2, an advanced high-speed network that connects institutions in California. In Illinois, I-WIRE optical network will provide the DTF with network capacity and will give Argonne and NCSA additional bandwidth for related network-research initiatives.

"Breakthrough discoveries in fields from biology and genomics to astronomy depend critically on computational and data management infrastructure as a first-class scientific tool," said Fran Berman, director of NPACI and SDSC and a principal investigator of the TeraGrid award. "The TeraGrid recognizes the increasing importance of data-oriented computing and connection of data archives, remote instruments, computational sites, and visualization over high-speed networks. The TeraGrid will be a far more powerful and flexible scientific tool than any single supercomputing system."

Each of the four DTF sites will play a unique role in the project.

* NCSA will lead the TeraGrid project's computational aspects with an IBM Linux cluster powered by the next generation of Intel® Itanium™ processors, code named McKinley. The cluster's peak performance will be 8 teraflops, combining the DTF-funded systems and other NCSA clusters, with 240 terabytes of secondary storage.

*SDSC will lead the TeraGrid data and knowledge management effort by deploying a data-intensive IBM Linux cluster based on Intel Itanium family processors (McKinley). This system will have a peak performance of just over 4 teraflops and 225 terabytes of network disk storage. In addition, a next-generation Sun Microsystems high-end server will provide a gateway to grid-distributed data for data-oriented applications.

*Argonne will lead the effort to deploy advanced distributed computing software, high-resolution rendering and remote visualization capabilities, and networks. This effort will require a 1-teraflop IBM Linux cluster with parallel visualization hardware.

*Caltech will focus on providing online access to very large scientific data collections and will facilitate access to those data by connecting data-intensive applications to components of the TeraGrid. Caltech will deploy a 0.4-teraflop IBM Itanium processor family (McKinley) cluster and an IA-32 cluster that will manage 86 terabytes of online storage.

"The NSF is a leading indicator of future bandwidth demand, and the launch of this network provides the latest and largest step in bandwidth demand, not unlike the NSFnet, which was the original core of the Internet," said Dr. Wesley Kaplow, chief technology officer of Qwest Government Systems Division. "A number of industries will be able to witness the power of these incredibly high-speed network and computer systems, fueling the demand for network bandwidth to connect their U.S. and worldwide supercomputing systems."

"The DTF project uses computational building blocks based on the Itanium Processor family to empower scientific and business communities to address problems of monumental importance. These problems range from astrophysical research such as black hole simulation, to molecular modeling for the discovery of new drugs and cures, to crash simulations that protect human

life while reducing costs in the automotive industry," said Abhi Talwalker, vice president and Assistant General Manager, Enterprise Platforms Group, Intel Corporation. "Intel is committed to support this kind of important research with Intel Architecture-based building blocks and solutions enabling efforts that allow governments, educational institutions and industry to afford and sustain the highest performance possible."

"IBM's leadership in supercomputing technology and our commitment to Linux and open standards enable us to provide the world's most powerful computers," said David Turek, IBM vice president of Linux emerging technologies. "Leveraging our strong relationship with Intel and Myricom in conjunction with Qwest, we look forward to building a Grid computing system

that represents an important step in the evolution of the Internet and scientific collaboration."

Building and deploying the DTF will take place over three years.

About NCSA and the Alliance

The National Center for Supercomputing Applications is the leading-edge site for the National Computational Science Alliance. NCSA is a leader in the development and deployment of cutting-edge high-performance computing, networking, and information technologies. The National Science Foundation, the state of Illinois, the University of Illinois, industrial partners, and other federal agencies fund NCSA.

The National Computational Science Alliance is a partnership to prototype an advanced computational infrastructure for the 21st century and includes more than 50 academic, government and industry research partners from across the United States. The Alliance is one of two partnerships funded by the National Science Foundation's Partnerships for Advanced Computational Infrastructure (PACI) program, and receives cost-sharing at partner institutions.

About SDSC and NPACI

The National Partnership for Advanced Computational Infrastructure (NPACI) is advancing science by creating a ubiquitous, continuous, and pervasive national computational infrastructure: the Grid. This infrastructure for the 21st century builds on dramatic advances in information technology to enable distributed research by interdisciplinary teams. NPACI is funded by the National Science Foundation and led by the San Diego Supercomputer Center. It joins some 30 other funded partners and 16 domestic and international affiliates and collaborates with the Alliance members on numerous projects.

SDSC is an organized research unit of the University of California, San Diego, and the leading-edge site of the National Partnership for Advanced Computational Infrastructure (NPACI). As a national laboratory for computational science and engineering, SDSC is funded by the National Science Foundation through NPACI and other federal agencies, the State and University of California, and private organizations.

About Argonne National Laboratory

The nation's first national laboratory, Argonne supports basic and applied scientific research across a wide spectrum of disciplines, ranging from high-energy physics to climatology and biotechnology. Since 1990, Argonne has worked with more than 600 companies and numerous federal agencies and other organizations to help advance America's scientific leadership and prepare the nation for the future. Argonne is operated by the University of Chicago as part of the U.S. Department of Energy national laboratory system.

About Caltech

Founded in 1891, Caltech has an enrollment of some 2,000 students, and a faculty of about 290 professorial members. The Institute has more than 19,000 alumni. Caltech employs a staff of more than 2,400 on campus and 4,800 at JPL. Over the years, 28 Nobel Prizes and four Crafoord Prizes have been awarded to faculty members and alumni. Forty-seven Caltech faculty members and alumni have received the National Medal of Science; and eight alumni (two of whom are also trustees), two additional trustees, and one faculty member have won the National Medal of Technology. Since 1958, 13 faculty members have received the annual California Scientist of the Year award. On the Caltech faculty there are 78 fellows of the American Academy of Arts and Sciences; and on the faculty and Board of Trustees, 70 members of the National Academy of Sciences and 46 members of the National Academy of Engineering.

---

Media Contacts:

Ann Redelfs SDSC Deputy Director, External Relations 858-534-5000 redelfs@sdsc.edu

David Hart SDSC External Relations 858-534-8314 dhart@sdsc.edu