Globus Links Environments for Scientific Discovery

PARTICIPANTS
The Globus Team, Argonne National Laboratory Information Sciences Institute Aerospace Corporation

he Globus Project is a five-year-old collaborative effort to make networked resources for the scientific community commonly accessible. NPACI and other institutions are supporting Globus development, and the project is producing one success after another. Globus software and tools allow scientists to log in securely at any access point and then to use varied applications programs, scientific instruments, databases, and tools for scientific visualization, all interactively. This article focuses on recent developments in the Globus Project, including new collaborations, newly connected applications and instrumentation, and the latest research on advanced concepts.

WHAT IS GLOBUS?

TELEMICROSCOPY TO MICROTOMOGRAPHY

SECURING COORDINATED RESOURCES

WHAT'S NEXT

WHAT IS GLOBUS?

"Since the last appearance of Globus in enVision (July-September 1998), there has been much activity in the Globus arena," said Carl Kesselman of the Information Sciences Institute (ISI) at the University of Southern California, co-leader of the Globus project. "The seamless environment for scientific discovery is growing up and branching out, in part because of the support of the NSF PACI centers."

Globus is a joint project of ISI, Argonne National Laboratory (ANL), and the Aerospace Corporation, and is a focus of research at both NPACI and NCSA. Led by Kesselman and Ian Foster of ANL and the University of Chicago, Globus develops fundamental infrastructure to enable distributed high-performance computing.

"Globus is a key element in the advanced computational infrastructure," agrees Anke Kamrath, SDSC liaison to the NPACI Metasystems thrust area and Associate Director for Resources at SDSC. "NPACI has been fortunate to work with the Globus project on applications, deployment, and advanced concepts."

Globus software and tools are deployed at more than 80 institutions around the world (three times the number last reported), including the PACI centers. The project participates in the DOE Accelerated Strategic Computing Initiative (ASCI) and Next Generation Internet (NGI) programs and recently worked successfully with NASA's Information Power Grid (IPG) program.

"Our objective is to manage rather than hide heterogeneity," Kesselman said, "making it possible to build high-performance, application-specific tools." Globus follows a layered approach to building grid infrastructure. The most fundamental layer consists of a set of core services, including resource management, security, and communications that enable the linking and interoperation of distributed computer systems. Toolkits, such as the Message Passing Interface (MPI) for parallel computing and CavernSoft (for collaborative virtual reality) provide higher-level, application-friendly access to Grid services.

"We've been working hard on three fronts," Foster said. "We've worked with applications groups to demonstrate distributed, interactive applications linked through Globus services. We've worked with members of the Globus Ubiquitous Supercomputing Testbed (GUSTO), which includes 40 sites in eight countries, to deploy Globus widely. And we've worked to develop and incorporate advanced concepts within the existing Globus framework."

---

Figure 1. Ant Microtomography

Globus and an NPACI-developed tomography code were combined to allow researchers at the ANL Advanced Photon Source inspect an ant head, about 2 mm in diameter. With the Globus-enabled X-ray microtomography program a fully 3-D reconstruction was obtained in less than 10 minutes using the acquisition hardware at the beamline and an SGI Origin 2000.

---

TELEMICROSCOPY TO MICROTOMOGRAPHY

"We enjoy working with applications groups to 'grid-enable' their software and software-driven instruments using Globus services and tools, because the process helps us keep Globus at the forefront just as it adds power to the individual group's research," Kesselman said. "And sometimes there's a lot of synergy."

One of these major applications groups is the Advanced Photon Source (APS) at ANL. This bright X-ray synchrotron device is among the largest and most powerful instruments for crystallographic and biomedical research, with users at many remote locations. Last year the Globus team worked with Mark Ellisman of the NPACI Neuroscience thrust area to make a telemicroscopy code by lab member Steve Lamont run via Globus. That code was modified for work at APS on microtomography by Mei-Hui Su of ISI and the Globus project team. APS scientists made high-resolution X-ray observations of complex but tiny structures such as micromechanical gears and, as an experiment, the head of an ant (Figure 1), reconstructing them as volume renderings via the Globus-enabled tomographic code.

"The ant so intrigued one biologist at APS that he has designed a project to obtain microtomography information from a spider," Kesselman said. "Another important interaction was with the NASA group developing an Information Power Grid." To test a widely used code called OVERFLOW, NASA ran a simulation on supercomputers in Illinois and California simultaneously. To avoid bottlenecks, the researchers employed a "deferred boundary" technique to hide communications latency.

"The development of such techniques is exactly the kind of limit-stretching of applications that Globus stimulates and enables," Foster said.

Now Globus is again working with Ellisman's group and with Fran Berman of the UC San Diego Computer Science and Engineering Department, to connect the applications-level scheduler AppLeS, developed by Berman's group. With AppLeS layered on top of Globus, Ellisman's group will automate use of Globus services. AppLeS decides which resources to use from those available in the Globus testbed. A multi-laboratory test of the system is planned.

Also worth mention, Kesselman said, is continuing work with the Cactus project, an astrophysical application under development by an international collaboration. Together with the National Laboratory for Applied Network Research and SDSC, the group mounted a demonstration of a code for the full-scale relativistic simulation of space and time at SC98 (Figure 2). Cactus collaboration leader Ed Seidel received the Heinz-Billing Award for the Advancement of Scientific Computation for 1998 in recognition of the work, and the Globus team itself won the HPC Challenge Award at SC98.

Figure 2. Colliding Neutron Stars

The Globus-based Cactus framework for creating portable parallel finite difference simulation codes was used to simulate colliding black holes and neutron stars in a demonstration at SC98. The simulations ran on supercomputers on both sides of the Atlantic Ocean -- from NPACI, Max-Planck-Institut für Gravitationsphysik, the Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB), and NCSA. These images, provided by Werner Benger, capture the simulation at different time steps.

SECURING COORDINATED RESOURCES

"Globus is proving itself as an infrastructure element in production mode," Kesselman said. "Ultimately, researchers will use Globus security authentication once and then run everywhere." The Globus security infrastructure has recently been deployed across the testbed, in particular as a basic public-key infrastructure across PACI resources. Foster noted that a commercial software developer (Van Dyke) has adapted the Globus security infrastructure in its latest release of Secure-CRT, an SSH client, a sign of Globus's wide influence. "We're planning to make Globus-authenticated usage of all our systems available from the NPACI HotPage," said Mary Thomas of the SDSC Scientific Computing group.

In addition to hardening Globus for production use, the Globus project is maintaining a focus on experiment. "We've developed an experimental multi-site advanced reservation system, called MARS, that enables users to reserve processors on the IBM SPs at SDSC and ANL and an SGI Origin at NCSA," Kesselman said.

A recent workshop at ANL on the advanced reservation system outlined plans for linking Globus-enabled sites. A further research focus, according to Foster, is support for what is called quality of service, guaranteed network bandwidth for bandwidth-intensive computational projects. The Department of Energy's Next-Generation Internet awards include support for this concept in a project involving the NPACI Data-Intensive Computing thrust area, the DOE's ESnet, and the Abilene network. Also, within NPACI, Globus has been working to integrate the Meta-Chaos project of the Programming Tools and Environments thrust area into the Globus environment.

"Globus now runs across an entire spectrum of production and research modes. It has developed fundamental technology that is in daily use across the sites of the national computational research establishment," Kesselman said.

WHAT'S NEXT

The project continues its focus on issues in designing and developing the applications that use Grid services. "As our tools become widely used, our overall concept gains acceptance and changes in response to developments that come from other sources. Our next steps are driven in part by the needs of our developing system and in part by the needs of the users of the Grid," Kesselman said.

A feature of the Globus project is an annual "retreat" at which participants discuss research, development, and deployment issues. The fifth Globus Retreat will be held on August 1-3, 1999, in Redondo Beach, California. Complete information is available at the Globus Web site. Tutorials for new and prospective users will be followed by two days of technical sessions. The retreat occupies the three days before the eighth IEEE High Performance Distributed Computing symposium, being held at the same location.

"These events are an opportunity for users and developers of the Globus distributed computing toolkit to discuss common interests," Kesselman said. "Members and prospective members of the Globus community are all invited." --MM