Ian Foster on Globus: Accomplishments, Vision, and Coming Attractions
|
1.23.04 -- One of the co-creators¹ of the Globus Toolkit, Ian Foster from Argonne National Laboratory, provided one of the keynotes on "Globus and the Grid: State of the Union" for some 450 attendees at this week's GlobusWORLD 2004 conference. The title alluded to the fact that President Bush had provided the State of the Union address to the nation just the night before.
Foster began and concluded his remarks with reference to Globus as a "balancing act" between addressing short-term needs (software engineering and development plus supporting user needs) and long-term needs (pushing development of standards, conducting research, and keeping the project on the "bleeding edge"). "Globus must be a research and a development organization," he said.
Accomplishments of 2003 included progress on standards, software development, and development of solutions in particular applications domains. With regard to standards, given Globus' goal of defining a service-oriented architecture, progress was made in addressing vital Grid requirements and building Web Services standards (a major focus for the last two years), which is leading toward establishment of an Open Grid Services Architecture. The OGSA, he admitted, is a work in progress, but it's moving forward rapidly.
Further, he acknowledged an important "bump in the road": Moving the Open Grid Services Infrastructure (defined last June and implemented in the Globus Toolkit) toward the Web Services Resource Framework. Foster said that the Globus community has been pivotal in moving the Grid community to Web services, in particular by identifying "state" as critical missing piece in the OGSI.
However, the OGSI was developed largely outside the Web Services community. The resulting pushback from that community compromised Globus' goal of ubiquity but it did serve the purpose of engaging them to resolve the problems, which, in turn, led to development of the Web Services Resource Framework.
Probably the biggest news at the conference, in fact, was an announcement by the Globus Alliance, IBM, and HP on details of the new WS-Resource Framework, a further convergence of Grid services and Web Services. "So the convergence between the Grid and Web Services can be considered completed," said Foster delightedly.
Several specifications have been finalized. As examples, Foster pointed to OGSI 1.0, x.509 proxy certification, and GridFTP. And, he said, "We are leading [in the Global Grid Forum] data access and integration, WS-agreement, and replica location efforts."
With respect to software development, Globus efforts have taken care of the existing user base, providing multiple 2.x releases to address user needs, continued support for pre-WS code, and numerous new features and improvements. In addition, they have introduced leading-edge technologies: The 3.0 release supports the Open Grid Services Infrastructure, a first attempt to link Grid and Web Services, and other new services, such a Reliable File Transfer and Replica Location Service. Community interest remains strong: Since the release of v2.4 in April last year, the Globus Web site has experienced more than 10,000 downloads per month.
Globus partners are creating strong GT-based Grid solutions, including the Virtual Data Toolkit, Platform Globus, the NSF Middleware Initiative distribution, the Butterfly Grid, the IBM Grid Toolkit, the Access Grid, the Earth System Grid, the BIRN Biomedical Grid (led by Calit² DeGEM participant Mark Ellisman), the TeraGrid, and the UK eScience Grid.
Examples of production Grid deployments include Grid2003 by the U.S. physics community to create a persistent U.S. Open Science Grid, which was stood up at SC2003 and is still operational. A second example is the NEESGrid, a multi-site Grid to support distributed earthquake engineering research.
The "community" includes the Globus Alliance, which "lives and breaths GT," according to Foster; academic affiliates and users who are major contributors to GT technologies; the Global Grid Forum, a locus for many standards activities; and vendor and user companies. Foster emphasized that the GT is very much a community effort, supported broadly by academic research labs, supercomputer centers and the national labs, and, increasingly, industry.
|
Then Foster turned to the Grid vision, defining the Grid as something that "coordinates distributed resources using standard, open, general-purpose protocols and interfaces to deliver required qualities of service" (also becoming known as "qualities of experience"). As part of this discussion, he referred attendees to the 2nd edition of The Grid, called The Grid 2: Blueprint for a New Computing Infrastructure, which he and Kesselman edited. This book was published in November 2003.
The Grid of the future, Foster said, will be a capability-rich environment that adapts easily to changing requirements. Routine, ubiquitous, and predictable, its services will provide the advanced capabilities needed to meet user requirements and user quality of experience needs.
However, he pointed to integration and management as fundamental challenges. The Grid will consist of many sources of data, services, and computation. Registries will organize services of interest. The services are likely to be found at many locations. And exploration and analysis may involve complex, multi-step workflows.
So the definition of the Grid implies that we must be able to describe resources in uniform ways, govern the behavior of resources that are not under our control and/or are oversubscribed, and exploit commonality across resources so as to reduce the complexity of integration.
The Grid also needs uniform resource management, a "primary driver of our work over last five years," Foster said. "We need a ubiquitous, uniform base for modeling and managing resources, and we are now well positioned to model and manage an increasingly rich resource set and their lifetime functions."
All interesting interactions on the Grid in the future, Foster envisioned, will be based on previously negotiated agreements between requestors and services (with encoded, negotiated quality of experience). The challenge will be to establish the framework so that agreements can be established in an oversubscribed environment; transformed, composed, and decomposed; managed like any other resource; and evolved as a result of faults. "Web Services Agreement," Foster said, "is the next step on this path."
The community also needs to identify and factor in commonalities across a broad range of applications and resources. Effort should be expended on task management (jobs, data transfers, workflows), data movement (files, database replication), registries (discovery, metadata, assertions), and faults and management (applications, computers, networks).
Goals for 2004 include stabilizing the infrastructure around WSRF in terms of usability, performance, reliability, scalability, documentation, and "internationalization where users' first language is not English." Globus developers must also realize new functionality in the pipeline, such as data access and integration/metadata management, enhanced GridFTP, a community scheduling framework, and monitoring and discovery frameworks. They should also expand the set of solution providers and increase engagement with the corporate community.
Users should anticipate the release of GT v3.2 in a couple of months. V4.0 (Beta) is expected in Q204, the production version in Q304 (WSRF and some new functionality), and v.4.2 (Beta) early next year. GT v3.2 will include many bug fixes; usability improvements; improvements to Grid FTP, RFT, RLS, and GRAM; a "preview" of OGSA data access and integration; an optimized core' eXtensible IO (XIO); and security.
V4.0 will implement the draft WSRF specifications, preserving the OGSI client and service APIs as much as possible and all pre-Web-Service components.
For more on the conference, see www.globusworld.org. Calit² was a "diplomat"-level sponsor of this conference and contributed a research exhibit. Calit² director Larry Smarr provided a keynote presentation on Tuesday.
¹With Carl Kesselman of the University of Southern California/Information Sciences Institute and Steve Tuecke of the Argonne National Laboratory.
Related Links