Extending OptIPuter to the Broader e-Science and HPC Communities

San Diego, CA, February 14, 2006 -- The OptIPuter All Hands Meeting (AHM), hosted by Calit2 UCSD, attracted some 70 participants from Europe, Asia, and various parts of the U.S. The annual AHM is a forum for face-to-face discussions of current activities, accomplishments, and future plans. Three days of working meetings were followed by an open house, during which attendees and guests were treated to a 4K digital cinema experience in the Atkinson Hall theater.

Calit2 director Larry Smarr, who is also the principal investigator of the OptIPuter project, led the meeting. OptIPuter is prototyping a 21st-century cyberinfrastructure, based on optical networking, to support data-intensive scientific research and collaboration. This infrastructure will enable scientists to collaborate with remote colleagues and large-scale data sets in real time over high-performance networks.

The OptIPuter architecture involves bandwidth matching the individual nodes of one cluster to another network-attached cluster, maximizing the amount of data that can be moved, thereby creating a globally distributed computer connected by a "backplane," rather than a network. This requires the data being sent to be parallelized, which impacts the entire software stack, from the network control management and transport protocols, to the middleware, toolkits and applications.

Maxine Brown, Electronic Visualization Laboratory, University of Illinois at Chicago (UIC), and Steve Peltier, National Center for Microscopy and Imaging Research (NCMIR) (left photo). Jason Leigh, EVL, and Cees de Laat, University of Amsterdam (right photo).

Two days of working meetings for funded and affiliate partners of OptIPuter were held January 18-19. These were preceded on January 17 by a Backplane Architecture Workshop, during which partners discussed OptIPuter optical networking control plane development. The nearly week-long event culminated with a one-day "open house" on January 20 for community members and potential partners interested in learning more about the project. Presentation slides from the meeting are available online (see link below).

Calit2 Experiment Operates Remote-Controlled Car over OptIPuter Link The OptIPuter AHM featured a demonstration of real-time middleware software that enabled an electrically powered car located at UCI to be controlled from UCSD, via a dedicated OptIPuter link (1 Gb, rather slow by Optiputer standards). Remote control of various kinds of instrumentation may be enhanced by this technology being developed by researchers at UCI. Jan Biermeyer, a visiting scholar from Paderborn, Germany, and Moon Kim, a graduate student in the EECS department in The Henry Samueli School of Engineering at UCI, recently demonstrated remote control of a three-foot electric car over a network set up as part of the Calit2 OptIPuter project. Jan Biermeyer operating the controls from UCSD ... ...while the eletric car responds at UCI. The car is the largest, electrically powered, remote-controlled car used for hobby racing and capable of a maximum speed of 60 miles per hour. During the demonstration, the car was located in the server room at Calit2/UCI. Biermeyer and Kim, at Calit2/UCSD, could see it on a computer monitor and remotely controlled it via a common joystick plugged into a computer linked to the OptIPuter network. Their joystick commands, related to steering angle and speed, were sent over the OptIPuter network to UCI, then through a short-haul wireless link to the car. On the car, a small computer board evaluated the commands and steered the car by sending its own commands, via a micro controller and driver board, to the motor and steering sensor. In response, a webcam at UCI plugged into a computer with an OptIPuter connection provided video feedback on the car's changing location through the OptIPuter link to the computer and monitor at UCSD, which guided Biermeyer's and Moon's subsequent interaction with the car. Said Biermeyer, "It takes some time to get used to the remote control with video feedback, as left and right are switched when the car is driving towards the camera. But once you're used to it, it works as well as if the joystick were plugged directly into the car." Video resolution was 640 x 480 at 20 frames per second. The joystick commands were conveyed 50 times per second so that messages could be sent between the two sites, about 100 miles apart, in about 1.5 milliseconds.  The Time-triggered Message-triggered Object (TMO) software developed in Kane Kim's DREAM lab at UCI together with the OptIPuter optical transmission made the video resolution and speed possible. Said Biermeyer, "You can imagine applying this technology to remote control of some activity that requires small hand movements, such as a surgical procedure, control of a microscope, or some manufacturing process." For additional information on this project, please see "OptIPuter Network Gives New Meaning to Remote Control ."

OptIPuter partners discussed the development of a "gold standard" custom infrastructure, to debut in November at SC06 in Tampa, FL. This standard will include a simplistic implementation: "prototyping the way science will be done in the future" such that "a single button can be pushed" to display large-scale images and information from a variety of applications ("digital posters" in the words of OptIPuter co-principal investigator Jason Leigh, Electronic Visualization Lab, University of Illinois at Chicago). It will also include a more sophisticated implementation in which one can operate remote instrumentation with real-time, high-definition feedback while analyzing and extracting relevant information. Four core OptIPuter sites -- Chicago, San Diego, NASA Goddard, and Amsterdam -- will set up a persistent "gold standard" environment, to which other partners will connect.

The overall theme of the week was "Extending the OptIPuter to the Broader e-Science and HPC Communities." There are a number of immediate possibilities for extending the OptIPuter's data-intensive, real-time collaboration capabilities into new science and HPC communities. Any research community needing to work with very large data sets remotely, interactively, and visually is likely to benefit from this infrastructure. Projects fitting this description -- from radio astronomy to genomics to ecology, in addition to current projects in bioscience and geoscience -- were identified and targeted for outreach. Partnerships with the national laboratories will also be explored. Los Alamos National Laboratories, for example, performs 1-billion-atom simulations of the behavior of chips, which could benefit from technologies developed in the OptIPuter project.

Establishing lambda capabilities on individual campuses enables researchers to have end-to-end connections to National LambdaRail and optical networking testbeds, such as CAVEwave, which is dedicated to OptIPuter research between Chicago and San Diego. Ten- and 40-lambda links are possible now, and 100-lambda capabilities are being investigated in corporate research laboratories. This is becoming possible due to a radical reduction in cost from carrier-grade dense wave-division multiplexing to tens of thousands of dollars, making it possible to link clusters end-to-end with lambda technology. "Parallel lambdas are to networking what PC processors were to parallel computing in the early 1990s," said Smarr.

"This year's OptIPuter All Hands meeting was a tremendous success," said Joe Keefe, new research project manager, Calit2 UCSD. "Plans were developed to move the project towards a more production-level deployment; persistence was the key word for this year's meeting."

Fundamental questions remain which will keep the researchers engaged. How can one terminate 10s and 100s of gigabits of bandwidth at the end points so that cluster nodes at one end point can talk to arbitrary cluster nodes at the other end point? What would it take to give researchers access to multiple lambdas? In fact, OptIPuter partners are already working on lambda control software which will allow researchers to schedule/reserve lambdas for hours, days, or even longer.

Project team leaders presented summaries of the preceding days' work at the open house. In addition, attendees and guests were treated to the stunning sights and sounds of a 4K digital cinema experience in the Atkinson Hall theater. Several super-high-definition film segments were shown, demonstrating what will eventually become mainstream in terms of the integration of new technology with science and entertainment.

The grand finale of the week-long meeting was a 4K digital cinema experience in the Atkinson Hall theater. Film segments included the National Center for Supercomputing Applications' scientific visualization "Evolution of an F3 Tornado in a Thunderstorm," shown here.

The program began with a real-time demonstration by John Graham of San Diego State University showing GeoFusion software used to process thousands of before-and-after aerial photos of the Gulf Coast areas hit by Hurricane Katrina. While this showed the visual story of the immense damage to the area, it also provided a glimpse at how researchers working to ameliorate, access and repair damage, and/or prepare for such catastrophic events, in the future,  might access and interact with large-scale data sets over advanced networks.

The technology producing the extraordinary imagery consisted of uncompressed 4K digital movies projected by a Sony 4K digital projector powered by a Silicon Graphics Prism visualization system, streaming at 24 frames per second from an SGI® InfiniteStorage RM-660 disk array. 4K refers to 4,000 horizontal pixels x 2,000 vertical scan lines, or 8-megapixel resolution. The accompanying audio was mixed live for a spatially enhanced surround-sound experience. The 200-seat digital theater is currently configured to deliver 10 channel audio (8.2 surround sound) during 4K playback.

"The Prism visualization system is connected by 10-Gb/second networking to the global OptIPuter," explains Tom DeFanti, Calit2 research scientist, "making the Calit2 theater the most advanced digital cinema theater in operation in the world."

OptIPuter is one of the largest Information Technology Research (ITR) projects funded by the National Science Foundation (NSF). Kevin Thompson, the NSF program manager for OptIPuter, says "OptIPuter's success is evident in its adoption and use across agencies and scientific disciplines. Such broad impact is exemplary for an ITR-funded project of this scope."

Related Links

OptIPuter AHM 2006 Presentations