Calit2 Researchers Work with Public Agencies to Enhance
2.4.03 - Behind the scenes and away from the glare of publicity, more than a dozen faculty and students affiliated with Calit² mounted a coordinated effort to put cutting-edge public-safety technologies in place for Super Bowl XXXVII. And they did so in a relatively concentrated period of time: barely three weeks from the first planning meeting to Game Day, Sunday, January 26. "It was a Herculean effort on the part of all our people and all their counterparts at local, state and national agencies," said Ramesh Rao, Calit²'s division director at UCSD, who oversaw the institute’s involvement. "Calit² partners stepped up to help in crucial ways at critical times to make this effort a success, and the entire exercise illustrates why San Diego has earned a reputation as a place where public and private sectors work together on solutions to important problems."
The project included newly-developed software to glean security data automatically from cameras wirelessly hooked up to the Internet. The cameras monitored the southern perimeter of QUALCOMM Stadium, a high-traffic intersection in downtown San Diego, and a waterfront command center staffed by San Diego Police and other security personnel. Separately, Calit² researchers developed a service that allowed police to call a toll-free number to get voice-activated traffic updates for major roads and highways. "While our systems played only a small part in the event’s overall security," added Rao, an electrical and computer engineering professor at UCSD’s Jacobs School of Engineering, "we were able to take experimental hardware and software code and turn them into working systems almost overnight - and they worked flawlessly!" San Diego spent roughly $2 million on security for the game (with half of that coming from the National Football League), including the cost of equipping QUALCOMM Stadium itself with more than 50 video surveillance cameras. But the City invited public agencies, private companies and research organizations to contribute - at their own cost - backup systems outside the stadium that could also be monitored by police and security personnel. "“It is remarkable how quickly and successfully San Diego technology institutions pulled together on this," said Calit²'s Rao. "And it would not have happened without the leadership of Assistant Chief of Police Bill Maheu, who was in charge of Super Bowl security, and of Bob Davis, a police advisor who coordinated the participation of technology providers."
Rao led the institute's effort, together with Calit²'s Intelligent Transportation and Telematics layer leader at UCSD, Mohan Trivedi. A Jacobs School professor, Trivedi and a team of 13 students and researchers from UCSD'’s Computer Vision and Robotics Research (CVRR) lab installed a variety of cameras, software and other equipment to capture and analyze video streams in real time over the Internet – turning the cameras into 'smartcams.' "On a temporary basis, we deployed roughly $150,000 worth of cameras, video server, computer and large disk storage equipment that we only have because of the support of agencies that fund our research on computer vision and homeland security," said Trivedi. "And even though most of the systems were up for just the weekend, we could only be successful because of the nearly four years of research - and funding - that allowed us to develop these technologies." Among the agencies sponsoring computer vision-related work in Trivedi's lab: the U.S. Defense Threat Reduction Agency, as well as the Technical Support Working Group, an inter-agency forum that coordinates R&D for combating terrorism; Space and Naval Warfare Systems Command (SPAWAR); Center for Commercialization of Advanced Technologies; University of California-based Industry-University Cooperative Research Program; Caltrans; Office of Naval Research (ONR); National Science Foundation’s Information Technology Research program; and the William J. von Liebig Center for Entrepreneurism and Technology Advancement. Trivedi's team of researchers included Ofer Achler [a Calit² undergraduate fellow in 2001], Jun Wen Wu [Calit² graduate fellow in 2001-02], and Dashan Gao [who is a Calit² graduate fellow in the current 2002-03 academic year]. The CVRR researchers deployed cameras and other equipment in differing configurations at multiple locations, with all feeds viewable - and, in some cases, customizable - over the Internet, from two command centers (one at San Diego Downtown, the other at San Diego's Seaport Village.) The research team also developed a series of software modules tailored to the needs of law-enforcement authorities and security personnel. Most were based on previous computer algorithms developed in the CVRR lab but required tweaking to fit the occasion. "Our algorithm development team was working almost 20-hour shifts all the way up to kickoff time," said Trivedi. "They were able to develop various applications for robust event recognition and event database generation." The four locations, and some of the hardware/software capabilities installed at each, included:
1. Riverbed/QUALCOMM Stadium Perimeter: The southern perimeter of the stadium'’s property borders on the San Diego River channel. To monitor possible intrusions and activity in the area, Trivedi deployed two cameras on a 60-foot tall cherry-picker supplied by UCSD Physical Plant Services, and positioned just south of the riverbed on an SDSU Foundation site that was a staging ground for SDSU's major ShadowBowl exercise (http://www.shadowbowl.org). "High-speed Internet connectivity and power was also provided by SDSU," said Trivedi. "We are indebted to Dave Warner's group that was staging the ShadowBowl activity, and to SDSU professor Eric Frost, one of our key partners in Calit², who helped establish the technical and logistical interaction." The equipment installed atop the crane included a high-resolution thermal imaging system able to detect humans (and animals, including several jackrabbits!) in unlit and visually cluttered scenes near the riverbed. CVRR researcher Ed Childers designed a specialized housing, optical filter and mounting assembly for the thermal imaging system. A team of technicians at the Scripps Institution of Oceanography's Engineering Shop developed the housing assembly within a day. Researchers also deployed a high-resolution, pan-tilt-zoom camera on top of the boom truck. It provided a live video feed at 30 frames per second, covering from the south side of the riverbed all the way to the stadium gates. The system provided temporally and spatially registered multimodal (thermal 8- 14 um and visible .4-.7 um) video from the site. "This is a very unique feature to allow development of surveillance systems which can operate day and night,"said Trivedi. "And law-enforcement officials were able to operate the camera over the secure Internet from their multiple locations." 2. Friars Road: Tapping into a video camera already monitoring traffic activity on the road in front of QUALCOMM Stadium, CVRR researcher Ofer Achler ran the video feed through the latest version of his experimental vision-based traffic analysis and notification software. The program uses algorithms to count cars and vehicle speeds, and worked in both day and night conditions. The live traffic flow information was also fed to a separate traffic monitoring system developed by Calit² researcher Ganz Chockalingam (see below).
3. Gaslamp District: San Diego Police and city technicians mounted a CVRR omni-directional camera with outdoor housing, axis server, power reset module, and wireless antenna on a traffic signal pole at the intersection of Sixth Avenue and G Street in downtown San Diego’s historic Gaslamp District. The real-time, 360-degree video was uploaded to the over a high-speed 802.11b-based wireless network provided by Sky River, a provider of broadband wireless communications for business. Noted Trivedi: "Sky River helped generously with access, network provisioning and installation support - and under very tight time constraints." Authorities were able to view the live panoramic video, as well as corrected (unwrapped) video views, and were able to customize their views of the video stream. That capability was available not just from both command centers, but also using a handheld Digital Tele-viewer (developed in Trivedi's lab) that was hooked up wirelessly to the Internet; it allowed officers to select pan, tilt, and zoom parameters for custom viewing. Trivedi also installed a "dynamic crowd status analyzer and recorder," using input from the Gaslamp omnicam. The brand-new software module used pattern recognition to estimate the size of the crowd gathered in the secured "party" area of the district, as well as the flow of pedestrians on both Sixth and G streets. The numbers were updated every two minutes in a database for query and analysis. The program was designed at CVRR to work in the daytime as well as with night-time illumination. "A number of officials indicated long-term interest in this program as a potentially very important tool for emergency response teams," noted Trivedi.
4. Seaport Village: To cover perimeter security around the command center building set up on the waterfront in Seaport Village (overseeing security operations in the village and in the Gaslamp District), the CVRR team installed a high-resolution rectilinear camera with full pan-tilt-zoom controls operated by authorities over the Internet. The video feed, at almost 30 frames per second, was also fed through a "virtual fencing and vehicle/person detector." Using the high-resolution feed, the software module detected and counted vehicles and pedestrians around the Seaport Village command center. All the information was also recorded and logged for later analysis. 'Powered by Calit²' Even on the Web, Caltrans only provides traffic flow information for major freeways in San Diego. But Calit² was already installing a camera to monitor traffic on Friars Road in front of the stadium. Using vision-based software developed in Mohan Trivedi's lab (see above), a computer estimated traffic flow on the road, and fed that information to the voice-assisted traffic service. Police could then call in from anywhere, ask for "Friars Road East," and get an update such as 'Traffic on Friars Road East is flowing at 20 miles per hour.' Notes Chockalingam: "The San Diego Police Department used the system to help manage and route traffic around QUALCOMM Stadium." The service, "Powered by Calit²," went online a few days before the Super Bowl, and has since been made available to all San Diegans, free of charge. "We quickly realized that this service can be a great tool for commuters, so we created a second toll-free line for the public at large," said Calit²'s Ramesh Rao. "This is a very tangible way in which innovation happens: we were working on the technology, and the security and safety requirements for of the Super Bowl made it important that we push this technology into the real world sooner than we had originally planned. Now, for the foreseeable future, we will continue to provide this service at no cost to commuters." The toll-free number is 1-866-500-0977, and the service operates around the clock. Regional Network for Homeland Security Going forward, Calit² researchers see the institute's participation in Super Bowl security efforts as a starting-point for future collaborations. "To give you one example," said Trivedi, "law-enforcement officials from Jacksonville saw some of our systems working in the Command Center and are talking to us about this capability for when they host the Super Bowl. I am extremely proud of our team. Without everyone's professionalism, dedication, technical competence, creativity, teamwork, and 'can-do' attitude, we could never have deployed so much technology so quickly." The effort also allowed the institute to engage new partners, including companies such as Sky River and WFI, a San Diego-based company specializing in network architecture and dimensioning of mobile and high speed wireless data systems. Concluded Calit²'s Rao: "This event gave us a chance to reach out not just at the agency level, but to first responders, who were able to give us feedback on the spot about how the technology could best serve their needs. We expect to continue getting valuable feedback from everyone who participated in this effort, and that feedback will allow us to develop future enhancements of these technologies - and get them deployed faster thanks to these new relationships that are being forged." |
Related Links
Computer Vision and Robotics Research Lab http://cvrr.ucsd.edu
San Diego State University http://www.sdsu.edu/
UCSD Jacobs School http://www.jacobsschool.ucsd.edu/
San Diego Supercomputer Center http://www.sdsc.edu/
San Diego Police Department http://www.sannet.gov/police/
ShadowBowl 2003 http://www.shadowbowl.org
County of San Diego Emergency Medical Service http://www2.sdcounty.ca.gov/hhsa/
Sky River http://www.skyrivercomm.com/
WFI http://www.wfinet.com/
Defense Threat Reduction Agency http://www.dtra.mil
Technical Support Working Group http://www.tswg.gov
Space and Naval Warfare Systems Command http://www.spawar.navy.mil
Center for Commercialization of Advanced Technologies http://www.ccatsandiego.org
UC Industry-University Cooperative Research Programhttp://uc-industry.berkeley.edu/
Caltrans http://www.dot.ca.gov
Office of Naval Research http://www.onr.navy.mil
NSF ITR Program http://www.itr.nsf.gov/
William J. von Liebig Center http://www.vonliebig.ucsd.edu/
Qualcomm Stadium http://www.sannet.gov/qualcomm/
UCSD Physical Plant Services http://pps.ucsd.edu/
SDSU Foundation http://www.foundation.sdsu.edu
Scripps Institution of Oceanography http://sio.ucsd.edu