Giving Students a Place to Prep for Tomorrow's Virtual (Reality) Economy

Computer science department opens high-tech teaching lab for undergraduates

San Diego, May 11, 2017 — It’s a laboratory that looks like a cross between a classroom and a tech pavilion at the annual Consumer Electronics Show in Las Vegas. There are virtual-reality headsets everywhere, and large, flat-screen 3D displays.  College students work at computers, while teammates wearing goggles look from side to side, occasionally ducking or recoiling, as they react and engage with the virtual environments visible in their head-mounted displays.

Welcome to the Virtual Reality Lab – the first of its kind at an American university. 

The University of California San Diego officially opened its VR Lab in early May with an open house. Located in the Computer Science and Engineering (CSE) department, the lab was created specifically to serve undergraduate students who take courses in software programming for virtual and augmented reality.

“Other universities may have labs that serve faculty and graduate students doing research in the field of virtual reality and they may have a few VR systems for research use only,” said CSE Chair Dean Tullsen. “But to our knowledge, the VR Lab is the first designed primarily for undergrads learning how to create VR content.”

According to computer science junior Connor Smith, president of the student-run Virtual Reality Club at UC San Diego, the very existence of the VR Lab has galvanized interest in virtual reality across the campus. “We now have the biggest VR classroom in the country,” he said. “It’s really cool to say our school has this, and it has multiplied the VR community on campus.” Indeed, in the past six months membership in the VR Club shot up from 30 to more than 300 members. (See sidebar for more on the VR Club.)

The lab was two years in the making. It is equipped with 25 standalone systems combining an Oculus Rift head-mounted display, two controllers for navigating inside VR environments, a computer workstation equipped with a high-end graphics card, various peripherals and a custom-made breakout box for plugging in and unplugging cables. “This way students don’t have to get on their hands and knees under the table every time they have to plug in the Oculus Rift,” noted the VR Club’s Smith.

With 10 more Oculus Rifts with controllers about to be added to the lab, the VR Lab now accommodates classes up to 70 students. “There will be a full set of equipment for each team of two students, as long as all teams don’t show up at the same time,” said CSE adjunct professor Jurgen Schulze cautiously. “When that happens, two teams may have to alternate using a single PC, or bring a laptop computer, and we just have to hope they won’t conflict too much as they develop their respective projects.”

Schulze – a research scientist and VR expert in the Qualcomm Institute – co-led the project with Music professor Shahrokh Yadegari, who also directs the Sonic Arts Research and Development group in the Qualcomm Institute (QI). 

In 2015 Schulze applied for a grant from the campus-wide Integrated Digital Infrastructure (IDI) program, which earmarked funds for technologies to support the university’s teaching mission. IDI provided $120,000 to purchase computers and other hardware, including 3D TVs, 360-degree cameras, Leap Motion hand-tracking devices, an HTC Vive VR system, and 40 Google Cardboard-compatible VR viewers (for viewing VR content with a smartphone). Information Technology Services (ITS), UC San Diego’s central IT organization, provided 26 graphics PCs, and Oculus gave the lab 10 Oculus Rift headsets.

CSE covered Schulze’s time on the project, and the department picked up the tab for improvements to the space, including all new furniture and lockers for the VR devices. “CSE made the biggest contribution,” observed Schulze, “because without the space itself, creating a standalone VR Lab would not have been workable.”

Some of the grant from IDI also went to the Qualcomm Institute’s Shahrokh Yadegari. He and students created a spatial audio library – i.e., a library of surround-sound audio processing routines. The library makes it much easier for students to add spatialized audio to the interactive visual experiences they create.

Yadegari then tasked his students, including electrical engineering graduate student Grady Kestler, to package the library for uploading to VR Lab computers. “We fully implemented the library as an application programming interface [API], and Professor Yadegari asked me to port the implementation to an audio server,” explained Kestler, who also holds an MFA in Sound Design from UC San Diego. “We are still adding new functionality, but by the beginning of next year, we’ll have it installed on all of the computers in the VR Lab. Students will be able to run the API to add spatialized audio effects to any VR programs they’re creating.”

“This was almost like packaging a new product for the educational market,” said Yadegari. “The library will allow students to create very realistic audio-visual VR experiences – even if they don’t have a background in sound design for games or virtual reality.”

The VR Lab is roomy enough to comfortably support a maximum class size of 70 students.  “This is what sets this lab apart from research labs,” said Schulze. “With 35 sets of VR hardware and software, each team has a complete VR system to work with.”

The opening of the lab coincides with the launch of new and updated courses in the CSE department that integrate VR into the computer-science curriculum. This spring, Schulze is teaching 70 students about Virtual Reality Technology (CSE 190), with an emphasis on immersive VR systems, including those based on head-mounted displays. Students are required to submit three separate VR projects during the quarter.

In the winter quarter, Schulze used a not-yet-fully-outfitted VR Lab to teach 3D User Interaction (CSE 165), requiring students to develop interaction techniques for the Oculus Rift and other VR devices. The course previously focused on user interaction design, with less emphasis on 3D.

Two students who took Schulze’s CSE 165 course in the winter quarter are graduating with computer-science degrees this summer. Both demonstrated VR apps at the open house, and both say they hope to find jobs that will allow them to use their new-found VR skills. “I never did VR before taking CSE 165 last quarter,” said Sun, a fifth-year computer science major. “The transition was not terribly difficult, especially with a great teacher like Professor Schulze. Now I have my VR app to show potential employers when I apply for a job.”

For his part, computer-science senior Ty Stahnke opted to convert a 2D rhythm game he was working on into a 3D environment, which he demonstrated at the open house. “A lot of what we learned in that course was how to make appropriate 3D interfaces for the user so the interaction is more intuitive and easier to play,” said Stahnke. “I definitely want to work in VR but don’t know if I want to go into entertainment, or medical, or another area. I guess I’ll apply to a lot of different VR-related jobs.”

“It was an awesome course and it was essentially new, because previously it didn’t have the VR aspect,” added Stahnke. “We got a ton of hands-on experience, but at the same time, we learned a lot about the concepts and what goes on behind the scenes when you develop VR content.”

This fall, CSE assistant research professor Nadir Weibel will teach Ubiquitous Computing (CSE 118), and he plans to use the VR Lab to give students the ability to understand the limitations and opportunities of virtual reality and augmented reality, focusing on a ‘mixed reality’ surgery app he and his students are developing.  At the open house, Weibel and first-year Ph.D. student Danilo Gasques-Rodrigues showcased a mobile augmented-reality app for surgeons designed to let surgeons “see” inside a patient on the operating table. At the open house, Weibel and Gasques-Rodrigues demonstrated an early prototype of the app on a Microsoft HoloLens system. It uses mixed reality – i.e., real medical scans of the patient draped over the patient in 3D – giving the surgeon with goggles a kind of “x-ray vision”.

“The field of view is still too narrow for what’s needed in the operating room, but we think that problem can be solved,” said Weibel in demonstrating the system. “The real question is whether mixed reality can be helpful to surgeons, for example when doing a biopsy. The jury is out, but we are working very hard to get the technology to the point where it will make an appreciable difference in surgical outcomes.”

CSE is also making the VR Lab available for undergraduate courses in other departments. In the spring quarter, Visual Arts professor Benjamin Bratton is teaching a course on Design Research Methods (VIS 135), including data visualization. Bratton is now scheduling time for students in the VR Lab to introduce them to VR applications and modeling tools.

Last quarter Music professor Shahrokh Yadegari – whose team developed the surround-sound audio library for the lab – taught Music Technology (MUS 176), focusing on spatial audio and VR applications. Students were given access to the lab to learn how to combine audio and video for the total VR experience. At the open house, Yadegari’s graduate student Grady Kestler demonstrated an interactive environment put together by combining visual effects with spatialized sound effects.

CSE professor Rajesh Gupta approved the plan to build the VR Lab in 2015 while he was department chair. “I wanted the space to be used for undergraduates and for teaching,” recalled Gupta. “When it became clear that CSE could create an entirely new facility with support from other partners on campus, we knew that the VR Lab would be a good fit for CSE’s future.”

Looking to the future, Schulze is revamping his Computer Graphics course (CSE 167) to include a hands-on component applying graphics to 3D virtual environments. He is also adapting his VR curriculum to learners around the world – through the first-ever (recently announced) series of online courses on VR App Development over the edX learning platform (jointly with CSE professor Ravi Ramamoorthy, Director of the Center for Visual Computing). The first course launches May 15.

Schulze also expects faculty in other departments will want to integrate the VR Lab into what they teach.  For example, Visual Arts professor Ricardo Dominguez will use the lab next year for his Virtual Environments (ICAM 120) course. In the past, the course focused largely on concepts, but Dominguez now sees a way to give his students first-hand experience with creating content for VR and augmented reality to reinforce the underlying theory.

In another sign that virtual reality is taking hold at UC San Diego, one of the hottest recent tech startups created by university alumni is a VR company that has hired 80 percent of its software development staff from among CSE graduates. Nanome, Inc., is winning awards and prizes for bringing VR tools to education in math, chemistry and other fields under its CEO, UC San Diego nanoengineering alumnus Steven McCloskey (B.S. ’15). The VR Lab and new VR courses are expected to funnel many more students to follow in the footsteps of Kai Wang. After working for nine months as an intern with Nanome, Wang (B.S. ’17) graduated in the winter quarter. “I worked with Jurgen Schulze on a VR project before graduating,” noted Wang, who demonstrated Nanome VR applications at the May 2 open house. “Before I graduated, I met Steve McCloskey, who was looking for interns to work on the VR software application, and I joined the company full-time after graduation.”

The VR Club’s president, Connor Smith, believes that the university can train many more students to work in this field. “The sky’s the limit,” he enthused. “I’m still not decided on a particular job after graduation, but I’ve even thought about doing a startup in VR, particularly using it for educational programming where it can make a big difference.”

Meanwhile, UC San Diego may not have the only undergraduate VR teaching lab for long. According to Schulze, UC Berkeley is now developing a similar classroom, and as more and more jobs open up in the emerging VR industry, it’s likely that other universities will follow suit. 

Related Links

Virtual Reality Club
Sidebar release about VR Club growth
CSE 190 Spring 2017
CSE 165 Winter 2017
Computer Science and Engineering
Qualcomm Institute

Media Contacts

Doug Ramsey, (858) 822-5825, dramsey@ucsd.edu