Calit2 Participates in 'Quantified Surgery' at UC San Diego

San Diego, March 9, 2017 — Turning a fairly routine procedure into a bold experiment, Calit2 Director Larry Smarr recently underwent a segmental colon resection (removing an eight-inch section of his large intestine) prefaced by the creation of dramatic, data-driven computer models of key organs in his abdomen. Both before and during the surgery, UC San Diego School of Medicine surgeon Dr. Sonia Ramamoorthy had access to these 3D visualizations made possible in conjunction with Calit2's virtual reality team. In an article published in today's This Week @ UC San Diego, "Visualizing the Future of Surgery", physicians and technologists talk about the operation and how this 'quantified surgery' translational pilot can impact the future of abdominal surgery. (Click here to read the This Week article on the new School of Medicine Quantified Surgery site, or click here to watch a four-minute video about the collaboration.)

To create a volumetric visualization of Smarr's abdomen,
QI researcher Jurgen Schulze stacked up
96 MRI DICOM images, creating a 3D volumetric data set
out of them, which is then loaded
as a 3D texture into the graphics processing unit (GPU) card.
The GPU card then renders it as a volume
using an interactive transfer function editor developed by Schulze,
which he uses to define which parts be translucent, which opaque.
To watch video of Schulze in QI's StarCAVE VR environment
interacting with the 3D volumetric visualization,
then creating the 3D organ segmentation model,
[click here] or click on Related Link for
Video: 3D Model and Organ Segmentation

What follows provides more detail on the technologies that make quantified surgery possible, and the role played by Calit2's Qualcomm Institute (QI) personnel before, during and after the operation.

From its beginning in 2000, Calit2 had a focus on Digitally Enabled Genomic Medicine, which foresaw a future in which a combination of self-tracking devices combined with analysis of the human and later microbiome genomes, would become pervasive in maintaining health. To get an early view of what such a future would look like, Calit2 director Smarr began developing time series of a wide variety of measurements on his body, including blood and stool samples, in the mid- 2000s.

Although in the beginning he was primarily focused on increasing exercise and modifying his nutrition to become healthier, he began to see that both blood and stool biomarkers indicated he was developing chronic inflammation which would episodically peak at levels far above normal. In summer 2011, when Smarr became the patient of UC San Diego’s Dr. William Sandborn, a national leader in clinical research on Inflammatory Bowel Disease (IBD), Smarr was surprised to find that one of his stool sample immune biomarkers had jumped to 125 times above the normal healthy upper limit. On reading the research literature Smarr found that this biomarker was a sensitive and specific indicator that the patient is suffering from active IBD. After a colonoscopy and CT abdominal imaging, Dr. Sandborn concluded that it was likely that Smarr was suffering from colonic Crohn’s disease, a subtype of IBD.

To print the plastic model on a 3D printer,
QI's Philip Weber used the National Library of
Medicine’s Insight Segmentation and Registration Toolkit to segment out the colon
from the rest of the MRI data
and turn it into a polygonal mesh.
This mesh was imported into MeshViewer,
an open-source viewing and editing tool
to clean up unwanted data around the colon,
cut out the part of the colon to be 3D printed,
and export the mesh of an STL file,
which the 3D printer can load.

Sandborn followed up with an MRI of Smarr in January 2012, after which Smarr asked for the DICOM files of the MRI slices and gave them to Qualcomm Institute Research Scientist Jurgen Schulze to convert the MRI slices to a 3D image of his abdomen. Schulze stacked the MRI DICOM images to create a 3D volumetric data set that could become transparent under interactive control (see figure above).

Schulze then went one step further and had his team create an accurate physical model (at left) of the diseased portion of his sigmoid colon by sending the coordinates of the 3D surface of the virtual sigmoid colon to a 3D printer.

Fast forward to 2016. After continuing to track his health in minute detail, Smarr had plenty of evidence that his Crohn’s disease was worsening. Indeed, the wall thickness of his sigmoid colon had, over the course of eight years, become increasingly thickened. In June 2016 Smarr traveled to Irvine, CA to have a millimeter-resolution full-body scan from Dr. Harvey Eisenberg, a pioneer of this technology. Dr. Eisenberg also performed a virtual colonoscopy (see picture below) in which the colon is inflated with air and then imaged with the CT scan. Remarkably, it showed that in the inflamed portion of the sigmoid colon the walls had become so thick (seven times normal thickness) that the air could barely get through. Dr. Eisenberg recommended a resection of the inflamed region.

Virtual colonoscopy
by Dr. Eisenberg
shows area of blockage
in Smarr's sigmoid colon.

Finally on August 26, 2016 Smarr came to the conclusion that his worsening condition could lead to an emergency surgery while he was on travel, so he emailed his GI doctor William Sandborn that he believed the time had come for the laparoscopic removal of his sigmoid colon with its kink.

With this possibility in mind, Smarr met with Dr. Catherine Mohr, VP of Strategy for Intuitive Surgical, at the Exponential Medicine Conference in San Diego, who showed him their third generation da Vinci Xi surgical robot, which was optimized for colorectal surgery. Knowing the UC San Diego had the da Vinci Xi, Smarr realized that the software of the da Vinci would allow for a 3D interactive model of his abdomen to be easily introduced into the da Vinci robot.

After talking with Smarr, Dr. Sandborn ordered a new MRI and colonoscopy and Dr. Ramamoorthy set a date, November 29, 2016, for the surgery. By early that month, an MRI and colonoscopy revealed the diameter of the channel ("the lumen") of the sigmoid colon had been reduced to 4mm from the normal 40mm.

“As long as I was going to go through surgery, I decided to turn it into a translational medicine pilot for what might be called ‘quantified surgery’,” observed Smarr.

(Left) Pre-surgical visualization of Smarr's gut with
large intestine (green); (right) one month post-surgery.
To watch video of before and after
side by side (yellow oval shows portion resected:
[click here]. or on Related Link for
Video: 3D Models: Before and After Surgery

Three weeks before the surgery, Smarr worked with the surgeon to host a pre-surgery planning meeting with about 16 physicians, engineers and scientists from surgery, radiology, GI, osteopathy, bioengineering, robotics, design and virtual reality. Smarr noted that physicians typically do not hold cross-disciplinary meetings for individual patient pre-surgical planning, but, he said, they were “quite excited to share their different insights.”

Then, in the two weeks leading up to the surgery, Smarr pulled together his own team at Calit2 to support the pre-surgery planning. Calit2 Research Scientist Jurgen Schulze took Smarr’s standard two-dimensional CAT and MRI scans and turned them into 3D volumetric computer models and organ segmentation visualizations. They then shared this 3D individualized "Google Map" of Smarr’s gut with the surgeon in pre-surgical planning.

“Essentially, our team applied software to the existing MRI digital inforrmation that was stored in the imaging section of my electronic medical record," noted Smarr. “By transforming the digital data into an intuitive, interactive 3D image, the surgeon was able to easily visualize the specific characteristics of Smarr’s internal anatomy before the patient was on the table and opened up.”

Surgeon Sonia Ramamoorthy
views 3D MRI-based colon image
in QI StarCAVE pre-surgery.

Smarr and his medical team, including UC San Diego surgeon Sonia Ramamoorthy, discovered three key factors that would prove to be critical during the surgery: 1) the sigmoid colon had major ‘kinks’ at both ends; 2) the sigmoid was sitting on the bladder; and 3) the upper part of the large intestine had three turns at the splenic flexure and was likely attached to the spleen.

“Dr. Ramamoorthy had never had this level of anatomical planning in her 20-year career and was so impressed, she drafted Jurgen Schulze into the operating room,” recalled Smarr. “She had him pipe our 3D anatomy into the da Vinci surgical robot, so she could see both the internal stereo laparoscopic camera view of me and the 3D anatomy as a guide.”

(Subsequent to the surgery, Dr. Ramamoorthy invited Smarr to give a talk to the doctors attending the Surgery Grand Rounds about making the leap. Click here to watch a video of Smarr's Grand Rounds presentation. Or click here to download his Powerpoint slides from the presentation.)

“The technical operator of the Da Vinci system
was able to add my laptop’s image to
the robot’s camera video feed whenever needed,”
explained QI's Schulze. “The TilePro’s output
went into a video management system from Stryker,
which allowed the recording of the video
as well as additional flexibility in the way the videos
could be displayed on the monitors in the OR.”
(To get a feel for what it was like in the operating room,
click here to watch a six-minute video of scenes
from the OR, including quotes from
Dr. Santiago Horgan, Chief, Minimally Invasive Surgery,
and William Sandborn, Chief of Gastroenterolody.
[Videography by Mike Sterner/Visual Media Group]

The da Vinci TilePro system permits viewing of the feed from the surgical camera on external monitors, which also permit viewing of additional external video sources. Once in the operating room, Schulze connected his laptop to the TilePro via HDMI and treated the da Vinci system as if the laptop were connected to a projector for a slide presentation.

Surgery was scheduled for 7:30 a.m. on Nov. 29, and Smarr notes that the team was only able to test their combined visualization system the previous afternoon.

“UC San Diego Health Sciences was so excited about this translational medicine pilot that they put a full video crew into the operating room,” noted Smarr.

Post-surgery, Smarr was in the Jacobs Medical Center for four days and then at home for several weeks for rehabilitation. “Fortunately, I have had very little post-surgery pain, my new colon started up in two days and is working well, and I got back to a normal diet in a week,” explained Smarr. “Since the docs said walking was the best therapy, I started the evening after the surgery – dragging my IV pole and oxygen tank down the hospital corridors.”

Within two weeks, Smarr had ramped up to walking more than five miles per day. Having lost the diseased eight inches of his sigmoid colon, for the first time in 12 years, Smarr’s CRP dropped below 1 and his Lactoferrin value returned to normal.

Jacobs Medical Center in La Jolla,
where the surgery and recovery took place.

"My greatest hope is that through my pilot experiment these consumer-driven visualization technologies will become a commonplace part of abdominal surgery in the near future," said patient Smarr. "We will do our best at UC San Diego to lead that change."

Calit2's Qualcomm Institute is now in discussion with UC San Diego Health and units on campus, including the QI Design Lab (directed by Don Norman), on forming a cross-disciplinary team to extend Smarr's pilot project to more patients. As is characteristic of Calit2, this team is hoping to include private sector companies such as GE Healthcare, which is currently exploring with UC San Diego future-oriented imaging and intervention paradigms that assimilate new technology with human-centered design processes.

Media Contacts

Doug Ramsey, 858-822=5825, dramsey@ucsd.edu

Newsroom > News Release

Calit2 Participates in 'Quantified Surgery' at UC San Diego