On the Trail of 3D Images

by Rachael Brady and Ginny Hudak-David

What do horse fetlocks, sperm tails, and chick embryos have in common? They're all being measured and studied in NCSA's virtual reality environment using software developed by the Biological Imaging Group at NCSA. The software system, named Crumbs, was developed in collaboration with the Beckman Institute and UIUC Department of Cell and Structural Biology. Crumbs marks a trail in 3D instrumental datasets.

NCSA Senior Research Programmer
Rachael Brady

As in the fable of Hansel and Gretel, Crumbs lets scientists explore paths or fibers in their datasets and drop bread crumbs, or markers, along the way to mark their trails. Paths can be measured and saved for quantitative analysis using NCSA's CAVE(tm) (Cave Automatic Virtual Environment). Crumbs facilitates single-fiber tracing in low contrast, dense, volumetric image datasets. Scientists immerse themselves inside a data volume, using Crumbs rendering tools to view specific regions and drop markers along a structure of interest. NCSA Crumbs developers include Rachael Brady, John Pixton, George Baxter, Patrick Moran, and Clint Potter. Bridget Carragher of the Beckman Institute and Andrew Belmont from the UIUC Department of Cell and Structural Biology are also members of the Crumbs team.

Veterinary research tool

Mark Martinelli, a veterinary surgeon formerly with the UIUC Department of Veterinary Clinical Medicine, is using magnetic resonance imaging (MRI) to study the anatomy of the equine metacarpophalangeal joint -- the fetlock, located at the back of the leg above the hoof (see figure below). The pathology of the musculoskeletal system of the equine athlete is considered one of the leading problems limiting performance.

A low-resolution volume rendering of an entire horse fetlock dataset. A full-resolution rendering of a subvolume is shown in the smaller box.

Martinelli's goal is to use 3D rendered images from MRI datasets to produce detailed anatomical descriptions of the equine metacarpophalangeal joint, especially articular cartilage, tendons and ligaments, and bone marrow. Creating anatomical descriptions of these 3D structures is difficult using traditional 2D displays of slice data or rendered volumes. Although Martinelli and other members of his group have spent considerable time trying to achieve the necessary segmentation using traditional volume analysis packages, their efforts have met with little success.

Segmentation was quickly and accurately achieved using the Crumbs environment. After Martinelli's MRI-generated fetlock data were put into the CAVE, he learned the Crumbs program and traced the cartilage in a single morning. The points marked using Crumbs will be used to generate a geometric model of the cartilage that can then be visualized in the context of the fetlock as a whole. These techniques -- segmentation with Crumbs as well as model generation -- have great potential for both research applications and for the creation of effective teaching tools.

Flying into basic research

Timothy Karr, a University of Chicago biologist and a faculty member of the neuronal pattern analysis group at the Beckman Institute, studies the fate of Drosophila (fruitfly) sperm following fertilization.

Little is known about what happens during and just after fertilization, the time when a sperm enters the egg. Karr's basic research program uses the fruitfly as a model, but his results could be applied to mammalian or human fertilization questions. This summer Karr discovered "a whole new area of biology" while working with Crumbs.

Drosophila wassermani sperm tail. The entire sperm tail (red) has been traced with Crumbs software.

Karr developed antisperm antibodies to use with confocal microscopy to image sperm lengths in twelve species of Drosophila. Four species were traced and measured in the CAVE. The complexity of the sperm structures traced using Crumbs is apparent in the figure above. The extreme length and compaction of the sperm in the egg prevented measuring its length using a standard workstation environment. He is excited about the contributions Crumbs is making to his research.

"The CAVE lets me look at things I could not have seen before, and Crumbs lets me trace the path of the sperm inside the egg. Think about trying to follow an unraveling ball of yarn inside an egg, and you have an idea of how difficult this would be without software like Crumbs," explains Karr. "The CAVE's 3D environment was needed to trace the entire length of the sperm. We made some major strides this summer at NCSA." The ability to trace the 3D structure of supramolecular complexes like sperm inside the egg will continue to provide unprecedented new information about structure and function of complex biological systems not previously amenable to experimental examination.

Karr plans to extend these studies to mammalian fertilization and to use the CAVE to carefully compare the 3D shape of sperm in various species of Drosophila.

Tracing the heart's development

Jo Ann Eurell, associate professor in the UIUC Department of Veterinary Biosciences, and graduate student Janet Sinn- Hanlon recently reconstructed a 48-hour-old chick embryo using serial section light microscopy (below). They examined the chick embryo in the CAVE and used Crumbs to trace the path of blood flow through the heart while the heart was changing from a straight tube to a four-chambered structure. The tracing will be used to help students conceptualize how the tube bends during the embryo's development.

Brady traces a chick embryo inside the CAVE.

Another feature of the chick embryo dataset that they were able to trace was the fusion of the dorsal aorta. This vessel is a paired structure during early development, but, as development progresses, it begins fusing into a single structure midway down its length. Eurell and Sinn-Hanlon were able to trace the paired vessel to its fusion site and then back again to a paired structure as they moved from the embryo's caudal (tail) region to its head region. Long-range plans for the project include the visualization of more datasets of earlier and later embryos (24-hour-old chick and 10-mm long pig). Each dataset has features that are difficult for students to conceptualize using current teaching methods. Currently students have to imagine a 3D structure using serial sections of the embryo as viewed individually under the microscope. Seeing 3D representations of anatomical features will be a unique "field trip" for students.

The human angle

David Kuehn, with colleagues Sandy Ettema and Lou Tomes of the UIUC Department of Speech and Hearing Science, turned to Crumbs this fall to measure the insertion angle of the Levator Veli Palatini (LVP) muscle into the soft palate of the human pharynx. Their ultimate goal is to help surgeons reconstruct children's cleft palates by simulating the attachment point of the LVP muscle on the cleft palate and testing that the palate closes properly.

Supercomputing '95

Crumbs was designed to visualize instrumental datasets, which tend to be large and often contain noise (like static on a radio) that can obscure the structures researchers are interested in analyzing. So Crumbs incorporated online image processing algorithms, such as convolution and media filtering, in a remote server that can also connect to distant instrumentation, letting scientists steer their experiments from inside the CAVE. At Supercomputing '95 the Crumbs development team will perform remote control of experiments on three instruments: an MRI system, a Scanning Tunnelling Microscope, and an electron microscope. The CAVE -- part of the GII Testbed event and connected to the I-WAY -- is a showpiece at SC'95.

In preparation for SC'95, the NCSA research team recently used a modified MRI system to test the distributed remote data acquisition interface of Crumbs. To demonstrate the capability, they put an earthworm in a soil sample and imaged it using an MRI system that sent a new volume to the Crumbs remote server every 30 seconds. The server forwarded the image to the CAVE, taking 2 to 3 seconds to update the entire data volume. The Crumbs package tracked the worm's structure, labeling its location, so changes in the worm's position were easily detected.

Further refinement of Crumbs is anticipated in 1996 as additional collaborations take place in other disciplines.


Rachael Brady is a Senior Research Programmer in the NCSA Biological Imaging Group.
Ginny David is an editor in the NCSA Publications Group.

Return to the Table of Contents.

NCSA Home Page


access / Fall 1995 / Email comments to NCSA Publications Group: pubs@ncsa.uiuc.edu