Slicer3:GPURayCaster

Return to Slicer Main Page
Return to Slicer 3.6 Documentation

GPU Ray Caster

This project involves moving the GPU Ray Caster from VTKEdge into VTK, then making this volume mapper available from Slicer.

Overview of GPU Ray Casting

VTK supports several volume rendering techniques for both regular, rectilinear grids (vtkImageData), and tetrahedral meshes (represented by vtkUnstructuredGrid). Some of these volume mappers primarily utilize the CPU (relying on the GPU only for the final display of the resulting image), while other mappers make use of the resources available on the GPU such as 2D and 3D texture memory and texture mapping functionality. The new vtkKWEGPURayCastMapper in VTKEdge uses the latest advancements available on recent GPUs including fragment programs with conditional and loop operations, multi-texturing and frame buffer objects in order to deliver significantly improved performance over the CPU-based ray casting, while still maintaining high rendering quality.

The basic ray casting concept is quite simple. For each pixel in the final image a ray is traced from the camera through the pixel and into the scene. As the ray passes through a volume in the scene the scalar data is sampled along the ray and those samples are processed and combined to form a final RGBA result for the pixel. The implementation of the ray casting algorithm within an object-oriented visualization system, such as VTK, that utilizes object-order rendering (polygonal projection) for other data objects in the scene is a bit more complex, requiring the ray casting process to consider the current state of the frame buffer in order to intermix the volume data with other geometric data in the scene. In our GPU implementation of ray casting, the volume data is stored on the GPU in 3D texture memory. Ray casting is initiated by rendering a polygonal representation of the outer surface of the volume. A fragment program is executed at each pixel to traverse the data and determine a resulting value that is combined with the existing pixel value in the frame buffer computed during the opaque geometry phase of rendering.

For more detailed information, please see the July 2008 edition of The Source, which can be found here: http://kitware.com/products/archive/kitware_quarterly0708.pdf.

Feature list

• Integration of the VTK GPU ray caster into the Volume Rendering module.
• High speed performance
• Flexibility to support a wide range of graphic card performances
• Interactive Framerate

VTK GPU RayCasting runs in a degraded mode to support 20fps when interacting with the view

• 2 modes of rendering: Composite and Maximum Intensity Projection

VTK GPU RayCasting in Composite mode

VTK GPU RayCasting in Maximum Intensity Projection mode

• Support scalar mapping: Opacity and Color

VTK GPU RayCasting with Custom Scalar mapping

• Clipping

VTK GPU RayCasting with clipping

• Frame/Z-buffer aware (do not hide other objects in the scene)

VTK GPU RayCasting within a scene

Integration into VTK

The GPU Ray Caster was initially funded by the following SBIR projects:

"Visualization of Massive Multivariate AMR Data" National Science Foundation Contract Number OII-0548729

"Rapid, Hardware Accelerated, Large Data Visualization" Department of Defense / Army Research Laboratory Contract Number W911NF-06-C-0179

The output of these projects was a new toolkit under GPL terms called VTKEdge. As part of this project, the GPU Ray Caster will be moved out of the VTKEdge toolkit into the BSD licensed Visualization Toolkit (VTK). In the process, the names of the classes will be changed to follow VTK conventions.

This relocation is currently in process and is expected to be complete by January 11th. The new classes will be located in the VolumeRendering kit of VTK.

Integration into Slicer

The plan is to review the current Volume Renderer module and work with Yanling Liu <vrnova@gmail.com> and Alex Yarmarkovich <alexy@bwh.harvard.edu> to integrate the new mappers to the current Volume Rendering GUI, so that they can be selected at run-time.

Result March 2010

The GPU Ray Caster has been moved into VTK 5.6 and Slicer, by shifting its official revision of VTK to 5.6, has support for the VTK ray caster (vtkGPUVolumeRayCast*).
In the Volume Rendering module, an entry has been added: VTK GPU Ray Casting.

VTK GPU Ray Casting

Open Issues

• Crash on MAC machines
  • A bug in the NVidia drivers for Apple has been reported on May 14th 2010
    • BugID:7986198 http://www.vtk.org/Bug/view.php?id=10716
  • Another bug report has been sent to Apple with more description
    • BugID: 8204106
  • To date (2010/10/13), the 2 bugs are still open (rank #1) and we didn't hear from Apple.
• When camera is inside the volume, the GPU mapper creates images that are different from the CPU mapper
  • views are clipped. See below for an example of clipping. The camera is inside the volume. This does not happen in the VTK CPU mapper.

Clipping problem when the camera is inside the volume

• • To date (2010/10/13), the problem is still open