Slicer3:GPURayCaster
GPU Ray Caster
This project involves moving the GPU Ray Caster from VTKEdge into VTK, then making this volume mapper available from Slicer.
Overview of GPU Ray Casting
VTK supports several volume rendering techniques for both regular, rectilinear grids (vtkImageData), and tetrahedral meshes (represented by vtkUnstructuredGrid). Some of these volume mappers primarily utilize the CPU (relying on the GPU only for the final display of the resulting image), while other mappers make use of the resources available on the GPU such as 2D and 3D texture memory and texture mapping functionality. The new vtkKWEGPURayCastMapper in VTKEdge uses the latest advancements available on recent GPUs including fragment programs with conditional and loop operations, multi-texturing and frame buffer objects in order to deliver significantly improved performance over the CPU-based ray casting, while still maintaining high rendering quality.
The basic ray casting concept is quite simple. For each pixel in the final image a ray is traced from the camera through the pixel and into the scene. As the ray passes through a volume in the scene the scalar data is sampled along the ray and those samples are processed and combined to form a final RGBA result for the pixel. The implementation of the ray casting algorithm within an object-oriented visualization system, such as VTK, that utilizes object-order rendering (polygonal projection) for other data objects in the scene is a bit more complex, requiring the ray casting process to consider the current state of the frame buffer in order to intermix the volume data with other geometric data in the scene. In our GPU implementation of ray casting, the volume data is stored on the GPU in 3D texture memory. Ray casting is initiated by rendering a polygonal representation of the outer surface of the volume. A fragment program is executed at each pixel to traverse the data and determine a resulting value that is combined with the existing pixel value in the frame buffer computed during the opaque geometry phase of rendering.
For more detailed information, please see the July 2008 edition of The Source, which can be found here: http://kitware.com/products/archive/kitware_quarterly0708.pdf.
Open Issues
- When camera is inside the volume, the GPU mapper creates images that are different from the CPU mapper
- Integration with dual view layout has not happened
Integration into VTK
The GPU Ray Caster was initially funded by the following SBIR projects:
"Visualization of Massive Multivariate AMR Data" National Science Foundation Contract Number OII-0548729
"Rapid, Hardware Accelerated, Large Data Visualization" Department of Defense / Army Research Laboratory Contract Number W911NF-06-C-0179
The output of these projects was a new toolkit under GPL terms called VTKEdge. As part of this project, the GPU Ray Caster will be moved out of the VTKEdge toolkit into the BSD licensed Visualization Toolkit (VTK). In the process, the names of the classes will be changed to follow VTK conventions.
This relocation is currently in process and is expected to be complete by January 11th. The new classes will be located in the VolumeRendering kit of VTK.
=Integration into Slicer
The plan is to review the current Volume Renderer module and work with Yanling Liu <vrnova@gmail.com> and Alex Yarmarkovich <alexy@bwh.harvard.edu> to integrate the new mappers to the current Volume Rendering GUI, so that they can be selected at run-time.
Result March 2010
The GPU Ray Caster has been moved into VTK 5.6 and Slicer, by shifting its official revision of VTK to 5.6, has support for the VTK ray caster (vtkGPUVolumeRayCast*).
In the Volume Rendering module, an entry has been added: VTK GPU Ray Casting.
