Slicer3:Volume Rendering With Cuda

Overview

On this page a description of the Volume Rendering algorithm with CUDA For Slicer3 is presented. A link to the IGT project page can be found on Slicer-IGT/GPU-IGT.

Involved in this projects are Benjamin Grauer (integration into vtk/Slicer3), Nicholas Harlambang (cuda algorithms) and Nobuhiko Hata (project supervisor).

Objective

The goal of this project is to implement a CUDA based Volume Rendering Technique inside of VTK and Slicer3.

Project Outline

Several steps have to be made to integrating a CUDA based volume rendering technique into vtk.

1. Implement and Integrate a CUDA Volume Rendering algorithm.
2. Integrate CUDA runtime API support into VTK using a class based approach.
3. Integrate the algorithm into a VTK pipeline (described below)
4. Create a user interface to control the flow of the Volume Rendering within Slicer3.

Cuda Volume Rendering Algorithm

The Code that we use so far has been implemented by Nicholas Harlambang and can be viewed here.

Feature Wishlist

Relevance	Feature	Status	Dependency	Description
+++	Z-Buffer	60%	Current Z-buffer to CUDA	Render and stop at the Z-buffer information
++	Different Data Types	100%		uchar, char, float, double, short, ushort, int
+++	Sample Distance	60%		The sampling step distance along the ray.
++	Transfer Function	100%
+	Lighting Model	30%
=	Multiple Lights	0%
-	Shading Model	0%	Lighting Model	Use Shaders to render
+++	Perspective and Planar Projection	100%		Perspective Projection using vtkCamera Projection Matrix
=	Memory To Texture	100%		Render CUDA directly onto Texture (for GeForce Cards only)
-	Partial Memory Update	0%		Partial Updates of the Input data for animation
--	vtkCudaVolumeProperty	100%	Transfer Function,	Implement all the features of vtkVolumeProperty
-	Animation	60%	Partial Memory Update	4D Animated Image Data
+	No more static Symbols	80%		Remove the need of all c-style static Symbols in the CUDA code
---	Multicore/GPU implementation	0%	Everything else	Multicore implementation of the Cuda Rendering
--	Gradients	?%		Not sure yet!

VTK Cuda Runtime API support

The c-interface to the CUDA objects and functions will be wrapped in a safe object oriented VTK-C++ class hierarchy.

• VTK Cuda Wrapper Classes
• VTK Cuda Memory Management Classes

Cuda Memory

Cuda memory comes in different shapes and forms:

• Host Memory: Non-Paged Memory on the host side for fast interaction with the CUDA devices. see vtkCudaHostMemory.h
• Linear Memory: Linerely arranged memory on the CUDA device. see vtkCudaMemory.h
• Pitch Memory: 2D memory arrays that are aligned in the correct fashion for a fast interaction on the cuda devices. (Arranged over the memory banks for quick access). see vtkCudaMemoryPitch.h
• Array Memory: Arrays allocated over the CUDA API. see vtkCudaMemoryArray.h
• Texture Memory: Not discussed here. see VTK Cuda Wrapper Classes

All these memories are derived from a base class (see vtkCudaMemoryBase.h) as shown in the following picture:

Cuda Memory Class Diagram

VTK Modules and Pipeline

These are the modules that will be created in order to integrate a Cuda based Volume Rendering into the Slicer3 pipeline

1. A vtkImageReader: to read Volume data as any kind of Volume Data
2. A vtkCudaImageFilter: A Filter to convert from the reader output to a cuda-able DataSet
3. A VtkCudaDataSet: A new vtkDataSet where that holds and handles the data transfer from and to one or multiple CUDA devices.
4. A vtkCudaVolumeMapper: A new vtkVolumeMapper that renders the scene using the
   1. lighting model
   2. a prepared Z buffer
   3. a camera position
   4. A vtkTexture and a Plane to render the result to
5. An actor that places the volume into the scene
6. Chain this pipeline together and attach it to a rendering window.

In the following image the pipeline is displayed together with the connections from the VTK, Slicer3 and CUDA frameworks.

CUDA Volume Rendering Pipeline

Description:

• The Fat line displays the pipeline flow.
• Green colored fields are classes from the Slicer3 or vtk implementation
• Red colored fields are CUDA and hardware specific parts
• Blue colored fields are specially implemented parts for the communication of cuda and vtk and the volume rendering algorithms.

A Slicer3 User Interface

The GUI will be designed using KWWidgets. With this the user will be able to change the rendering resolution, step size, control the color sceme and load and store further information.

Tasks and Timeline

Tasks and Completion

Task	Status
Overall Status			75%
Integrate into VTK		85%
VTK Memory Class Definitions	90%
VTK Access Classes	80%
VTK Algorithm	90%
CUDA Volume Rendering Algorithm		80%
Adapt to Z-buffer	70%
Adapt lighting model	0%
Adapt to input Image	100%
Integration into Slicer		80%
Integrate as a Module	100%
GUI	90%

Timeline

From	To	Task
12/14/07	05/31/08	Master Thesis Duration
12/14/07	12/31/07	Reading into the Project, Defining Basic Classes
01/01/08	01/06/08	Basic Class Definition Ready for Hands on Meeting
01/07/08	01/12/08	Hands on meeting in Salt Lake City
01/14/08	02/01/08	Further Implementation
05/01/08	29/05/08	Write Master Thesis
06/14/08		Hand in Masters Thesis in Zurich

Progress

See the daily progress here.

• (01/03/08): Successfully proved that CUDA runs inside of Slicer3 with the code from Nicholas Harlambang
• (02/01/08): Cuda inside of Slicer3 and external Application, 4D imaging
• (02/05/08): Integration Rendering with Cuda Code and Transfer Functions
• (02/11/08): Integration with Network Module OpenIGTLink See here
• (03/06/08): Model is aligned correctly in Slicer RAS space.

Resources

1. For testing purposes: File:Heart256.raw Store this file in Slicer3/Modules/VolumeRenderingCuda/Testing/heart256.raw
2. Also checkout the main project page at Slicer-IGT/GPU-IGT
3. Currently I am working in this branch
   1. on CudaSupport: Libs/CudaSupport
   2. and CudaVolumeRendering: Modules/VolumeRenderingCuda
4. Slicer Version using CUDA: media:SlicerWithCuda.exe

People Involved

• Nobuhiko Hata: Supervisor
• Nicholas Harlambang: Implementer of the Cuda Code and Algorithms
• Benjamin Grauer: Implementer of the VTK/Cuda integration and CudaSupport Library
• Andreas Freudling: First Implementation of Volume Rendering in Slicer3 and great supporter

Screen Captures

• 

  The First Volume Render using CUDA within the Slicer Window

• 

  The First 4D Volume Rendering in CUDA using VTK

• 

  The First Volume Render using CUDA within the Slicer Window

• 

  Volume Rendered Head with a Interpolation Smoother