Slicer4:VectorImageVisualization

Back to Slicer4 Projects

Introduction

The main goal of this page is to set the grounds and goals for a common visualization framework for vector images.

Description of vector images

We consider vector-valued images for the purpose of this project any image composed by several scalar images in which the dimensions and spatial transforms are the same.

Examples of these are:

• DCM-MRI / functional MRI, nowadays handled by Slicer using Junichi's 4D Image Module and 4D Analysis module.
• 4D CT/MRI/US
• Deformation fields which are the result of a registration process.
• DWI-MRI, currently handled by Slicer.
• DTI-Images could be set under this category but their visualization use-cases are, maybe, far too different from the others.
• More complex representations of diffusion MRI (DSI, Q-Ball, Multishell)
• RTSTRUCT, radiotherapy planning structures

Image type	4th axis semantics	Number of samples on the 4th axis
DCM-MRI / fMRI / 4D CT, MRI US	Time	not fixed
Multi-Channel Data	Contrast	not fixed
Deformation field	X,Y,Z components of the transform	3
DWI-MRI	a function with domain angle x b-value	not fixed
DTI-MRI	Components of a symmetric positive definite matrix	9 (or 6 as there are only 6 independent components)
More complex representations of diffusion MRI	spherical harmonic coefficients, angular functions, etc	not fixed
RTSTRUCT	structure labeling (one bit per structure)	not fixed

Basic functionalities

• Multi-component scalar-volume visualization: as with the DWI-images where there is a slider enabling the user to choose which component to visualize
• Split and merge into individual volumes. Concept is partially implemented in split and merge in the interactive editor
  • 4D Image editing function is also partially implemented in 4D Image Module.
• Visualization of derived values:
  • dMRI: Glyph-based visualization of tensors, scalar visualization of color by orientation, fractional anisotropy
    • Visualization as vectors (deformation fields, principal diffusion direction)
    • Visualization as Ellipsoids or multiquadrics (representation of DT-MRI)
  • DCE: visualization of the parameter values extracted from model fitting such as tofts model.
• 2D-plot exploring of a single voxel along the 4th axis as in FourDAnalysis module
• Colormap visualization: Mapping from a 3D vector to the RGB components. Examples are
  • DT-MRI visualization
  • Direction of the deformation field at each voxel
  • Combinations of multichannel volumes (microscopy)
• For RTSTRUCT
  • Dynamically increase (or decrease) vector size
  • Select a bit within the vector
  • Display outline, shading, other visualization

Use Case Scenario

• R: RECIST analysis
• M: Time Series Analysis of MRI's from an MS Patient
• P: Time Series Analysis of SUV values in MR/CT PET
• D: Dynamic Contrast Enhancement (DCE)

Common Features and Needs

• These are all time series of scalar volumes and need to be organized in a number of dimensions: space, time, signal intensities, feature space (such as channels in multichannel data, gradient strength in dMRI data, etc.)
• Typically the volumes need to be registered to each other in order to organize them in space: Fusion between the channels, fusion across the time points
  • What is the best way to pass 4D image data to command line modules?
• bias correction is often needed: MR, PET, CT(?)
  • No bias correction for CT
• Window/Level:
  • Allow manual setting of w/l for the reference time point.
  • CT: one w/l for all time points, PET, MR: the SUV concept could be expanded to MRI: use one or two user selected/drawn ROI to adjust window level across time points.

Viewing and Analysis

• view across time: movie (loop, bounce)
• graphing: resample across time
• need to do analysis across time: ROI's (label map) for voxel statistics across time. Create spreadsheet / graph as result.
• need to store the organizational frame work. When the patient comes for the next scan we need ability to add a time point, fix data (registration, w/l, update spreadsheet/graph)

Workflows

Initial Setup of a MultiVolumeContainer

• Assume that all data have been loaded into Slicer already
• Create a new MultiVolumeContainer.
  • MVC's are always multi-channel. Single channel is just a special case.
  • User specifies the number of channels.
  • one of the volumes is assigned as the reference volume (default is the first loaded, user can change manually by clicking a checkbox)

Things you can do with MVC's

• editing the MVC by adding or removing time points or changing their order
• change the reference volume
• assign registration types between the channels and between the time points
• execute the registration cascade
• adjust W/L across time for each channel using histogram equalization with manual override
• across time analysis: label map with multiple ROI's will give across time statistics
  • special case for SUV across time
• RECIST: determination at each time point, use previous one as initialization
• Formulas: DCE requires Tofts, NukMed uses other, similar formulas, DWI to DTI uses Stejskal.

Implementations

MultiVolumeContainer MRML Node

• vtkMRMLDiffusionImageVolumeNode is a good start?
• Class hierarchy (names are tentative)
  • vtkMRMLScalarVolumeNode
    • vtkMRMLTensorVolumeNode (?)
      • vtkMRMLMultiVolumeNode
        • vtkMRMLTimeSeriesVolumeNode
        • vtkMRMLMultiChannelVolumeNode
        • vtkMRMLVectorFiledVolumeNode
        • vtkMRMLDiffusionWeightedImageVolumeNode
        • vtkMRMLDiffusionTensorImageVolumeNode
        • <More complex representations of diffusion MRI>
        • vtkMRMLStructureLabelingVolumeNode

4D Image Editor Module

• Create / edit multi volume container

Time-series Image Analysis Modules

• Time-intensity plot + curve analysis

4D Image Control Widget

• GUI widget that can be embedded in any GUI modules
• Node selector + slider to switch volume + 4th value display (time, b-value, etc.)