Documentation/4.5/Developers/MRML

For the latest Slicer documentation, visit the read-the-docs.

Slicer MRML Overview

• MRML Library provides API for managing medical image data types (Volumes, Models, Transforms, Fiducials, Cameras, etc) and their visualization.
• Each data type is represented by a special MRML node.
• MRML Scene is a collection of MRML nodes.
• Slicer MRML data model is implemented independent of the visualization and algorithmic components of the system.
• Other Slicer components (Logic and Qt Widget) observe changes in MRML scene and individual nodes and process change MRML events.

For historical references, these slides described the MRML architecture as it was implemented in Slicer3. Some of the concepts still apply.

MRML Scene

• MRML Scene manages MRML nodes : add, delete, find, find by type, etc.
• MRML Scene provides persistence of MRML nodes (reading/writing to/from XML file).

MRML Nodes

• MRML nodes are designed to store the state of the Slicer application, both raw data and visualization, and storage parameters. A set of core MRML nodes that store the state of core Slicer modules is listed here.

• MRML nodes are organized into C++ class hierarchies, all derived from vtkMRMLNode class. For example, vtkMRMLTransformableNode is the parent class of Volume, Model, Fiducial, and Transformation nodes; vtkMRMLVolumeNode is a parent of vtkMRMLScalarVolumeNode and vtkMRMLVectorVolumeNode.

• All MRML nodes have to implement certain standard API: ReadAttributes, WriteAttributes, Copy, etc.

• The same data can be visualized in various ways and can be stored in different formats. Therefore, information is stored separately in three node types:
  • Data node: store the raw data (such as vtkMRMLScalarVolumeNode stores the voxel of a volume, spacing, position, orientation).
  • Display nodes: describes how the data should be visualized (there can be multiple display nodes for the same raw data, e.g., one for volume rendering and one for displaying as an image slice)
  • Storage node: describes how the data should be stored persistently on disk (file format, file name)

• When a data node is created, it is also necessary to create the corresponding display and storage nodes to allow proper visualization and saving to file. This is where MRMLLogic can be helpful in providing function allowing to create a node and is associated display and data node if needed. Note also that in some cases, Slicer detects if the display and storage node is missing and tries to create a default nodes, but the developers should not rely on this error-recovery mechanism.

MRML node attributes

MRML nodes can store custom attributes as (attribute name; attribute value) pairs.

To avoid name clashes custom modules that adds attributes to nodes should prefix the attribute name with the module's name and the '.' character. Example: the DoseVolumeHistogram module can use attribute names such as DoseVolumeHistogram.StructureSetName, DoseVolumeHistogram.Unit, DoseVolumeHistogram.LineStyle.

MRML Node References

There are two mechanism for specifying that nodes are related: hierarchy nodes and node references. The node reference is the simpler of the two methods. MRML nodes can reference and observe other MRML nodes using the node reference API. The framework will automatically take care of (1) read/write/copy of node references, (2) updating references on scene import, and (3) adding and deleting nodes. A node may reference multiple nodes, each performing a distinct role, and each addressed by a unique string.

Data node can optionally implement interfaces like Displayable, Storable and Transformable. Each one of these interfaces have its own requirements. For example, a transformable node will expected to be associated with a transform node. Such association are done by creating MRML References.

For more details, see here.

MRML Events and Observers

• Changes in MRML scene and individual nodes propagate to other observing nodes, GUI and Logic objects via vtk events and command-observer mechanism.
• Use vtk AddObserver() and InvokeEvent() methods. vtkSetMacro generates ModifiedEvent.
• The command-observer mechanism for MRML is implemented using helper vtkObserverManager, class, MRML Observer macros, and ProcessMRMLEvents method.
• Observers should store a registered pointer to a MRML node to prevent callbacks on a deleted object.

• MRML observer macros are defined in Libs/MRML/vtkMRMLNode.h
• vtkSetMRMLObjectMacro - registers MRML node with another vtk object (another MRML node, Logic or GUI). No observers added.
• vtkSetAndObserveMRMLObjectMacro - registers MRML node and adds an observer for vtkCommand::ModifyEvent.
• vtkSetAndObserveMRMLObjectEventsMacro - registers MRML node and adds an observer for a specified set of events.
• SetAndObserveMRMLScene[Events]() method is used in GUI and Logic to observe Modify, NewScene, NodeAdded, etc. events.
• ProcessMRMLEvents method should be implemented in MRML nodes, Logic, and GUI classes in order to process events from the observed nodes.

Creating Custom MRML Node Classes

• Custom MRML nodes provide persistent storage for the module parameters.
• Custom MRML nodes should be registered with the MRML scene using RegisterNodeClass() so they can be saved and restored from a scene file.
• Classes should implement the following methods:
• CreateNodeInstance() – similar to VTK New() method only not static.
• GetNodeTagName() – return a unique XML tag for this node.
• ReadXMLAttributes() – reads node attributes from XML file as name-value pairs.
• WriteXML() – writes node attributes to output stream (as in interpolate="1" ).
• Copy() – copies node attributes.

• If the node has references to other nodes the following additional methods should be implemented:

–UpdateReferenceID() - updates the stored reference to another node. 
–UpdateScene()- updates other nodes in the scene depending on this node or updates this node if it depends on other nodes when the scene is read in. 
  This method is called automatically by XML parser after all nodes are created. 

• An example of a custom MRML node implementation: vtkMRMLGradientAnisotropicDiffusionFilterNode in Modules/GradientAnisotropicDiffusionFilter directory.
• To add node to the MRML scene:

–In the code: use standard vtk New() and add node to the scene using vtkMRMLScene::AddNode(vtkMRMLNode *)
–By user request: use vtkSlicerNodeSelectorWidget that creates a new node from the module’s UI.

MRML Hierarchy Node

This section is an attempt to describe the vtkMRMLHierarchyNode.

H = hierarchy node, D = display node, M = model node

Other Useful References

MRML API Documentation

The detailed documentation of MRML API can be found in Slicer 4.5 Doxygen pages

Slice view pipeline

VTK/MRML pipeline for the 2D slice views: File:SliceView.pptx.

Another view of the ScalarVolumeDisplayNode pipeline:

Notes: the MapToWindowLevelColors has no lookup table set, so it maps the scalar volume data to 0,255 with no "color" operation. This is controlled by the Window/Level settings of the volume display node. The MapToColors applies the current lookup table to go from 0-255 to full RGBA.

History

See Data Model notes in AHM 2006 Programming week project.

Path-based MRML proposal

Mike's proposal for a path-based MRML3 representation, based on extending the Coordinate Space Manager ideas to the entire MRML3 tree

Slicer Daemon

The goal of the Slicer Daemon project is to allow remote editing of the MRML data model by external programs over a socket.

EventBroker Discussion

The Event Broker is a proposal to improve the way events are managed inside slicer.

Slicer 2.6 MRML

Data Represented in MRML in Slicer 2.6

• Volumes
  • IJK->RAS (VTK->RAS)
  • Scalar Types
  • Multicomponent (RGB, Displacement Vector)
  • Tensor Volumes
  • Label Maps
  • Reference to Lookup Table

• Models
  • vtkPolyData
    • Named Field Data (scalars, vectors, labels) at points and cells (FreeSurferReaders)
    • Polylines with tensor point data (DTMRI Module)
  • Color, Clipping State, Visibility, Scalar Visibility, LookupTable

• Transforms
  • Matrix4x4

• Lookup Tables
  • vtkLookupTable info

• Fiducials
  • Position, Quaternion
  • Name, Selection State, Type (endoscopic, normal)
  • Glyph Size, Text Size

• Fiducial Lists
  • Name, Slze, Color, Selection State

• Colors
  • Name, Label#, Diffuse/Ambient/Specular

• Model Groups

• Application State (not to be carried to Slicer3 MRML)
• Locator (not to be carried to Slicer3 MRML)
• Module Specific Parameters (not to be carried to Slicer3 MRML)

Operations On MRML Scene

• Load from File
• Save to File
• Traverse Nodes in Tree
• Insert Node
• Delete Node
• Register Tree Observer
• Update MRML
• Get Transformations (e.g. IJK to World through transform tree)

• Data Type Specific Operations
  • Get/Set Node MetaData
  • Get/Set Data (e.g. as vtkImageData)

General References on XML

A wikibook on XML

The section on ID/IDREF implementations, which are similar to what we use in MRML.

Developer FAQ: MRML

How to add a MRML node into the scene ?

• Generic pattern

vtkNew<vtkMRML???Node> nodeToAdd;
...
mrmlScene->AddNode(nodeToAdd.GetPointer());

• Add a polydata to the scene

vtkNew<vtkMRMLModelNode> modelNode;
modelNode->SetPolyData(polyData);
mrmlScene->AddNode(modelNode.GetPointer());

• Load a polyData from file

vtkSlicerModelsLogic* modelsLogic = ...;
//modelsLogic->SetMRMLScene(mrmlScene);
modelsLogic->AddModel(polyDataFileName);

What to do if you get 'No LookupTable was set but number of components in input doesn't match OutputFormat' when loading a MRML scene ?

If you get the following error messages:

ERROR: In /path/to/VTK/Imaging/vtkImageMapToColors.cxx, line 153
vtkImageMapToColors (0x268f190): RequestInformation: No LookupTable was set but number of components in input doesn't match OutputFormat, therefore input can't be passed through!

ERROR: In /path/to/VTK/Imaging/vtkImageExtractComponents.cxx, line 239
vtkImageExtractComponents (0x26947e0): Execute: Component 1 is not in input.

ERROR: In /path/to/VTK/Imaging/vtkImageExtractComponents.cxx, line 239
vtkImageExtractComponents (0x26947e0): Execute: Component 1 is not in input.

[...]

Make sure the colorNodeRef attribute is set on each VolumeDisplay node.

How to change the volumes in the 2D views ?

appLogic = slicer.app.applicationLogic()
selectionNode = appLogic.GetSelectionNode()
selectionNode.SetReferenceActiveVolumeID(bg)
selectionNode.SetReferenceSecondaryVolumeID(fg)
appLogic.PropagateVolumeSelection()

Source: https://github.com/fedorov/ChangeTrackerPy/blob/master/Wizard/Helper.py#L82