Documentation/Labs/SubjectHierarchy

Motivation

• Slicer has a great number of modules that the user has to be able to juggle. This leads to confusion for many users.
• Data module allows exploring the loaded data, but
  • It presents the nodes in a flat list that is hard to manage
  • Offers multiple modes (transforms, displayable), but it makes it more confusing
  • The user has to find the modules needed to operate on the data, so this module cannot act as a "home" module
• Subject hierarchy aspires to be a convenient central organizing point for data loaded into Slicer and many of the operations that Slicer and its extensions perform

Subject hierarchy features

• Convenient central organizing point where users start their workflows: "home" module
  • Nice and intuitive way of organizing and handling data in a tree view that allows easy identification of the data through icons
• Bring basic features in a data-centered tree view, such as
  • Show/hide of any data
  • Transform object or branch
  • Edit properties. Same idea as in Data module, takes the user to the module handling that specific type. The difference is that in subject hierarchy it is possible to do further steps, such as populating the module with inputs, or set suitable parameters automatically.
  • Expand tree to certain level
  • Dynamic help box populated by the plugins
  • Rename, remove, reparent
• Provides a multitude of features through the plugin mechanism

Subject hierarchy 2.0

Although the subject hierarchy mechanism is in place and is already used, there are several issues and limitations that cannot be addressed in its current form:

• Subject hierarchy needs to be disabled by default, because
  • A second node is created for each data node and the scene can grow very large in case of many data nodes
  • Model hierarchies and subject hierarchies are not fully compatible, so problems may arise in case of complex model hierarchies (such as atlases)
  • Because of this, a popup is shown to the user, which is confusing for users. It prevents some from using SH, and is annoying for the others
• Expanded state is reset to all extended when switching to another module and back. This can be very frustrating when there are many subjects or folders, because the user sees it very differently after returning to SH
• Manual re-ordering of nodes is not possible
• Node selector based on subject hierarchy (e.g. where it's possible to start selecting by patient in some way) cannot be implemented without major changes

These issues are due to

• The overly complex Qt MRML scene model infrastructure, which is rigid in some ways, and bloated in others
  • Can only show nodes, so if an item is needed to be shown, a new node needs to be created (thus the lot of SH nodes and the virtual branches, see below). Also pointer tracking of nodes complicates things.
  • Index calculation is very complex and gets even more complex in the sub classes.
  • The features that are needed for specific models but are implemented in the base model overcomplicate the implementation (e.g. pre- and post items)
• Unstable workarounds implemented in the current subject hierarchy infrastructure to accommodate certain use cases
  • Nested associations that use the MRML hierarchy associations in a nested way so that the same nodes can be added both to model and subject hierarchies
  • Virtual branches to accommodate data that do not correspond to MRML nodes one-on-one (segments contained by segmentations, fiducials contained by markups)
  • Postfixes of the SH nodes to differentiate them from the associated data nodes

This necessitates a re-design of the subject hierarchy mechanism. The proposed solution is to store the whole subject hierarchy tree in one node, the internals of which are synchronized with the data nodes of the scene. A new Qt item model needs to be implemented for this node that supplies the information to the tree view and the future node selectors.

• The proposed pseudo-singleton vtkMRMLSubjectHierarchy node
  • Contains items that have unique identifier numbers that is strictly incremental
  • Item information can be accessed using accessors, the item class itself is private
• Automatically populate from added nodes (as before), observe modified and removed events
• Implement new Qt item model that is synchronized with the subject hierarchy node instead of the scene
• Import model hierarchies to SH node on entering SH module if new detected (thus model hierarchies can be fully used from
• Subject hierarchy logic makes sure there is only one SH node and it is valid (at import, deletion, potential undo, etc.)

New features

• Considerably increased speed due to the single subject hierarchy node (instead of duplicate SH node for each data node)
• Enabled by default, because it does not clutter the MRML scene and there are no special mechanisms to support other hierarchies
  • No pop-up window when entering the module, it can be used immediately
• Full support of model and annotation hierarchies in subject hierarchy
  • It shows the same tree that the Models module shows, and it is handled the same way (atlases, drag&drop, show/hide, etc.)
• Improved drag&drop
  • Re-ordering of items is supported, and is restored correctly when reconstructing the hierarchy (loading a scene etc.)
  • Drag&drop to empty area reparents to the scene
• Expand and collapsed states of the items are preserved, and restored when reconstructing the hierarchy
  • (this and re-ordering was supposed to be implemented in Models, but it seems to have never worked, and there is a lot of dead code in the repository)
• Subject hierarchy combobox showing a drop-down tree
  • It can be added in the modules instead of a simple MRML node selector. This, besides allowing an easier overview of the data, makes it possible to differentiate between nodes that have the same name but belong to different patient or study
• More versatile filtering options in the tree and combobox (name, attribute, level)
• More control over the subject hierarchy Qt model in general, more room for further improvements

Programmatic access and manipulation

Get the pseudo-singleton subject hierarchy node

It manages the whole hierarchy and provides functions to access and manipulate

 shn = slicer.vtkMRMLSubjectHierarchyNode.GetSubjectHierarchyNode(slicer.mrmlScene)

Create subject hierarchy item

 # If it is for a data node, it is automatically created, but the create function can be used to set parent:
 hn.CreateItem(parentItemID, dataNode)
 # If it is a hierarchy item without a data node, then the create function must be used:
 shn.CreateSubjectItem(parentItemID, name)
 shn.CreateFolderItem(parentItemID, name)
 shn.CreateHierarchyItem(parentItemID, name, level) # Advanced method to set level attribute manually (usually subject, study, or folder, but it can be a virtual branch for example)

Get subject hierarchy item

Items in subject hierarchy are uniquely identified by integer IDs

 # Get scene item ID first because it is the root item:
 sceneItemID = shn.GetSceneItemID()
 # Get direct child by name
 subjectItemID = shn.GetItemChildWithName(sceneItemID, 'Subject_1')
 # Get item for data node
 itemID = shn.GetItemByDataNode(dataNode)
 # Get item by UID (such as DICOM)
 itemID = shn.GetItemByUID(slicer.vtkMRMLSubjectHierarchyConstants.GetDICOMUIDName(), seriesInstanceUid)
 itemID = shn.GetItemByUIDList(slicer.vtkMRMLSubjectHierarchyConstants.GetDICOMInstanceUIDName(), instanceUID)
 # Invalid item ID for checking validity of a given ID (most functions return the invalid ID when item is not found)
 invalidItemID = slicer.vtkMRMLSubjectHierarchyNode.GetInvalidItemID()

Manipulate subject hierarchy item

Instead of node operations on the individual subject hierarchy nodes, item operations are performed on the one subject hierarchy node.

 # Set item name
 shn.SetItemName(itemID, 'NewName')
 # Set item parent (reparent)
 shn.SetItemParent(itemID, newParentItemID)

Branch on GitHub

https://github.com/cpinter/Slicer/tree/stop-subject-hierarchy-node-duplication

Subject hierarchy 1.0

Subject hierarchy plugin mechanism

The plugins extend the abstract plugin class that provides broad API offering many customizations. These plugins are the building blocks that make subject hierarchy a really useful central organizing point.

Plugin types

• Role plugins: each node in the tree is "owned" by a certain plugin
  • A plugin is automatically assigned to added nodes by collecting confidence values for the new node from all registered plugins and assigning the one returning the highest confidence value
  • When a subject hierarchy node is modified, a plugin search is automatically performed unless disabled
  • Role-related features
    • Role icon, visibility icon
    • Displayed name, tooltip
    • Show/hide
    • Edit properties
    • Add node to subject hierarchy / reparent inside subject hierarchy
  • Plugins can define levels (e.g. Series, Subseries by DICOM plugin), and parent-children level pairs for child creation actions
• Function plugins: not offering roles but functions, cannot own nodes
  • Returned confidence value is always 0
  • Provides context menu actions (for nodes or the scene) based on rules defined in the plugin

List of implemented plugins

• Core plugins
  • Default: fallback role plugin if no registered plugin can own a node; Provides create subject and study actions
  • Clone node: Make a copy of a node
  • Parse local data: Adds a scene context menu action that re-creates the directory structure of data loaded from local storage
  • DICOM: provides study, series, and sub-series roles, levels, and parent-children pairs
  • Register: Allows user to select two volumes to register, then offers registration methods. When the user chooses one, that module will become active and the inputs and basic parameters are set
  • Volumes: adds Volume role, identifies labelmaps with a separate icon, provides show volume icon and action, adds "show volumes in study" context menu action, toggle labelmap outline/fill
  • Models: role
  • Markups: role
  • Charts: provides Chart role. Layout is changed to quantitative when shown
• RT plugins (to illustrate further possibilities)
  • Contours + ContourSets (will be integrated into core soon): Provides roles for contours and contour sets. Allows creation of contour nodes from labelmaps or models. Handles associated color table when adding/reparenting. Provides show/hide
  • RT image: special type of volume that has a different icon, and is displayed as a textured plane in the 3D view
  • DVH: role for double array nodes having a specific attribute. Allows show/hide plot in chart
  • Gamma: Output of gamma dose comparison, automatically created and added to hierarchy by the DoseComparison module
  • ...

Branch on github

https://github.com/cpinter/Slicer/tree/subject-hierarchy-integration

For code examples, see C++ and python tests

Slicer core changes

• SubjectHierarchy module added in Modules/Loadables
• Subject hierarchy plugins added in Markups, Models, and Volumes
  • Plugins for other modules have to be put in the <module>/SubjectHierarchyPlugins folder
• qMRMLNodeAttributeTableView and qMRMLNodeAttributeTableWidget moved from Data module widgets to MRMLWidgets, because they are now used both in Data and Subject hierarchy modules
• addSeriesInSubjectHierarchy function added to abstract DICOMPlugin class.
  DICOM plugins should call this in their load() function with the loadable and the loaded node to add it to subject hierarchy as a series. It creates subject (patient) and study nodes based on the DICOM tags
  • Subject hierarchy support added in DICOMScalarVolumePlugin.py: temporary conditional call of a function injected from SlicerRT replaced with the actual function
  • Also added to DICOMDiffusionVolumePlugin
• vtkMRMLHierarchyNode::GetAssociatedChildrendNodes renamed to GetAssociatedChildrenNodes and made virtual.
  This change could break extensions if this is used in them, although there was only one usage in the whole Slicer source tree
• vtkMRMLHierarchyNode::GetAssociatedNode made virtual (overridden in vtkMRMLSubjectHierarchyNode to support nested associations)
• Export MRMLCLI_INCLUDE_DIRS CMake variable in SlicerConfig.cmake for subject hierarchy plugins in extensions
• Added two subject hierarchy python self tests in Applications/SlicerApp/Testing

Issues

• Not all data is automatically added to subject hierarchy
  • DICOM data is covered
  • Data loaded from local storage can be added to the hierarchy, but the user has to know about the feature
  • The user can drag&drop supported nodes in the hierarchy, but currently only one can be moved at a time
  • Outputs of analysis and processing modules have to be added manually, unless it supports subject hierarchy
• Refresh problems related to scene model
  • The tree collapses when loading data when subject hierarchy module is active, and sometimes after drag&drop. As a workaround, the user can use the expand to level option of the scene item.
  • The tree can completely disappear. The user has to switch to another module and back.

Future features

• [DONE] DICOM export. A simple control starting the export process, which opens a window with the tree and a DICOM tag editor on the bottom
• Set/change level manually
• AddNode and Reparent should make sure the invalid drag&drop operations are not allowed
• [DONE in 2.0] Allow re-ordering nodes in the tree
• [DONE] Add a "reference inspector" widget where MRML node references can be explored and edited (it can be part of the current MRML Node Inspector widget or a separate one)
• Improve visibility controls - multiple 3D and 2D views, volume rendering, etc.
• [INVALID] Select and drag&drop multiple nodes from the potential list
• [DONE in 2.0] Create subject hierarchy based node selector that can replace qMRMLNodeComboBox in places where the user wants to have a nicer (and familiar) view of the loaded data that is available for that role. Provides a small window with a simple tree view showing only the nodes of the requested type (and attributes)