Slicer3:Multiple Cameras

Goals

Bring core support for multiple cameras and 3D views in Slicer3.

Features

The multi views and cameras framework is available from the "Camera" modules. This module displays two pull-down menus. The first one, "View", lists all available views and lets the user create new 3D views. The second one, "Camera", lists all available cameras, and lets the user create new cameras.

Only one camera is used by a view at a time. When a view is selected from the pull-down, the camera it is currently using is automatically selected in the second pull-down.

• Create a new camera. In the "Camera" pull-down menu, select "Create New Camera": a new camera node is created (most likely named "Camera1", as opposed to the default "Camera" node), and automatically assigned to the currently selected view node (named "View" by default). Try interacting with the 3D view.

• Assign a camera to a view. Select any camera node from the "Camera" pull-down menu: the camera is assigned to the currently selected view. For example, try selecting back the "Camera" node if you have created a "Camera1" node in the previous step, and you should notice the 3D view on the right update itself to reflect the different point of view. Note: a camera can not be shared between two views: selecting a camera already used by a view will effectively swap the cameras between the two views.

• Create a new view: select the "Tabbed 3D Layout" from the layout button in the toolbar. In the "View" pull-down menu, select "Create New View": a new view node is created (most likely named "View1", as opposed to the default "View" node), and displayed on the right under a new tab. You can select which view to display by clicking on the corresponding tab in the "Tabbed 3D Layout". Interacting in that view will automatically mark it as "active"; there can only be one "active" view at a time in Slicer3, and it always feature a thin dark blue border. Since a view can not exist without a camera, a new camera node is automatically created and assigned to this new view (most likely named "Camera2" if you have created "Camera1" in the previous steps). At this point, you can assign any of the previously created cameras to this new view (say, the "Camera" or "Camera1" nodes).

Implementation

Here are a few pointers related to the new framework. What happens under the hood, and what events developers and module maintainers can intercept to use the new features.

The workflow so far (Slicer 3.6, as of November 2009):

When a vtkMRMLViewNode is created and added to the MRML scene (say by a module, or by the main app, or a third party):

• vtkSlicerApplicationGUI, which listens to vtkMRMLScene::NodeAddedEvent, sees that a new view node has been

created, and calls vtkSlicerApplicationGUI::OnViewNodeAdded().

• OnViewNodeAdded() adds a few observers to the view node, and if there is no other vtkMRMLViewNode around (i.e. if it's the first one), it is automatically tagged as "Active". vtkSlicerApplicationGUI::UpdateMain3DViewers() is then called.

• UpdateMain3DViewers(), a fair amount is done here, but the gist is: the function loops over all the vtkMRMLViewNode in the scene, and if it finds out that one doesn't have an associated vtkSlicerViewerWidget yet (i.e. the graphical part that really makes up the view), it allocates and Create()'s one and set its ViewNode ivar to the vtkMRMLViewNode pointer. This is the only place in Slicer3 where vtkSlicerViewerWidget::New() is created (this wasn't the case before, I factorized it). Note that some MRML nodes like vtkMRMLCamera maintain and allocate their VTK counterpart automatically (vtkCamera). This is not the case for vtkMRMLViewNode (I don't know why, but I didn't want to break that for backward compatibility, therefore that design didn't change).

In a similar fashion, vtkSlicerApplicationGUI listens to vtkMRMLScene::NodeRemovedEvent, calls vtkSlicerApplicationGUI::OnViewNodeRemoved, which remove previous observers, and calls UpdateMain3DViewers() as well. This function still does what is described above, but also loops over all the vtkSlicerViewerWidget it knows: if the corresponding vtkMRMLViewNode doesn't exist anymore (since it was just removed), it deletes that viewer widget, which should free the corresponding graphic resources.

Module maintainers should not have to worry about the creation of Slicer Viewer Widgets, as this is taken care of as described above. What a module should do:

• listen to vtkMRMLScene::NodeAddedEvent, sent by the MRML scene: if a vtkMRMLViewNode has been added, that means you may have to prepare yourself for allocating the resources you will put in that view (vtkSlicerViewerWidget ),

• listen to vtkMRMLScene::NodeRemovedEvent, sent by the MRML scene: if a vtkMRMLViewNode is being removed, this is definitely the time for your module to remove your resources from that view, and probably free them.

• listen to vtkMRMLViewNode::ActiveModifiedEvent, sent by a vtkMRMLViewNode node (i.e. when a view node is added, it's up to you to add that observer on the view node, when the node is removed, you remove that observer): if a view becomes "active", it means that it is "selected". Right now, in dual view mode, the active view is the last one that was interacted with: it has a darker outline. This active event is useful essentially because of the left panel UI. If you have some UI that reflects the parameters used in VolumeRendering for example (say a LUT editor), it is possible you may want to apply those parameters only to the resources pertaining to a specific view. In any case, when you receive an Active event, your UI is given the opportunity to update the parameters that are only relevant to that specific view.

What is said in that first bullet point is not exactly helping though: vtkMRMLScene::NodeAddedEvent is triggered when a vtkMRMLViewNode is added, but it is still too early for your module to add resources to that view (i.e. the renderers), because the corresponding vtkSlicerViewerWidget has not been created; this is done in UpdateMain3DViewers(). So even though you may want to listen to NodeAddedEvent to prepare some resources, you still can't add them, no renderers are ready for your actors. What you need is a way to know that the corresponding SlicerViewerWidget has been created. To fix that situation, your module should ultimately:

• listen to vtkMRMLViewNode::GraphicalResourcesCreatedEvent, sent by a vtkMRMLViewNode node (same as #3, it's up to you to add that observer on the view node, by first listening to vtkMRMLScene::NodeAddedEvent). This event carries a pointer to the corresponding vtkSlicerViewerWidget that was just created in UpdateMain3DViewers(). Now seems a good time for you to add your actors to the renderers.

Note that this doesn't necessarily conflict with the idea that a viewer widget, as Jim Miller suggested, could keep track of what is visible or not in its own render window... I guess for the time being we could let the modules automatically add their actors to each new view when they are added to the scene, as described above, and then later on we can come up with something to turn the visibility on/off of actors in each viewer...

To sum up:

• listen to vtkMRMLScene::NodeAddedEvent and vtkMRMLScene::NodeRemovedEvent on the scene,
• received vtkMRMLScene::NodeAddedEvent and its a new vtkMRMLViewNode? Allocate your graphical resources (don't add them yet), add vtkMRMLViewNode::GraphicalResourcesCreatedEvent and vtkMRMLViewNode::ActiveModifiedEvent to this new view node to know when graphical resources have been created for that view (the renderwindow, renderers, etc), and when the view becomes "the active one",
• received vtkMRMLViewNode::GraphicalResourcesCreatedEvent? time to add your actors to the corresponding view,
• received vtkMRMLViewNode::ActiveModifiedEvent? time to refresh your UI in the left panel
• received vtkMRMLScene::NodeRemovedEvent and it's a vtkMRMLViewNode? Time to remove your resources from that view.

Now of course the choice of allocating resources *per view* depends on the module. One can technically add the same actor to different renderers, but this doesn't always do the trick (especially not for volume rendering). For example, fiducials use vtkFollower actors to make sure labels always face the camera. Since the camera is different for each renderer, this resource will have to be allocated for *each view*. It's likely up to the module to maintain some sort of STL map associating a view to the resources for that view.

Acknowledgment

This work is part of the National Alliance for Medical Image Computing (NAMIC), funded by the National Institutes of Health through the NIH Roadmap for Medical Research, Grant U54 EB005149.
The Cameras module was contributed by Sebastien Barre, Kitware Inc.

TODO

As of Slicer 3.6:

• there is one 'full featured' 3D viewer, which is the one at the top in the default layout. This is the one where volume rendering, fiducials, ROIs (box widget), and other actions take place,
• there can be multiple 'secondary' 3D views that have independent cameras but only see the models (essentially just the vtkActors but not widgets or volumes).

All modules that display an actor on screen are technically concerned and need to be revisited to use the new framework. Now that the multi camera/views logic is in the core, it's a matter of letting the module maintainers know that they can upgrade their code to use this particular set of functionality (which didn't exist when modules like Fiducials, Volume Rendering, etc were first developed).

Modules that need to be upgraded (please keep updating this list):

• Fiducials
• Volume Rendering