Slicer3:UIDesign:WorkingProblems:MouseModes:Draft1

Back to Slicer3 UI Design

Back to MouseModes in Slicer3

User-suggestions for interaction patterns

• various lassos would be nice to select multiple of the thing current mode indicates
• single click to select one of the thing the current mode indicates
• control-click to select multiple of the thing the current mode indicates
• a shift-click to select ALL of the thing the current mode indicates

Proposed mouse mode design

The following four mouse modes are proposed:

• Pick (or select),
• Manipulate (or transform selected),
• Place ( or create new object), and
• Transform View.

Transform View is the default mode. All other modes can be invoked in a non-persistent manner from this mode with hot-keys. (which keys to choose?) The proposed design attempts to accommodate user requests listed above, and to provide keyboard+mouse commands for transform view and transform object that are consistent across modes.

Current bindings:

• 3D viewer: left mouse button >> rotate
• 3D viewer: middle mouse button >> pan
• 3D viewer: mouse wheel button >> zoom in/out
• 3D viewer: right mouse button, pull drag >> zoom in
• 3D viewer: right mouse button, push drag >> zoom out

• Slice viewer: left mouse button >> select/move a fiducial point
• Slice viewer: middle mouse button >> pan
• Slice viewer: mouse wheel button >> scroll slices
• Slice viewer: right mouse button, pull drag >> zoom in
• Slice viewer: right mouse button, push drag >> zoom out
• special key-bindings enable placing of fiducials

Proposed bindings:

Pick mode:

• left-mouse-button click (mouse up) for single selection
• ctrl+left-mouse-button click (mouse up) for multiple selection
• shift+left-mouse-button click (mouse up) for selecting all of a datatype, where appropriate

Manipulate mode:

• left-mouse-button press+drag to rotate
• middle-mouse-button press+drag to translate
• right-mouse-button press+drag to scale
• shift+left-mouse-button press+drag to translate
• ctrl+shift+left-mouse-button press+drag to scale

Place mode:

• left-mouse-button-down creates and initially places an object
• drag translates the object in X,Y
• left-mouse-button-up finalizes place operation.

Transform view mode:

• left-mouse-button press+drag to rotate
• middle-mouse-button press+drag to pan
• right-mouse-button press+drag to zoom
• shift+left-mouse-button press+drag to pan
• ctrl+shift+left-mouse-button press+drag to zoom

Selection behavior

Once the concepts of 'active' and 'selected' are folded together, then all transformable data types will be either selected or unselected. When selected they should display a consistent visual treatment that makes this state clear. This presents a number of challenges that we need to address.

Challenge 1: Consistent mouse-over behavior for widgets and selectable data objects. Ideally, when the application is in "Pick" mode, there will be consistent mouse-over and selection behavior between VTK widgets and VTK objects representing datasets in the scene. How does an application currently know whether the mouse is over a VTK widget -- is a pick constantly running against a list of instanced widgets? In order to have datasets or sub-sets highlight on mouseover (highlight-rendering aside), the mouse would continuously have to pick, which is too computationally expensive to be practical.

Approach: One approach is to attach VTK widgets to each object in the scene, as is implemented in the fiducials module. This could be visually noisy, and it may be hard to tell which object a widget is attached to. Maybe better: we could adapt the QueryAtlas picker to display a model name, label name, cell ID (whatever is appropriate for the datatype being picked) on mouse-over in pick mode.

Challenge2: Clear and clean select feedback for data objects in the scene, compatible with rendering of active vtk widgets.

Approach:

Challenge3: Ability to easily clear selections. Currently, 'active' models and volumes etc. are persistent, but since their 'active' state isn't visually indicated in the scene, this state remains kind of hidden -- and is only revealed in the vtkSlicerNodeSelectorWidgets in the GUI panel. But once 'active' and 'selected' represent the same concept and the visual presentation of any/all selected datatypes is displayed in Slicer viewers, this change of interactive behavior will be very noticeable to users.

Approach: Once the scene display becomes affected by any object's selected state, users are likely going to want to have a QUICK & EASY option to deselect all after they perform some processing on a selected dataset. Candidate approaches would be to inclue a "deselect all" option in the drop-down pick menu, or an explicit "deselect all" icon in the toolbar.

Manipulation behavior

A vtkSphereWidget could be created and displayed surrounding a single or a set of selected objects when Manipulate mode is invoked. The widget would remain active until Manipulate mode was exited or all objects were deselected.

Tests: visually rendering selected objects

What values of ambient, specular and color are used in the first image?

• 

  Changes to color, ambient, specular and ambient ( all muscles are the same color in the original)

• 

  Using higher ambient setting and red colour to show selected state (inherited from Slicer2)

• 

  Using ambient setting to show selected state

• 

  Using a bounding box (vtkBoxWidget2) to show selected, bonus of allowing interaction with the model (one off test via the tkcon, need to design the callbacks to allow moving the selected model)

• 

  Using a sphere widget (vtkSphereWidget2) to show selected. Not recommended, as this is more used to position a point on the sphere, and probably won't work as intuitively as the bounding box