Difference between revisions of "Documentation/Nightly/Modules/ShapeVariationAnalyzer"

From Slicer Wiki
Jump to: navigation, search
Tag: 2017 source edit
 
(29 intermediate revisions by 3 users not shown)
Line 8: Line 8:
 
{{documentation/{{documentation/version}}/module-introduction-start|{{documentation/modulename}}}}
 
{{documentation/{{documentation/version}}/module-introduction-start|{{documentation/modulename}}}}
 
{{documentation/{{documentation/version}}/module-introduction-row}}
 
{{documentation/{{documentation/version}}/module-introduction-row}}
−
This work was supported by the National Institutes of Dental and Craniofacial Research and Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Number R01DE024450.<br>
+
Extension: '''ShapeVariationAnalyzer'''<br>
 +
Acknowledgments: This work was supported by the National Institutes of Dental and Craniofacial Research and Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Number R01DE024450.<br>
  
−
Author: Priscille de Dumast (University of Michigan)<br>
+
Authors: Mateo Lopez (university of North Carolina) and Priscille de Dumast (University of Michigan)<br>
 
Contributor1: Laura Pascal (University of Michigan)<br>
 
Contributor1: Laura Pascal (University of Michigan)<br>
 
Contributor2: Lucia Cevidanes (University of Michigan)<br>
 
Contributor2: Lucia Cevidanes (University of Michigan)<br>
 
Contributor3: Juan Carlos Prieto (University of North Carolina)<br>
 
Contributor3: Juan Carlos Prieto (University of North Carolina)<br>
−
Contact: Priscille de Dumast, <email>prisgdd@umich.edu</email><br>
+
Contact: Mateo lopez, <email>lpzmateo@unc.edu</email> <br>
 +
Priscille de Dumast, <email>prisgdd@umich.edu</email> <br>
  
 
License: [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0]
 
License: [http://www.apache.org/licenses/LICENSE-2.0.html Apache License, Version 2.0]
Line 27: Line 29:
 
<!-- ---------------------------- -->
 
<!-- ---------------------------- -->
 
{{documentation/{{documentation/version}}/module-section|Module Description}}
 
{{documentation/{{documentation/version}}/module-section|Module Description}}
 +
 
{{documentation/{{documentation/version}}/module-description}}
 
{{documentation/{{documentation/version}}/module-description}}
−
Shape Variation Analyzer allows the classification of 3D models, according to their morphological variation.
+
Shape Variation Analyzer (SVA) allows the computation of PCA decomposition of groups of shapes in order be able to represent them in a lower dimensional space. SVA also allows the user to explore the generated PCA space and to evaluate the quality of the generated models. The aim of this extension is to help understand morphological variations.
−
This tool is based on a deep learning neural network.<br>
 
−
The module is composed of multiple panels to perform the different steps of the process: create the classification groups, compute their average shapes, train the classifier and classify shapes.
 
 
<br><br>
 
<br><br>
  
−
=====What is an artificial neural network?=====
+
The module is composed of multiple panels to perform the different steps of the process: create the groups, pre-visualize the groups and update them if necessary, generate explore and evaluate the generated models.
−
A neural network is a computing system, inspired by our own human brain. It learns from a large dataset (training dataset) containing both the input and the expected output (in our case, the 3D model and its morphology classification). During the training, the network’s settings are adjusted until we achieve automatic classification that matches the expert classification.<br>
+
<br><br>
−
The network can then be tested with any input, preferably not from the training dataset. This allows to evaluate the performance of the network.
 
−
 
 
  
 
<!-- ---------------------------- -->
 
<!-- ---------------------------- -->
 
{{documentation/{{documentation/version}}/module-section|Use Cases}}
 
{{documentation/{{documentation/version}}/module-section|Use Cases}}
 
{|
 
{|
 +
|[[Image:ShapeVariationAnalyzer-Logo.png|thumb|200px|Logo for ShapeVariationAnalyzer]]
 +
|[[Image:SVA-exploration.png|thumb|300px|Interface of ShapeVariationAnalyzer]]
 +
|[[Image:SVA-full-view.png|thumb|600px|ShapeVariationAnalyzer in Slicer]]
 
|}
 
|}
 +
  
 
<!-- ---------------------------- -->
 
<!-- ---------------------------- -->
 
{{documentation/{{documentation/version}}/module-section|Tutorials}}
 
{{documentation/{{documentation/version}}/module-section|Tutorials}}
 +
===Requirements===
 +
 +
* Windows
 +
To use this extension on Windows, you '''must''' run 3D Slicer as administrator.
 +
 +
* MacOS
 +
Mac OS 10.11.0 or later
 +
 +
* Linux
 +
 +
<br><br>
 +
 
===Prerequisities===
 
===Prerequisities===
  
−
The classification can be performed with as many groups as desired. <br>
+
Both the 3D models into the training dataset and those to classify must have the same number of points. The most correspondent the shapes are, the better it is for the computation.
−
Please note that the amount of data is very important. The more data you have, the more accurate the classification will be. Moreover, it is better to have a dataset equally divided into the classes. <br>
+
This can be performed thanks to ShapeAnalysisModule and improved with RigidAlignment/Groups.
−
<br>
 
−
Both the 3D models into the training dataset and those to classify must have the same number of points. The most correspondent the shapes are, the best it is for the computation.
 
−
Those 2 steps can be performed thanks to ShapeAnalysisModule and RigidAlignment/Groups.
 
  
 
<br><br>
 
<br><br>
Line 60: Line 72:
 
|
 
|
 
This tab allows the user to create a CSV file gathering all the VTK file paths and their corresponding group.<br>
 
This tab allows the user to create a CSV file gathering all the VTK file paths and their corresponding group.<br>
−
This CSV file will be necessary at each step of the classification. <br>
+
This CSV file lists all the files used in the <u>training dataset</u>. It will be necessary at each step of the process. <br>
 +
<br><br>
 
The file is automatically loaded into the next steps when it is exported.<br>
 
The file is automatically loaded into the next steps when it is exported.<br>
 
|}
 
|}
Line 109: Line 122:
 
<br>
 
<br>
  
−
===Computation of mean groups===
+
 
 +
===Generating and saving the models===
  
 
{|
 
{|
−
|[[Image:ComputationAverage-1.png|350px]]
+
This step will compute the PCA model of each group and will save those models for a future exploration. <br>
−
|
+
You’ll need to provide the CSV file which contains the groups.
−
This tab allows the computation of the average shape of each group, export those resulting models locally and allows their visualization into 3D Slicer. <br>
 
−
You’ll need to provide the CSV file which contains the classification groups. <br>
 
−
The exported CSV file will be automatically loaded into the next step.
 
 
|}
 
|}
 
<br>
 
<br>
Line 122: Line 133:
 
* Perform the computation
 
* Perform the computation
 
{|
 
{|
−
|[[Image:ComputationAverage-computation.png|350px]]
+
# Load the CSV file containing the groups if you haven't performed the previous steps
−
|
+
# Clic on '''Process and Export'''
−
# Load the CSV file containing the classification groups, if you haven't performed the previous steps
+
# The exploration interface will appear, it gives you all the tools to explore the generated models.  
−
# Clic on '''Compute mean group'''
 
−
# Chose a repository to export the average shapes and a CSV File listing them.  
 
−
Average shapes : ''meanGroup[i].vtk'' <br>
 
−
CSV File mean shapes : ''MeanGroups.csv''
 
 
|}
 
|}
 
<br>
 
<br>
  
−
* Preview of Group's mean
+
* Save the models
 
{|
 
{|
−
|[[Image:ComputationAverage-preview.png|350px]]
+
# Clic on save exploration.
−
|
+
# Choose the folder in wich the exploration should be saved and give it a name (ex: 'exploration-name.json').
−
# Load the CSV file containing all the shapes if you haven't performed the computation step.
+
# Saving the exploration will create files as follow:
−
# Select the control group - it will appear in a different color.
+
 
−
# Clic on '''Preview''' to visualize the average shapes
+
JSON file : ''exploration-name.json'' <br>
−
# Check the eye icon on the left to choose which shape you want to display
+
Average shapes : ''exploration-name_[i]_mean.vtk'' <br>
 +
PYC file : ''exploration-name.pyc'' <br>
 
|}
 
|}
 
<br>
 
<br>
  
−
===Morphological classification===
+
* Load an exploration
 +
 
 +
To load an exploration select a JSON file previously generated ('exploration-name.json') in the JSON File field.<br>
 +
 
 +
===Model Exploration===
 
{|
 
{|
−
|[[Image:MorphologicalClassification.png|350px]]
+
[[File:SVA-interface.png|thumb|right|exploration interface]]
−
|
+
This section is divided into 3 part:  
−
This tab is divided into 2 panels:  
+
# The PCA exploration
−
# The creation and the training of the neural network
+
# The color modes description and parameterization
−
# The classification using a neural network previously trained
+
# The plots visualization and interaction
 +
 
 +
 
  
 
|}
 
|}
 
<br>
 
<br>
 +
<br>
 +
<br>
 +
<br>
 +
<br>
 +
<br>
 +
<br>
 +
<br>
 +
<br>
 +
  
−
* Create Network
+
'''The PCA exploration'''
 
{|
 
{|
−
|[[Image:MorphologicalClassification-CreateNetwork.png|300px]]
+
[[File:SVA-exploration.png|thumb|right|exploration]]
−
|
+
To start exploring a PCA space, select the group you want to explore in the group option.<br>
−
#  
+
Use the sliders to see the shape transform. Each slider represent an eigenvalue of the PCA model, the explained variance value represent how significant this eigenvalue is compared to the other ones.<br>
−
#
+
<br>
 +
 
 +
If you want to play with more sliders, you can use the 'Minimum Explained Variance' and 'Maximum Number of eigenvalues' parameters:
 +
*The 'Minimum Explained Variance' parameter will make appear only sliders that have a higher explained variance than the defined value.
 +
*The 'Maximum Number of eigenvalues' parameter is the maximum number of sliders that can be displayed.
 +
<br>
 +
If you want to see (or hide) the mean shape of the group in a superposition of the current shape, you can use the 'toggle mean shape' option.
 +
 
 +
 
 +
'''The color modes description and parameterization'''<br>
 +
[[File:SVA-group-color.png|thumb|right|group color|200px]]
 +
[[File:SVA-unsigned-distance.png|thumb|right|unsigned distance|200px]]
 +
[[File:SVA-signed-distance.png|thumb|right|signed distance|200px]]
 +
 
 +
There are 3 different color modes to visualize the current shape that can be selected in the Color Mode parameter:
 +
# the group color mode
 +
# the unsigned distance to mean shape mode
 +
# the signed distance to mean shape mode
 +
<br>
 +
 
 +
*The group color mode:
 +
This mode will display the shape with the color associated with his group.<br>
 +
To change the color of a group, click on the white colored square next to the group selection and select a new color.<br>
 +
<br>
 +
<br>
 +
<br>
 +
*The unsigned distance to mean shape mode
 +
This mode will display, at each point of the shape, a color representing the unsigned distance between the point and the corresponding point of the group mean shape.<br>
 +
The color scale used is white(0) to red(maximal distance). You can change the color range (ie: the maximal distance) by using the Maximum distance parameter.<br>
 +
<br>
 +
<br>
 +
<br>
 +
*The signed distance to mean shape mode
 +
This mode will display, at each point of the shape, a color representing the signed distance between the point and the corresponding point of the group mean shape.<br>
 +
The color scale used is blue(minimal distance) to white(0) to red(maximal distance). You can change the color range (ie: the maximal or minimal distance) by using the Maximum distance and Minimal distance parameter. In any case, the white color is fixed to a 0 distance.<br>
 +
A distance will be signed negatively if the associated point is inside the mean shape of the group and positively if the associated point is outside the mean shape of the group.<br>
 +
 
 
|}
 
|}
 +
<br>
 +
<br>
 +
<br>
 
<br>
 
<br>
  
−
* Classify shape(s)
+
'''The plots visualization and interaction'''
 
{|
 
{|
−
|[[Image:MorphologicalClassification-Classify.png|300px]]
+
 
−
|
+
After generation of the models, 2 plots are available:
−
#
+
*The Variance plot
−
#
+
*The Projection plot (interactive)<br>
 +
<br>
 +
If an evaluation of the models has been processed, 3 other plots are available:
 +
[[File:Plot-menu.png|300px|thumb|right|plot area menu]]
 +
*The Generalization plot
 +
*The Specification plot
 +
*The Compactness plot
 +
We describe them in the Model Evaluation section. <br>
 +
<br>
 +
 
 +
They can be visualized by selecting the corresponding plot chart in the plot area menu (top left button of the plot area).
 +
 
 +
<br><br>
 +
[[File:SVA-explained-variance.png|300px|thumb|right|explqined variance plot]]
 +
[[File:SVA-projection.png|300px|thumb|right|projection plot]]
 +
*The Variance plot <br>
 +
This plot shows, for the selected group, the explained variance ratio of each eigenvalue in a bar plot and the cumulated sum of the explained variance ratio in a scatter plot. <br>
 +
The number of components displayed corresponds to the number of sliders available.<br>
 +
<br><br>
 +
*The Projection plot (interactive)<br>
 +
This plot shows the projection on the 2 first component of the selected group.<br>
 +
You can interact with this plot using a point selection mode ('select points' or 'freehand select points') available in the interaction mode option of the plot area menu.<br>
 +
<br>
 +
Selecting one point on this plot will modify the sliders values to show the corresponding shape in the 3D window.<br>
 +
If you select more than 1 point, the mean shape of all the selected points will be displayed.<br>
 +
<br>
 +
The point selection modifies ALL the sliders, even those who are not displayed on the interface. if you want to show the shape only using the displayed sliders and not the other ones, you can uncheck the 'Use hidden eigenmodes' option, at the bottom of the sliders.
 
|}
 
|}
 
<br>
 
<br>
  
−
===Results/Analysis===
+
 
 +
===Model Evaluation===
 
{|
 
{|
−
|[[Image:ResultsAnalysis.png|350px]]
+
To evaluate the generated models, 3 values can be computed:
−
|
+
# The generalization
−
#
+
# The specificity
−
#
+
# The compactness
 +
 
 +
'''launching the evaluation'''<br>
 +
two parameters are available for the evaluation. one is the number of random shapes that have to be generated to compute the specificity. The other one is defined by the number of sliders currently available if we have 8 sliders, the evaluation is computed on 8 components.<br>
 +
To launch the evaluation, use the Evaluate models option. As the evaluation of the models can take a long time, you have the possibility to abort the evaluation by using the same button.<br>
 +
<br>
 +
'''Visualize the evaluation'''<br>
 +
When the evaluation is completed, 3 new plots are available in the plot area, one for each value.<br>
 +
You can visualize them by choosing the corresponding plot chart in the plot area menu.
 +
<br>
 +
<gallery>
 +
SVA-specificity.png|Specificity
 +
SVA-generalization.png|Generalization
 +
SVA-compactness.png|Compactness
 +
</gallery>
 +
 
 
|}
 
|}
 +
<br>
  
  
Line 191: Line 296:
 
<!-- ---------------------------- -->
 
<!-- ---------------------------- -->
 
{{documentation/{{documentation/version}}/module-section|Information for Developers}}
 
{{documentation/{{documentation/version}}/module-section|Information for Developers}}
−
This extension has been designed to work with this [https://ec2-52-42-49-63.us-west-2.compute.amazonaws.com:8180/DCBIA-OrthoLab/public/ website] and should not work with other architectures.
+
The source code of this module is available on GitHub: https://github.com/DCBIA-OrthoLab/ShapeVariationAnalyzer <br>
−
If you want to use this plugin on another server, you need to make sure your documents are stored by CouchDB and your documents contains a field "type" set to "morphologicalData".
 
−
Your user would need to connect using [https://jwt.io/ JWT] and have a "scope" field.<br>
 
−
For more information, you can take a look at the website source code [https://github.com/DCBIA-OrthoLab/shiny-tooth here].<br>
 
−
The source code is available on [https://github.com/DCBIA-OrthoLab/DatabaseInteractorExtension/tree/release github]
 
  
 
<!-- ---------------------------- -->
 
<!-- ---------------------------- -->
 
{{documentation/{{documentation/version}}/module-footer}}
 
{{documentation/{{documentation/version}}/module-footer}}
 
<!-- ---------------------------- -->
 
<!-- ---------------------------- -->

Latest revision as of 22:54, 20 July 2020

Home < Documentation < Nightly < Modules < ShapeVariationAnalyzer


For the latest Slicer documentation, visit the read-the-docs.


Introduction and Acknowledgements

Extension: ShapeVariationAnalyzer
Acknowledgments: This work was supported by the National Institutes of Dental and Craniofacial Research and Biomedical Imaging and Bioengineering of the National Institutes of Health under Award Number R01DE024450.

Authors: Mateo Lopez (university of North Carolina) and Priscille de Dumast (University of Michigan)
Contributor1: Laura Pascal (University of Michigan)
Contributor2: Lucia Cevidanes (University of Michigan)
Contributor3: Juan Carlos Prieto (University of North Carolina)
Contact: Mateo lopez, <email>lpzmateo@unc.edu</email>
Priscille de Dumast, <email>prisgdd@umich.edu</email>

License: Apache License, Version 2.0

  • University of Michigan

  • NAMIC

  • UNC

Module Description

Shape Variation Analyzer (SVA) allows the computation of PCA decomposition of groups of shapes in order be able to represent them in a lower dimensional space. SVA also allows the user to explore the generated PCA space and to evaluate the quality of the generated models. The aim of this extension is to help understand morphological variations.

The module is composed of multiple panels to perform the different steps of the process: create the groups, pre-visualize the groups and update them if necessary, generate explore and evaluate the generated models.

Use Cases

Logo for ShapeVariationAnalyzer
Interface of ShapeVariationAnalyzer
ShapeVariationAnalyzer in Slicer


Tutorials

Requirements

  • Windows

To use this extension on Windows, you must run 3D Slicer as administrator.

  • MacOS

Mac OS 10.11.0 or later

  • Linux



Prerequisities

Both the 3D models into the training dataset and those to classify must have the same number of points. The most correspondent the shapes are, the better it is for the computation. This can be performed thanks to ShapeAnalysisModule and improved with RigidAlignment/Groups.



Creation of CSV File

CreationofCSV-1.png

This tab allows the user to create a CSV file gathering all the VTK file paths and their corresponding group.
This CSV file lists all the files used in the training dataset. It will be necessary at each step of the process.


The file is automatically loaded into the next steps when it is exported.


CreationofCSV-AddGroup.png
CreationofCSV-ModifyGroup.png
CreationofCSV-ModifyRemove.png

Add Group Add one by one the directories containing the VTK files for each group.
For a new group, the group number is automatically selected.

Modify Group Each group can be edited, with a modification of its corresponding repository.
For this, review the group already added thanks to the spin box.

Remove Group Only the last group added can be removed.
Select the group with the spinbox and remove it.



Preview/Update of the classification groups

PreviewUpdateGroups-1.png

This tab allows the visualization of the 3D models in ShapePopulationViewer, and update the group assigned to the shapes if needed.
If the groups are updated, a new CSV file will be generated.


  • Preview with ShapePopulationViewer
PreviewUpdateGroups-details.png
  1. Select the groups or specific shapes you would like to visualize
  2. Colors that the meshes will have in Shape Population Viewer with the attribute "DisplayClassificationGroup"
  3. Possibility to change the group of a shape thanks to the spinbox in the group row.
  4. Clic on the Preview button, and select all the VTK Files in the pop-up window
  5. If some changes were made on the groups, please export those modifications.



Generating and saving the models

This step will compute the PCA model of each group and will save those models for a future exploration.
You’ll need to provide the CSV file which contains the groups.


  • Perform the computation
  1. Load the CSV file containing the groups if you haven't performed the previous steps
  2. Clic on Process and Export
  3. The exploration interface will appear, it gives you all the tools to explore the generated models.


  • Save the models
  1. Clic on save exploration.
  2. Choose the folder in wich the exploration should be saved and give it a name (ex: 'exploration-name.json').
  3. Saving the exploration will create files as follow:
JSON file : exploration-name.json
Average shapes : exploration-name_[i]_mean.vtk
PYC file : exploration-name.pyc


  • Load an exploration

To load an exploration select a JSON file previously generated ('exploration-name.json') in the JSON File field.

Model Exploration

exploration interface

This section is divided into 3 part:

  1. The PCA exploration
  2. The color modes description and parameterization
  3. The plots visualization and interaction












The PCA exploration

exploration

To start exploring a PCA space, select the group you want to explore in the group option.
Use the sliders to see the shape transform. Each slider represent an eigenvalue of the PCA model, the explained variance value represent how significant this eigenvalue is compared to the other ones.

If you want to play with more sliders, you can use the 'Minimum Explained Variance' and 'Maximum Number of eigenvalues' parameters:

  • The 'Minimum Explained Variance' parameter will make appear only sliders that have a higher explained variance than the defined value.
  • The 'Maximum Number of eigenvalues' parameter is the maximum number of sliders that can be displayed.


If you want to see (or hide) the mean shape of the group in a superposition of the current shape, you can use the 'toggle mean shape' option.


The color modes description and parameterization

group color
unsigned distance
signed distance

There are 3 different color modes to visualize the current shape that can be selected in the Color Mode parameter:

  1. the group color mode
  2. the unsigned distance to mean shape mode
  3. the signed distance to mean shape mode


  • The group color mode:

This mode will display the shape with the color associated with his group.
To change the color of a group, click on the white colored square next to the group selection and select a new color.



  • The unsigned distance to mean shape mode

This mode will display, at each point of the shape, a color representing the unsigned distance between the point and the corresponding point of the group mean shape.
The color scale used is white(0) to red(maximal distance). You can change the color range (ie: the maximal distance) by using the Maximum distance parameter.



  • The signed distance to mean shape mode

This mode will display, at each point of the shape, a color representing the signed distance between the point and the corresponding point of the group mean shape.
The color scale used is blue(minimal distance) to white(0) to red(maximal distance). You can change the color range (ie: the maximal or minimal distance) by using the Maximum distance and Minimal distance parameter. In any case, the white color is fixed to a 0 distance.
A distance will be signed negatively if the associated point is inside the mean shape of the group and positively if the associated point is outside the mean shape of the group.





The plots visualization and interaction

After generation of the models, 2 plots are available:
  • The Variance plot
  • The Projection plot (interactive)

If an evaluation of the models has been processed, 3 other plots are available:
plot area menu
  • The Generalization plot
  • The Specification plot
  • The Compactness plot

We describe them in the Model Evaluation section.

They can be visualized by selecting the corresponding plot chart in the plot area menu (top left button of the plot area).



explqined variance plot
projection plot
  • The Variance plot

This plot shows, for the selected group, the explained variance ratio of each eigenvalue in a bar plot and the cumulated sum of the explained variance ratio in a scatter plot.
The number of components displayed corresponds to the number of sliders available.


  • The Projection plot (interactive)

This plot shows the projection on the 2 first component of the selected group.
You can interact with this plot using a point selection mode ('select points' or 'freehand select points') available in the interaction mode option of the plot area menu.

Selecting one point on this plot will modify the sliders values to show the corresponding shape in the 3D window.
If you select more than 1 point, the mean shape of all the selected points will be displayed.

The point selection modifies ALL the sliders, even those who are not displayed on the interface. if you want to show the shape only using the displayed sliders and not the other ones, you can uncheck the 'Use hidden eigenmodes' option, at the bottom of the sliders.



Model Evaluation

To evaluate the generated models, 3 values can be computed:
  1. The generalization
  2. The specificity
  3. The compactness
launching the evaluation
two parameters are available for the evaluation. one is the number of random shapes that have to be generated to compute the specificity. The other one is defined by the number of sliders currently available if we have 8 sliders, the evaluation is computed on 8 components.
To launch the evaluation, use the Evaluate models option. As the evaluation of the models can take a long time, you have the possibility to abort the evaluation by using the same button.

Visualize the evaluation
When the evaluation is completed, 3 new plots are available in the plot area, one for each value.
You can visualize them by choosing the corresponding plot chart in the plot area menu.

  • Specificity

  • Generalization

  • Compactness



Similar Modules

N/A

References

N/A

Information for Developers

The source code of this module is available on GitHub: https://github.com/DCBIA-OrthoLab/ShapeVariationAnalyzer


Retrieved from "https://www.slicer.org/w/index.php?title=Documentation/Nightly/Modules/ShapeVariationAnalyzer&oldid=63347"