Difference between revisions of "Documentation/Nightly/Developers/Tutorials/UploadTestData"

From Slicer Wiki
Jump to: navigation, search
m (Created page with '= Introduction = Slicer has adopted [http://slicer.kitware.com/midas3 MIDAS] to store testing and algorithm validation dataset. For this purpose, we have created a dedicated [ht…')
 
 
(23 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
<noinclude>{{documentation/versioncheck}}</noinclude>
 
= Introduction =
 
= Introduction =
  
Line 4: Line 5:
  
 
In the NA-MIC community [http://slicer.kitware.com/midas3/folder/301 <code>Slicer/Data</code>] folder, you will find:
 
In the NA-MIC community [http://slicer.kitware.com/midas3/folder/301 <code>Slicer/Data</code>] folder, you will find:
−
* Application
+
* [http://slicer.kitware.com/midas3/folder/302 Application]
−
* Modules
+
* [http://slicer.kitware.com/midas3/folder/310 Modules]
  
 
To organize the test data based on Slicer major release, you will find sub folders like:
 
To organize the test data based on Slicer major release, you will find sub folders like:
 
* <code>Nightly</code>
 
* <code>Nightly</code>
−
* <code>4.2</code>.  
+
* <code>4.2</code>
 +
* <code>4.3</code>
  
−
{{note}} Each time a  release of Slicer is done, for both the <code>Application</code> and <code>Modules/&lt;ModuleName&gt;<code> folders, the current <code>Nightly</code> is copied into an <code>X.Y</code> folders where X and Y corresponds to the major and minor release number of Slicer.
+
Test data are expected to be stored in folders like:
  
−
Test data are expected to be stored in folders likes:
+
* <code>Data/Application/Nightly/Testing/Baseline</code>
 +
* <code>Data/Application/Nightly/Testing/Input</code>
 +
* <code>Data/Modules/&lt;NameOfYourModule&gt;/Nightly/Testing/Baseline</code>
 +
* <code>Data/Modules/&lt;NameOfYourModule&gt;/Nightly/Testing/Input</code>
  
−
* Nightly/Testing/Baseline
+
* <code>Data/Application/X.Y/Testing/Baseline</code>
−
* Nightly/Testing/Input
+
* <code>Data/Application/X.Y/Testing/Input</code>
−
* X.Y/Testing/Baseline
+
* <code>Data/Modules/&lt;NameOfYourModule&gt;/X.Y/Testing/Baseline</code>
−
* X.Y/Testing/Input
+
* <code>Data/Modules/&lt;NameOfYourModule&gt;/X.Y/Testing/Input</code>
  
−
#'''Input data folder''': Used to store testing input data.
+
where
−
#'''Baseline dataset folder''': Used to store the baseline images for regression tests.
 
  
−
= How to add a new tubetk test =
+
* <code>&lt;NameOfYourModule&gt;</code>: name of the folder in the source repository, it should also match the module pages created on the slicer wiki.
 +
* <code>Input</code>: Used to store testing input data.
 +
* <code>Baseline</code>: Used to store the baseline images for regression tests.
  
−
#If the input data is not already in Midas,
+
= Upload test data =
−
##Upload the data to the "Data" folder
 
−
##Download the key file (*.md5) and store it in the MIDAS_KEYS directory at the top level of tubetk source code tree
 
−
#Add baseline dataset for the new test ( if your test is a regression testing )
 
−
##Upload the dataset to the appropriate CLI or module folder. If your test is part of a new "module" or CLI, create a new a folder in MIDAS and then upload the dataset
 
−
##Download the key file  (*.md5) for the baseline image and store it in the MIDAS_KEYS directory.
 
−
#Modify cmake list files in the Testing directory to "add" unit tests to use the data from MIDAS.
 
−
##Use MIDAS_FETCH_ONLY macro, if you would like to fetch the dataset but not use it directly in the test
 
−
##Use MIDAS_DIRECTORY, if you would like to specify a whole directory as an input command line argument to your test
 
  
−
Examples:  
+
<ol start="1" style="list-style-type: decimal;">
  
−
Test using a data store in MIDAS
+
{{:Documentation/{{documentation/version}}/Developers/Tutorials/CreateMidasAccount/steps}}
  
−
MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test4
+
<li>Send an email on the developer list asking to be added to the <code>DataManager</code> group on NA-MIC community. That will grant you read/write permissions to the <code>Data</code> folder and sub-folders.</li>
−
            ${PROJ_EXE}
+
<li>Create a sub folder for your Module.</li>
−
            MIDAS{SyntheticTubePlusSurfaceImage_oriented.mha.md5}
+
<li>Create a <code>Nightly</code> sub-folder.</li>
−
            ${TEMP}/SyntheticTubePlusSurfaceEEDFilterResultImage_oriented.mha )
+
<li>Upload data in folders <code>Nightly/Testing/Baseline</code> and <code>Nightly/Testing/Input</code></li>
  
−
Test with a directory as input argument: DICOMSeries
+
</ol>
  
−
MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test1
+
<gallery widths=330px heights=220px perrow=4>
−
            ${PROJ_EXE}
+
File:Midas-test-data-upload-1.png|Browse, select the destination folder (1) and click on View (2)
−
              -i
+
File:Midas-test-data-upload-2.png|Select an item (3) and click "Upload here" (4)
−
              -r
+
File:Midas-test-data-upload-3.png|Select one or more files (5) and click "Upload" button (6)
−
              -a Test1 001
+
</gallery>
−
              -w
 
−
              -o ${TEMP}
 
−
              -b DSC_
 
−
              MIDAS_DIRECTORY{DICOMSeries} )
 
  
−
Test that demonstrates fetch only option: tubeEdgeEnhancingAnisotropicDiffusionMIDAS-Test1
+
= FAQ =
  
−
MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test1
+
== What happen when Slicer is released ? ==
−
            ${PROJ_EXE}
 
−
              MIDAS{CroppedWholeLungCTScan.mhd.md5}
 
−
              ${TEMP}/itkAnisotropicEdgeEnhancingDiffusionImageFilterTest.mha
 
−
              MIDAS_FETCH_ONLY{CroppedWholeLungCTScan.raw.md5}
 
−
              )
 
  
−
When setting up test dependencies in your CMakeLists.txt files describing the tests, use something like:
+
Each time a  release of Slicer is done, for both the <code>Application</code> and <code>Modules/&lt;NameOfYourModule&gt;</code> folders, the current <code>Nightly</code> is copied into an <code>X.Y</code> folders where X and Y corresponds to the major and minor release number of Slicer.
  
−
  set_property(TEST ${PROJECT_NAME}-TestSphereNoReg-Compare
+
== Should data for extension be organized differently ? ==
−
                      APPEND PROPERTY DEPENDS ${PROJECT_NAME}-TestSphereNoReg )
 
  
−
= MIDAS macro =
+
For now, let's keep all of them at the same level. We will have to coordinate with the development of the release script allowing to version the Data tree. See [http://slicer-devel.65872.n3.nabble.com/Versioning-of-test-data-tree-on-slicer-kitware-com-What-does-this-mean-for-you-tt4026608.html here]
−
The following are the essential midas macros for setting up a test using data stored in midas.
+
 
−
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#How_to_create_a_test_that_will_download_and_use_MIDAS-hosted_data MIDAS_ADD_TEST]
+
It means if you develop an extension named <code>Salad</code> bundling two modules respectively named <code>Tomato</code> and <code> Mozzarella</code>, the following folders would have to be created:
−
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_TGZ_option MIDAS_TGZ ]
+
 
−
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_FETCH_ONLY_option MIDAS_FETCH ONLY]
+
* <code>Data/Modules/Tomato/Nightly/Testing/Baseline</code>
−
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_DIRECTORY_option MIDAS_DIRECTORY]
+
* <code>Data/Modules/Tomato/Nightly/Testing/Input</code>
 +
* <code>Data/Modules/Mozzarella/Nightly/Testing/Baseline</code>
 +
* <code>Data/Modules/Mozzarella/Nightly/Testing/Input</code>
 +
 
 +
Consider also monitoring issue [http://www.na-mic.org/Bug/view.php?id=2722 #2722]: Think about organization of Data/Modules directory on midas to accomodate extension relationship

Latest revision as of 15:23, 27 August 2013

Home < Documentation < Nightly < Developers < Tutorials < UploadTestData


For the latest Slicer documentation, visit the read-the-docs.


Introduction

Slicer has adopted MIDAS to store testing and algorithm validation dataset. For this purpose, we have created a dedicated NA-MIC MIDAS community

In the NA-MIC community Slicer/Data folder, you will find:

To organize the test data based on Slicer major release, you will find sub folders like:

  • Nightly
  • 4.2
  • 4.3

Test data are expected to be stored in folders like:

  • Data/Application/Nightly/Testing/Baseline
  • Data/Application/Nightly/Testing/Input
  • Data/Modules/<NameOfYourModule>/Nightly/Testing/Baseline
  • Data/Modules/<NameOfYourModule>/Nightly/Testing/Input
  • Data/Application/X.Y/Testing/Baseline
  • Data/Application/X.Y/Testing/Input
  • Data/Modules/<NameOfYourModule>/X.Y/Testing/Baseline
  • Data/Modules/<NameOfYourModule>/X.Y/Testing/Input

where

  • <NameOfYourModule>: name of the folder in the source repository, it should also match the module pages created on the slicer wiki.
  • Input: Used to store testing input data.
  • Baseline: Used to store the baseline images for regression tests.

Upload test data

  1. Create an account on the extension server: http://slicer.kitware.com by clicking on the Register link in the top right corner

    2. Slicer-midas-extensions-server-registration.png

    This image shows the top portion of http://slicer.kitware.com after it has been expanded by clicking the Register button.

  2. Go to NA-MIC community and click on Join community

  3. Send an email on the developer list asking to be added to the DataManager group on NA-MIC community. That will grant you read/write permissions to the Data folder and sub-folders.
  4. Create a sub folder for your Module.
  5. Create a Nightly sub-folder.
  6. Upload data in folders Nightly/Testing/Baseline and Nightly/Testing/Input

  • Browse, select the destination folder (1) and click on View (2)

  • Select an item (3) and click "Upload here" (4)

  • Select one or more files (5) and click "Upload" button (6)

FAQ

What happen when Slicer is released ?

Each time a release of Slicer is done, for both the Application and Modules/<NameOfYourModule> folders, the current Nightly is copied into an X.Y folders where X and Y corresponds to the major and minor release number of Slicer.

Should data for extension be organized differently ?

For now, let's keep all of them at the same level. We will have to coordinate with the development of the release script allowing to version the Data tree. See here

It means if you develop an extension named Salad bundling two modules respectively named Tomato and Mozzarella, the following folders would have to be created:

  • Data/Modules/Tomato/Nightly/Testing/Baseline
  • Data/Modules/Tomato/Nightly/Testing/Input
  • Data/Modules/Mozzarella/Nightly/Testing/Baseline
  • Data/Modules/Mozzarella/Nightly/Testing/Input

Consider also monitoring issue #2722: Think about organization of Data/Modules directory on midas to accomodate extension relationship

Retrieved from "https://www.slicer.org/w/index.php?title=Documentation/Nightly/Developers/Tutorials/UploadTestData&oldid=34696"