Slicer3:XCEDE use cases

Slicer3 use cases for XCEDE

Slicer use-case #1: Managing image processing project in a Neuroscience Lab

Overview

• Step0: Investigator creates a new project on the hosted repository
• Step1: Investigator uploads data from a study.
• Step2: Later, the investigator queries for all “Acquisitions” uploaded as part of the study.
• Step3: The investigator then assigns each acquisition to a researcher for a segmentation task.
• Step4: A researcher queries for all acquisition segmentations assigned to them.
• Step5: A researcher downloads some subset of those assignments in an XCEDE catalog, opens the catalog with Slicer and creates a label map.
• Step6: When finished, a researcher uploads completed work or work in progress and tags the work appropriately.

Notes

For example, we’ll use the public OASIS brains dataset on XNAT, and let “composite ID” for a visit be OAS1_0006_MR1, or OAS1_0009_MR1, etc.

• Each visit (such as OAS1_0006_MR1) may contain multiple acquisitions, assessments, and derived data.

• Each element's ID is made unique by the back end, so that each visit, acquisition, subject, study, etc. has a locally unique ID.
• CompositeIDs are made globally unique by appending the uri of the hosted repository.
• Each element can also have a user-assigned label without a uniquenss requirement (like scan001), which may be a more comfortable way for the user to refer to it .

There are different ways in which an ID might be generated for an element.

• user may supply a preferred placeholder ID to which the back end applies a uniqueness check. The requested ID is used if unique.
• no placeholder ID is supplied and the system just generates a unique ID.
• system-generated ID is returned to user, who can accept or reject that ID.

Assuming also that PUT and POST are swapped on the spreadsheet Jeff sent last week. Below, we use the following:

• PUT = CREATE NEW
• POST = UPDATE EXISTING

Stepping thru details

---

Step0. An investigator creates a new project in the hosted repository:

(not sure how to do this on command line -- should discuss this.) 

---

Step1. An investigator uploads all tagged data from a study (using an upload script, or a webGUI or something outside of Slicer) – for instance:

curl –X PUT -T localFile –u user:passwd http://central.xnat.org/CENTRAL_OASIS_CS/OAS1_0009_MR1/uploadFileName 

QUESTIONS:

• is uploadFileName the same as {id}?
• need to bundle data file and XCEDE wrapper somehow?

Response: Script looks for return value, parses error XML and posts message if upload failed.

---

Step2. Investigator queries for all acquisitions uploaded as part of the study, tagged with him as the investigator and with a particular protocol, which have not yet been assigned for segmentation. Maybe this is done via web form, maybe with a script or query tool inside Slicer.

curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&investigator=Marek%20Kubicki&protocol=fBIRN&technician=NULL

Response: Receives back XCEDE XML describing list of uris for appropriate acquisitions and return code; If return code is good, parse XML to get list of uris and display them using some tool or another. Gets a list like:

http://central.xnat.org/acquisition/OAS1_0006_MR1

http://central.xnat.org/acquisition/OAS1_0009_MR1

http://central.xnat.org/acquisition/OAS1_0054_MR1

…

http://central.xnat.org/acquisition/OAS1_0450_MR1

---

Step3. Investigator updates the metadata with some segmentation assignments.

curl –X POST http://central.xnat.org/acquisition/OAS1_0006_MR1?technician=Tech1&status=assigned 
curl –X POST http://central.xnat.org/acquisition/OAS1_0009_MR1?technician=Tech2&status=assigned 
curl –X POST http://central.xnat.org/acquisition/OAS1_0054_MR1?technician=Tech2&status=assigned
…
curl -X POST http://central.xnat.org/acquisition/OAS1_0450_MR1?technician=Tech2&status=assigned

Response: Get XCEDE back: QUESTION: what is returned here?

---

Step4. Technician “Tech2” queries for all of their assignments from this particular investigator.

curl –X GET http://central.xnat.org/acquisitions?technician=Tech2&investigator=Marek%20Kubicki&status=assigned 

Response: Receives back XCEDE XML describing list of uris for acquisitions they need to segment. If return code is good, parse XML to get list of uris and display them using some tool or another.

---

Step5. Technician downloads an XCEDE catalog describing a single acquisition

curl –X GET http://central.xnat.org/acquisition/OAS1_0009_MR1?view=XCAT

Or, technician downloads all of their assigned acquisitions:

curl –X GET http://central.xnat.org/acquisitions?project=CENTRAL_OASIS_CS&technician=Tech2&view=XCAT

QUESTION: are multiple XCEDE catalogs returned, or one composite catalog?

Technician opens the catalog in Slicer and works on the segmentation.

Response: If return code is not good, post error.

---

Step6. At the day’s end, he wants to upload his work in progress with updated status

QUESTION: doing a ‘post’ will presumably overwrite the original data (if it were present in the DB). Does the back end do some automatic versioning? Or should we do this explicitly on the client side, and then perform a ‘put’ of the new version instead of a ‘post’.

Post a file that already has a uri on the DB, retrieved from the XCAT:

curl –X POST {uri}?status=in%20progress

Put a newly created file:

curl –X PUT -T updated-aseg.mgz http://central.xnat.org/acquisition/OAS1_0009_MR1/{path}/aseg.mgz?  {all tags from the acquisition?}  &status=completed

QUESTION: what’s a good way to describe with metadata, and to know the appropriate path?

Response: Get XCEDE back. QUESTION: again, what's in here, and what to do with it?

---

Slicer use-case #2: Query Atlas Module

Contents:

This is a record of the way we're using the xcede2.0 catalog in order to load in a query atlas scene that contains:

• One FreeSurfer "lh" model and one FreeSurfer "rh" model at most
• the FreeSurfer structural brain.mgz image volume
• the FreeSurfer example functional example_func.nii
• some FIPS-generated Statistical volumes
• the FreeSurfer aparc+aseg.mgz label map volume
• the FreeSurfer lh.aparc.annot annotation file
• the matrix to align the example_func + statistics to the brain.mgz: anat2exf.register.dat

Assumptions:

• we are assuming a single lh or rh model (or one of each) is in the catalog. (Reason: Slicer will load the lh.aparc.annot file as a scalar overlay onto a model; so the overlay needs to be associated with a model automatically during load. Since the catalog attribute list contains no references to other datasets, the only way to associate the lh.aparc.annot to the lh.pial surface is by matching on the "lh." and "rh." in the model and annotation names.)

• we are detecting whether a matrix with the name "anat2efx" is in the catalog (we should probably check the uri instead) -- and if so, we are wrapping the list of all FIPS-generated statistical volumes and the example_func(noted above) in a derived transform in the load routine. We recognize files with the string "brain.mgz" "example_func" and "anat2exf" in them, in order to do some automatic calculation of the Slicer transform that does the Slicer ras2ras registration. Then we stuff volumes with "example_func" or "stat" in their name into the derived transform.
  • the tests to detect these files are weak -- need a better test: now, a structural volume with the string 'stat' embedded in it will be stuffed into the transform by mistake.
  • no registration transform is computed if volumes matching the brain.mgz, example_func and an anat2exf.register.dat file are not detected.

New value inventions for attributes:

• Freesurfer:matrix-1 (this gives us a clue about file format).
• Freesurfer:overlay-1 (for files that need to be associated with a model in order to load in Slicer)
• Freesurfer:label-1 (for mgz files that are label maps) This gives us a hint about what kind of color map to use to display the image data. The default colormap for volume data is greyscale.

Validation:

Slicer use-case #3: Qdec Module

Description needed...

Prototype XCAT Files

<XCEDE>
<catalog ID="ID0">

<catalogList>

<catalog ID="ID1">
<entryList>

<entry ID="ID2" name="anat2exf" description="matrix that registers anatomical information to functional datasets" format="FreeSurfer:matrix-1" content="anat2exf" cachePath="" modelID="ID2" uri="./anat2exf.register.dat"/>

<entry ID="ID3" name="lh.pial" description="pial surface of left hemisphere" format="FreeSurfer:surface-1" content="lh.pial" cachePath="" uri="./lh.pial"/>

<entry ID="ID4" name="rh.pial" description="pial surface of right hemisphere" format="FreeSurfer:surface-1" content="rh.pial" cachePath="" uri="./rh.pial"/>

<entry ID="ID5" name="brain" description="extracted brain mri" format="FreeSurfer:mgz-1" content="brain" cachePath="" uri="./brain.mgz"/>

<entry ID="ID6" name="exf" description="example functional dataset" format="nifti:nii-1" content="example_func" cachePath="" uri="./example_func.nii"/>

<entry ID="ID7" name="zstat7" description="7th zstatisic contrast E1+E3+E5>Fix" format="nifti:nii-1" content="zstat7" cachePath="" uri="./zstat7.nii"/>

<entry ID="ID8" name="zstat14" description="14th zstatisic contrast P1+P3+P5>Fix" format="nifti:nii-1" content="zstat14" cachePath="" uri="./zstat14.nii"/>

<entry ID="ID9" name="zstat17" description="17th zstatisic contrast Learn>Fix" format="nifti:nii-1" content="zstat17" cachePath="" uri="./zstat17.nii"/>

<entry ID="ID10" name="aparc+aseg" description="parcellation and segmentation label map" labelmap="true" format="FreeSurfer:mgz-1" content="aparc+aseg" cachePath="" uri="./aparc+aseg.mgz"/>

<entry ID="ID11" name="lh.aparc.annot" description="annotations for surface of left hemisphere" format="FreeSurfer:overlay-1" content="lh.aparc.annot" cachePath="" uri="./lh.aparc.annot"/>

<entry ID="ID12" name="rh.aparc.annot" description="annotations for surface of right hemisphere" format="FreeSurfer:overlay-1" content="rh.aparc.annot" cachePath="" uri="./rh.aparc.annot"/>

</entryList>
</catalog>

</catalogList>

</catalog>
</XCEDE>

XCEDE REST Interface

A straw man REST interface to XCEDE. The implementation is using Grails (Groovy on Rails). Grails is a rapid application development environment patterned after Rails (Ruby on Rails). Grails, however, is completely Java-based utilizing industry standard components (Spring, Hibernate, Groovy). In addition, Grails supports many of the common AJAX frameworks and can be compiled and deployed as a .war file.

REST Interaction

REST support four basic operations akin to CRUD, Create, Read, Update and Delete. These verbs operate on resources (nouns) exposed as URIs, with CRUD relating to HTTP operations:

HTTP	CRUD
POST	Create, Update, Delete
GET	Read
PUT	Create, Update
DELETE	Delete

A server has been set up on slicerl using code in the NAMICSandbox (see next section for details). Two interfaces are available, a human-friendly web site and a machine-friendly REST interface. The command line program curl can interact with the REST interface, and is useful for learning and debugging.

Retrieve a list of available Projects

7: curl -X GET http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/
<?xml version="1.0" encoding="UTF-8"?><list>
  <project id="1">
    <createdDateTime>2008-06-18 21:36:50.291</createdDateTime>
    <description>Neurological implications of working at BWH</description>
    <name>BWH-0001</name>
    <subjectGroups>
      <subjectGroup id="1"/>
    </subjectGroups>
  </project>
  <project id="2">
    <createdDateTime>2008-06-18 21:36:50.332</createdDateTime>
    <description>Bar</description>
    <name>Foo</name>
    <subjectGroups>
      <subjectGroup id="2"/>
    </subjectGroups>
  </project>
</list>

Update an existing Project (Note how Project 2 has been changed)

8: curl -X GET http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/2 > Project2.xml
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   242    0   242    0     0    802      0 --:--:-- --:--:-- --:--:--     0

9: cat Project2.xml 
<?xml version="1.0" encoding="UTF-8"?><project id="2">
  <createdDateTime>2008-06-18 21:36:50.332</createdDateTime>
  <description>Bar</description>
  <name>Foo</name>
  <subjectGroups>
    <subjectGroup id="2"/>
  </subjectGroups>

11: perl -e 's/Bar/Garf/g' -pi Project2.xml

12: curl -X PUT -H 'Content-Type: application/xml' -d @Project2.xml http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/2
<?xml version="1.0" encoding="UTF-8"?><project id="2">
  <createdDateTime>2008-06-18 21:36:50.332</createdDateTime>
  <description>Garf</description>
  <name>Foo</name>
  <subjectGroups>
    <null/>
  </subjectGroups>
</project>

Create a new project (Note using POST creates a new project, while PUT updates an existing project, this command created a project with id=3)

15:  curl -X POST -H 'Content-Type: application/xml' -d @Project2.xml http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/
<?xml version="1.0" encoding="UTF-8"?><project id="3">
  <createdDateTime>2008-06-18 21:43:04.790 EDT</createdDateTime>
  <description>Garf</description>
  <name>Foo</name>
  <subjectGroups>
    <null/>
  </subjectGroups>
</project>

Delete a project

17: curl -X DELETE http://slicerl.bwh.harvard.edu:8080/XCEDE/rest/v1/project/3

Grails/XCEDE development

So, you're ready to kick the tires? Great! The first step is to check out the code from Subversion (make sure you do the second step!).

svn co http://www.na-mic.org/svn/NAMICSandBox/trunk/XCEDE

Resources that come with the Grails binary installation are omitted from Subversion and must be generated.

cd XCEDE
grails upgrade

and answer "yes" to both questions.

The main code is in XCEDE/grails-app/controllers/. Grails uses an MVC model and "coding by convention". Thus, if code is put in the correct place, it works without configuration. This is handy if you know what you are doing, frustrating if not. I would suggest going through "Getting Started with Grails" as a first tutorial.