Slicer3:Batchmake Integration

Introduction

Goals

• Enable Slicer3 execution modules to process multiple datasets with minimal user interactions ("batch processing")
• Enable the exploration of the parameter spaces of Slicer3 execution modules ("parameter space evaluations")
• Support using local computer resources or local GRID resources for batch process and parameter space evaluations.

Summary

• BatchMake integration with Slicer3 is complete.
• Real-world applications (EMSegmenter) are using BatchMake from within Slicer.
• Examples are given on how to use BatchMake with your Slicer3 Execution Modules (GaussianBlur example).

Background

• BatchMake provides an open-source toolkit for launching a controlled sequence of applications on the local computer or a GRID.
• Condor is used as the open-source toolkit that provides access to GRID resource. It also provides an easy-to-install and easy-to-configure GRID implementation/network. We highly recommend it for your GRID needs.
• BatchMake and Condor work together to sequence tasks and to allocate and monitor resources and job progress.

Status

Completed

1. BatchMake Library
   • Distributed via Slicer3's getbuildtest2.tcl script
     • Slicer3 code location: Slicer3-lib/BatchMake
   • Default distribution does not include Condor
     • Can be installed by users
     • Changing two BatchMake variables will configure BatchMake to use Condor
2. EMSegmenter BatchMake module
   • Distributed via Slicer3's getbuildtest2.tcl script
     • Slicer3 GUI Access: Filters/BatchProcessing/EMSegmenter BatchMake Module
     • Slicer3 code location: Slicer3/Applications/CLI/BatchMake
   • Applies EMSegmenter algorithm to a collection of data using local computer or GRID
3. GaussianBlur BatchMake module
   • Distributed via Slicer3's getbuildtest2.tcl script
     • Slicer3 GUI Access: Filters/BatchProcessing/GaussianBlur BatchMake Module
   • Provides a simple example of how to batchmake-ify your Slicer3 modules

To do

• Provide documentation
• Determine how to integrate BatchMake with Slicer's new focus on iPython for distributed computing
• Consider general (no programming required) solution to using BatchMake/iPython with other Slicer modules
  • User browses to execution modules and specifies where wildcards should be substituted into command line
• Port lesson's learned to the iPython development effort
  • e.g., leave-one-out looping mechanism, taguchi factor analysis, etc.

Related links

• iPython
• Large scale experiment control brainstorming