Documentation/4.1/Developers/CommandLineModule

For the latest Slicer documentation, visit the read-the-docs.

Command-line interface (CLI) improvements for Slicer4

Summary of CLI improvements discussions

• Date: Sep 27-Oct 11, 2011
• Contributors: Andriy Fedorov, Jean-Christophe Fillion-Robin, Julien Finet, Andras Lasso, Isaiah Norton, Steve Pieper
• See the original thread at: http://massmail.spl.harvard.edu/public-archives/slicer-devel/2011/thread.html#7201

Current CLI is good because

• It's easy to learn (just itk and xml)
• Quite flexible
• Reasonably fast (when MRMLIO is available)

Limitations of current CLI implementation

• Using CLI module as a SharedObject (or Library, so with MRMLIO) it can often take 10-30 seconds for it to "spin up" before the progress bar actually starts. This is the case in Slicer4 and it was also the case in Slicer3, including in Release mode.
• When a module is loaded as SharedObject it makes Slicer instable (when module is started as a separate process none of these issues exist)
  • Module crash cause Slicer crash
  • Memory or handle leak in module causes permanent memory leak in Slicer (if it was the module was in a separate process when the module is completed all the resources are automatically deallocated)
  • Out-of-memory issue in module causes out-of-memory issue in Slicer (typically leading to crash). Most severe on 32-bit builds and/or with large images.
  • Difficult to reproduce problems (inputs and outputs are not captured in fiels; if inputs are written to files and the module is tested with the exe interface then the problem sometimes is not reproducable)
  • Modules are tightly linked to Slicer (advisable to be build the modules with Slicer with the same compiler, options, etc.)
• When a module is started as a separate process then transferring image data through files can be very slow. There are a few use cases where the speed difference between the exe version and the shared library version is very noticable - for example, diffusion tensor estimation where the original DWI files are very large.
• Two interfaces (SharedObject, separate process) exist for the same functionality. It means more workload to maintain add features. It could be beneficial to just maintain & improve one interface and drop the other.
• Calling of one module from another doesn't seem to work with the SharedObject interface (would be useful to have some API for the developers to be able to convert the memory pointers passed to their CLI modules, so that the data items can be processed and passed to the nested CLI invocations)

Potential CLI enhancements/new features

• Load module when the user activates a module, instead of loading it at startup (to reduce loading time)
• Random access to only a small part of the volume data is required (for streamline tractography)
• Hot update when parameters change (ability to re-run a CLI module with slightly different parameters)
• Support for new data types (any specific?)
• Returning (array of) string(s), number(s) for displaying to the user

Tasks

• Short-term: Add run-time option (configurable through GUI) to choose which interface to prefer (separate process/SharedObject) if both are available for a specific module
• Define use cases
• Collect measurement data:
  • Test speed improvements with ITK4
  • Create examples, tests and benchmarks on all supported platforms, using different datasets
• Analyze data, improve current design:
  • Separate the two design questions: We can choose to safely run each module in a separate process (using the exe interface) and pass large data sets efficiently (e.g., by using shared memory).
    1. How to transfer the data between Slicer and the module? (generic file/direct access to memory within a process/shared memory/memory-mapped file/...) - Note: Feedback from some of the paraview developers a few years ago was that they had major headaches supporting shared memory code across platform and suggested avoiding it at all costs. But, that could be due to their support for old OSes on clusters or supercomputers and may no longer be a practical issue.
    2. Where to run the module code? (in separate process / within the Slicer process)

Additional Considerations

Slicer3 supported the use of the CLI XML structure for creating user interfaces for modules implemented in python but that is currently missing from slicer3.

There are other places where it may be convenient for developers of loadable modules to be able to use an interface defined using the execution model XML convention.