Documentation/Labs/SlicerCondaIntegration

Slicer currently builds a custom python distribution from source, but not all useful packages are available. But Anaconda provides a set of binaries that might provide a more comprehensive ecosystem.

Before attempting to use the whole Anaconda package, the first approach has been to use Miniconda, which is a reduced version of Anaconda.

Note: the platform used is MacOSX 10.9.5 with XCode 6

Compile Slicer against miniconda

Mac

cmake -DCMAKE_BUILD_TYPE:STRING=Debug \
	-DQT_QMAKE_EXECUTABLE:FILEPATH=/usr/bin/qmake \
	-DPYTHON_EXECUTABLE:FILEPATH=${HOME}/miniconda/bin/python \
	-DPYTHON_INCLUDE_DIR:PATH=${HOME}/miniconda/include/python2.7 \
	-DPYTHON_LIBRARY:FILEPATH=${HOME}/miniconda/lib/libpython2.7.dylib \
 	-DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.9 \
	-DCMAKE_OSX_SYSROOT:FILEPATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk  \
	-DSlicer_BUILD_CLI:BOOL=OFF \
	-DSlicer_USE_PYTHONQT_WITH_TCL:BOOL=OFF \
	../Slicer

At the moment there are also some additional actions to take, like setting some environment variables:

 export PYTHONHOME="${HOME} /miniconda/"
 
 export PYTHONPATH="{HOME}/miniconda/:{HOME}/miniconda/lib/:{HOME}/miniconda/lib/python2.7/site-packages/:{HOME}/miniconda/lib/python2.7/lib-dynload/
 :{HOME}/miniconda/lib/python2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.5-x86_64.egg/SimpleITK/:{HOME}/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/"
 
 export DYLD_LIBRARY_PATH="{HOME}/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/SimpleITK-build/lib/:{HOME}/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/"

Linux

The following worked for me (Steve Pieper, 2015-02-04) using ubuntu 13.10 and miniconda 3.8.4-py27_0

cmake \
  -DCMAKE_BUILD_TYPE:STRING=Debug \
  -DQT_QMAKE_EXECUTABLE:FILEPATH=/usr/bin/qmake \
  -DPYTHON_EXECUTABLE:FILEPATH=${HOME}/miniconda/bin/python \
  -DPYTHON_INCLUDE_DIR:PATH=${HOME}/miniconda/include/python2.7 \
  -DPYTHON_LIBRARY:FILEPATH=${HOME}/miniconda/lib/libpython2.7.so  \
  -DSlicer_BUILD_CLI:BOOL=OFF \
  -DSlicer_USE_PYTHONQT_WITH_TCL:BOOL=OFF \
  ../Slicer

make -j20

export PYTHONHOME=${USER}/miniconda

./Slicer-build/Slicer

But after it ran there was slicer's build of numpy inside miniconda, so I needed to do this:

bin/conda uninstall numpy
bin/conda install numpy

I tried using this:

 -DSlicer_USE_SYSTEM_NUMPY:BOOL=ON

but got this error:

...
-- SuperBuild -         CTKAPPLAUNCHER[OK] (SYSTEM)
CMake Error at SuperBuild/External_CTKAPPLAUNCHER.cmake:21 (message):
  Enabling Slicer_USE_SYSTEM_CTKAPPLAUNCHER is not supported !
Call Stack (most recent call first):
  CMake/ExternalProjectDependency.cmake:561 (include)
  SuperBuild/External_python.cmake:21 (ExternalProject_Include_Dependencies)
  CMake/ExternalProjectDependency.cmake:561 (include)
  SuperBuild/External_VTKv6.cmake:11 (ExternalProject_Include_Dependencies)
  CMake/ExternalProjectDependency.cmake:561 (include)
  SuperBuild/External_teem.cmake:11 (ExternalProject_Include_Dependencies)
  CMake/ExternalProjectDependency.cmake:561 (include)
  CMake/ExternalProjectDependency.cmake:607 (ExternalProject_Include_Dependencies)
  SuperBuild.cmake:320 (ExternalProject_Include_Dependencies)
  CMakeLists.txt:599 (include)

-- Configuring incomplet

Windows

TBD

Compile Slicer using System Python

In order to compare the behaviors, I also compiled another version of Slicer using my System Python:

 ccmake -DCMAKE_BUILD_TYPE:STRING=Debug -DQT_QMAKE_EXECUTABLE:FILEPATH=/usr/bin/qmake  -DCMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.9  
 -DCMAKE_OSX_SYSROOT:FILEPATH=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk  
 -G  "Eclipse CDT4 - Unix Makefiles" -DSlicer_USE_SYSTEM_python:BOOL=TRUE ../Slicer

Checking my System Python folder, everything seems to be properly installed.

Common issues in Miniconda and System python

There is a difference between executing Slicer-Superbuild/Slicer-build/bin/Slicer.app and Slicer-Superbuild/Slicer-build/Slicer when I open a Slicer Python console. In the former case, Slicer cannot find some libraries. When I open the console I get:

 Python 2.7.5 (default, Mar  9 2014, 22:15:05) 
 [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
 >>> Traceback (most recent call last):
 File "<string>", line 1, in <module>
 File "/Users/Jorge/Projects/BWH/Slicer-Superbuild-SystemPython/Slicer-build/lib/Slicer-4.4/qt-scripted-modules/SimpleFilters.py", line 3, in <module>
   import SimpleITK as sitk
 File "/Library/Python/2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.9-intel.egg/SimpleITK/__init__.py", line 1, in <module>
   from .SimpleITK import *
 File "/Library/Python/2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.9-intel.egg/SimpleITK/SimpleITK.py", line 28, in <module>
   _SimpleITK = swig_import_helper()
 File "/Library/Python/2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.9-intel.egg/SimpleITK/SimpleITK.py", line 24, in swig_import_helper
   _mod = imp.load_module('_SimpleITK', fp, pathname, description)
 ImportError: dlopen(/Library/Python/2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.9-intel.egg/SimpleITK/_SimpleITK.so, 2): Library not loaded: libSimpleITKBasicFilters12-0.9.1.dylib
 Referenced from: /Library/Python/2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.9-intel.egg/SimpleITK/_SimpleITK.so
 Reason: image not found

If I run Slicer-Superbuild/Slicer-build/Slicer, everything seems to work fine.

However, in both cases Slicer is loading my System python!

Next step: isolating the Python console problem

Following the advice of Steve Pieper, I tried to replicate the problem just in the CTK Python console, without running Slicer. To do that, I enabled the setting CTK_APP_ctkSimplePythonShell, inside Slicer-Superbuild/CTK-build/CTK-build. I had exactly the same behavior that I had within Slicer.

I tried to set exactly the same settings in my "Slicer Python" version and in my "Miniconda" version, replacing just the settings PYTHON_EXECUTABLE, PYTHON_INCLUDE_DIR and PYTHON_LIBRARY. Still some behavior...

In this process, I found out two things that are different in the "Slicer Python" and "Miniconda":

If I run otool in my ctkSimplePythonShell, there is a difference in libpython2.7.dylib

~/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CTK-build/bin$ otool -L ctkSimplePythonShell

 ctkSimplePythonShell:
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CTK-build/bin/libCTKScriptingPythonWidgets.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CTK-build/bin/libCTKVisualizationVTKCore.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CTK-build/bin/libCTKWidgets.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/QtTesting-build/libQtTesting.dylib (compatibility version 0.0.0, current version 0.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CTK-build/bin/libCTKScriptingPythonCore.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkInteraction-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkFiltersHybrid-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkOpenGL-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkImagingHybrid-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkIO-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
 /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 20.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkTestingRendering-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkRendering-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkFilters-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkalglib-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkverdict-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkftgl-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkfreetype-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 48.0.0)
 /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 59.0.0)
 /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkImaging-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CMakeExternals/Install/lib/libPythonQt.dylib (compatibility version 0.0.0, current version 0.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkWrappingPython27Core-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtkCommon-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/python-install/lib/libpython2.7.dylib (compatibility version 0.0.0, current version 0.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/VTKv6-build/lib/libvtksys-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Debug/CTK-build/CTK-build/bin/libCTKCore.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 QtOpenGL.framework/Versions/4/QtOpenGL (compatibility version 4.8.0, current version 4.8.6)
 QtXmlPatterns.framework/Versions/4/QtXmlPatterns (compatibility version 4.8.0, current version 4.8.6)
 QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current version 4.8.6)
 QtXml.framework/Versions/4/QtXml (compatibility version 4.8.0, current version 4.8.6)
 QtSql.framework/Versions/4/QtSql (compatibility version 4.8.0, current version 4.8.6)
 QtNetwork.framework/Versions/4/QtNetwork (compatibility version 4.8.0, current version 4.8.6)
 QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current version 4.8.6)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
 /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)

~/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin$ otool -L ctkSimplePythonShell

ctkSimplePythonShell:
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin/libCTKScriptingPythonWidgets.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin/libCTKVisualizationVTKCore.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin/libCTKWidgets.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/QtTesting-build/libQtTesting.dylib (compatibility version 0.0.0, current version 0.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin/libCTKScriptingPythonCore.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkInteraction-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkFiltersHybrid-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkOpenGL-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkImagingHybrid-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkIO-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /System/Library/Frameworks/AGL.framework/Versions/A/AGL (compatibility version 1.0.0, current version 1.0.0)
 /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 20.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkTestingRendering-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkRendering-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkFilters-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkalglib-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkverdict-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkftgl-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkfreetype-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 48.0.0)
 /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 59.0.0)
 /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkImaging-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CMakeExternals/Install/lib/libPythonQt.dylib (compatibility version 0.0.0, current version 0.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkWrappingPython27Core-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtkCommon-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 libpython2.7.dylib (compatibility version 2.7.0, current version 2.7.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/libvtksys-6.2.1.dylib (compatibility version 1.0.0, current version 1.0.0)
 /Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin/libCTKCore.0.1.dylib (compatibility version 0.1.0, current version 0.1.0)
 QtOpenGL.framework/Versions/4/QtOpenGL (compatibility version 4.8.0, current version 4.8.6)
 QtXmlPatterns.framework/Versions/4/QtXmlPatterns (compatibility version 4.8.0, current version 4.8.6)
 QtGui.framework/Versions/4/QtGui (compatibility version 4.8.0, current version 4.8.6)
 QtXml.framework/Versions/4/QtXml (compatibility version 4.8.0, current version 4.8.6)
 QtSql.framework/Versions/4/QtSql (compatibility version 4.8.0, current version 4.8.6)
 QtNetwork.framework/Versions/4/QtNetwork (compatibility version 4.8.0, current version 4.8.6)
 QtCore.framework/Versions/4/QtCore (compatibility version 4.8.0, current version 4.8.6)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
 /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)

So, it looks like there is a problem with the rpath of the library. Even when I tried to copy the python executable and the library itself to the same bin folder where the ctkSimplePython is (Slicer-Superbuild-Conda-Eclipse/CTK-build/CTK-build/bin), I still got the same behavior. Following JC advice (thanks!), I used instal_name_tool to add manually the rpath to libpython2.7.dylib:

 install_name_tool -id /Users/Jorge/miniconda/lib/libpython2.7.dylib /Users/Jorge/miniconda/lib/libpython2.7.dylib

Voila! Now we can rebuild and the ctkSimplePythonShell will use the correct Python! However, we will have to set up some environment variables in order than all the libraries are found. I tried some of the most important and checked that they were properly loaded in the Python console, but we will need the same ones that we are adding when building the current "SlicerPython":

 export PYTHONHOME="/Users/Jorge/miniconda/"
 export PYTHONPATH="/Users/Jorge/miniconda/:/Users/Jorge/miniconda/lib/:/Users/Jorge/miniconda/lib/python2.7/site-packages/:
 /Users/Jorge/miniconda/lib/python2.7/lib-dynload/
 :/Users/Jorge/miniconda/lib/python2.7/site-packages/SimpleITK-0.9.0.dev656-py2.7-macosx-10.5-x86_64.egg/SimpleITK/
 :/Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/"
 export DYLD_LIBRARY_PATH="/Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/SimpleITK-build/lib/
 :/Users/Jorge/Projects/BWH/Slicer-Superbuild-Conda-Eclipse/VTKv6-build/lib/"

I also checked that the Python console works inside Slicer, as long as I launch the Slicer binary from a terminal that has all these environment variables initialized.