Documentation/Labs/CMake-ified Python

Following r21863, CMake'ified python has been integrated into Slicer.

COMP: Update build-system to use CMake'ified python

Latest stable version 2.7.3 is now used instead of ancient 2.6.6

Directory layout has been updated:
 - Python-2.7.3   : source
 - python         : build system
 - python-build   : build tree
 - python-install : install tree

Associated external project has been simplified. Removed unneeded
custom configure, make and install steps. Also removed unneeded
patch steps.

Note that the pythonrun.h and pythonrun.c are still patched in order
to prevent crash (related to dll boundaries [1][2]) when building Slicer debug
against a released python.

[1] http://stackoverflow.com/questions/2322095/why-does-this-program-crash-passing-of-stdstring-between-dlls
[2] http://social.msdn.microsoft.com/Forums/en-US/vcgeneral/thread/57e0d522-a42c-4add-963d-c87c6e76f161

Since only library located in the "lib-dynload" folder are not duplicated
in the "site-packages" folder, removed the exclude pattern
from the fixup script and update the candidate pattern to consider the
python extension located in lib/python2.7/lib-dynload folder.

Source: http://viewvc.slicer.org/viewvc.cgi/Slicer4?view=revision&revision=21863

Python 2.7.3

# Download python source and CMake build system
## Windows/Unix: Open your unix terminal or windows git bash console, then run:

git clone git://github.com/jcfr/python-cmake-buildsystem -b next
curl -O http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar -xzvf Python-2.7.3.tgz
mkdir Python-2.7.3-build
cd Python-2.7.3-build


# Configure the project
## Windows:

Open cmake-gui
 Source dir : /path/to/python-cmake-buildsystem
 Build dir  : /path/to/Python-2.7.3-build
 Configure & Generate
Open /path/to/Python-2.7.3-build/Python27.sln
 Build all
Run /path/to/Python-2.7.3-build/bin/python.exe


## Unix (MacOSX, Linux)

cd Python-2.7.3-build
cmake ../python-cmake-buildsystem
make -j8
./bin/python



Available build options:

 BUILD_SHARED:   Build python static library
 BUILD_STATIC:     Build python shared library

 ENABLE_<EXTENSION>: If ON, the corresponding python extension will be built.
 BUILTIN_<EXTENSION>:  If ON, the corresponding python extension will be built-in the built python librarie(s). No need to use Setup.local


To report problem, create an issue: https://github.com/jcfr/python-cmake-buildsystem/issues

Build results

• 02/04/2013 - 2.7.3/VS2008/Build - Steve Pieper

Motivation

Within Slicer, we have been struggling to compile python on VS2008. Moving forward, we would like to also compile it on VS2010 and VS2012 ... 

Currently, we are building python 2.6.6. We would also like to move to a more recent version of python (2.7.x). 

It gonna very probably be a long and a painful process. [[ I am not even talking about building tkinter module allowing to bridge tcl and python ... ]]

I just found out that there is some work done regarding the CMake'ification of Python:

[1] https://github.com/davidsansome/python-cmake-buildsystem
[2]  http://www.cmake.org/Wiki/BuildingPythonWithCMake

Ideally, it would be great to contribute back the CMakeLists files. In the mean time, we could either help maintain the project [1]  or fork the python source code mirror available on github [3] and contribute to it.

[3] https://github.com/jonashaag/cpython

I did some experiment and the build system of "David Sansome" looks very promising. It allows a lot of flexibility in the way python can be built and embedded in existing application.

From the author:

This cmake-based buildsystem is better because:

  * It's much faster to compile: 7 seconds instead of 58 seconds in my
    unscientific test.
  * No compiled program for the target architecture is used in the build
    itself.  This makes cross-compiling possible.
  * Same project files for all platforms - there's no need to maintain the
    unix build separately from four different MSVC builds.


It also provided patches making cross-compilation very easy. (for example compilation for windows from Linux. See end of readme)

I think we should contribute and support that well documented build system :) 

In a nutshell, in addition to the option: 

 ENABLE_SHARED  "Build a shared libpython library"
 ENABLE_STATIC     "Build a static libpython library"


it also provides options like 

 BUILTIN_ARRAY, BUILTIN_AUDIOOP, ....,  BUILTIN_UNICODEDATA ,  BUILTIN_ZLIB
 allowing to selectively include python modules in either the static or shared python lib.

 ENABLE_ARRAY, ENABLE_AUDIOOP, ....,  ENABLE_UNICODEDATA ,  ENABLE_ZLIB
 allowing to specify with a given module should be built or not.


Let's also notice options like:

 EXTRA_PYTHONPATH  "A colon (:) separated list of extra paths to add to the PYTHONPATH"
 USE_LIB64   "Search for dependencies and install to prefix/lib64 instead of prefix/lib"
 USE_LIBEDIT  "Use libedit instead of readline"


Under the hood, it implements to CMake modules, that would probably make port of project like "numpy" easier ... 
  
  Assembler.cmake:  
  
    # CMake ignores .S files, so we have to add compile commands for them manually
    macro(add_assembler ADD_ASSEMBLER_OUTPUTVAR ADD_ASSEMBLER_FILE)

  
  Extensions.cmake
  
  # This function adds a python extension to the buildsystem.
#
# Usage:
#
# add_python_extension(
#     extension_name
#     SOURCES source1.c source2.c ...
#     [ REQUIRES variable1 variable2 ... ]
#     [ DEFINITIONS define1 define2 ... ]
#     [ LIBRARIES lib1 lib2 ... ]
#     [ INCLUDEDIRS dir1 dir2 ... ]
#     [ BUILTIN ]
# )