Documentation/Nightly/Developers/Tutorials/DashboardSetup
For the latest Slicer documentation, visit the read-the-docs. |
Contents
- 1 Prerequisites
- 2 General Instruction
- 3 Factory machines setup
- 4 Remarks
Prerequisites
- Install recommended compiler for your system
- Install/build prerequisites as described here:
- CMake (use the latest CMake stable release)
- git
- svn
- Qt
- NSIS (only on Windows)
General Instruction
Prerequisites
- Disable detachedHead warnings:
git config --global advice.detachedHead false
- On Linux, to use SlicerBuildEnvironment docker images, install docker.
Create dashboard client build directories
- On Linux, create the following directories:
/home/username/Dashboards /home/username/Dashboards/Logs /home/username/Dashboards/DashboardScripts
- On macOS, create a directory like
/Users/username/Dashboards /Users/username/Dashboards/Logs /Users/username/Dashboards/DashboardScripts
- On windows, create a directory like
C:/D C:/D/Logs C:/D/DashboardScripts
Notes:
- username should be changed to match your system.
- On Windows, a short directory name is required to overcome path length limitation.
- The Dashboards directory will be used to set the DASHBOARDS_DIR variable in the client configuration script.
Customize template dashboard client script
Download template scripts
Download the template script: http://svn.slicer.org/Slicer4/trunk/CMake/SlicerDashboardScript.TEMPLATE.cmake
On Linux and macOS:
wget http://svn.slicer.org/Slicer4/trunk/CMake/SlicerDashboardScript.TEMPLATE.cmake mv SlicerDashboardScript.TEMPLATE.cmake <MachineName>_<OperatingSystem>-slicer_preview_nightly.cmake
On windows, from a powershell terminal:
wget http://svn.slicer.org/Slicer4/trunk/CMake/SlicerDashboardScript.TEMPLATE.cmake -OutFile <MachineName>_<OperatingSystem>-slicer_preview_nightly.cmake
Customize script
Edit the file <MachineName>_<OperatingSystem>-slicer_preview_nightly.cmake
to match your machine settings.
- Example of windows script: overload-vs2015-slicer_preview_nightly.cmake
- Example of macOS script: factory-south-macos-slicer_preview_nightly.cmake
- Example of linux script: metroplex-slicer_preview_nightly.cmake
The following directories will be created by the dashboard script:
# Source directory : <DASHBOARDS_DIR>/<Slicer_DASHBOARD_SUBDIR>/<Slicer_DIRECTORY_BASENAME>-<Slicer_DIRECTORY_IDENTIFIER> # Build directory : <DASHBOARDS_DIR>/<Slicer_DASHBOARD_SUBDIR>/<Slicer_DIRECTORY_BASENAME>-<Slicer_DIRECTORY_IDENTIFIER>-build
To easily identify build options associated with source and build directories, the associated build options will be combined together to create an empty text file with the following name:
<DASHBOARDS_DIR>/<Slicer_DASHBOARD_SUBDIR>/<Slicer_DIRECTORY_BASENAME>-<Slicer_DIRECTORY_IDENTIFIER>-build - ${CTEST_BUILD_NAME}-${CTEST_BUILD_CONFIGURATION}-${SCRIPT_MODE}.txt
Publish and version control the dashboard scripts
To backup your scripts and help the Slicer community setup additional dashboard clients by adapting your scripts, consider publishing your scripts in a GitHub repository.
Assuming your organization is named Awesome, you could create the Github project: Awesome/SlicerDashboardScripts.
For example, the official Slicer dashboard script are published here: https://github.com/Slicer/DashboardScripts
Scheduling on Unix-like
So that your dashboard can automatically start at 11pm daily, you could add an entry similar to the following one in your crontab
. (You could run crontab -e
to edit the current user cron list and crontab -l
to dislay the current list.)
0 23 * * * /path/to/cmake-3.13.4/bin/ctest -S /Users/username/DashboardScripts/<MachineName>_<OperatingSystem>-slicer_preview_nightly.cmake -VV -O /Users/username/Dashboards/Logs/<MachineName>_<OperatingSystem>-slicer_preview_nightly.cmake.log
Scheduling on Linux
Install SlicerBuildEnvironment docker image
Read more details at https://github.com/Slicer/SlicerBuildEnvironment#readme
mdir /home/username/bin env_name=qt5-centos7 script=/home/username/bin/slicer-buildenv-${env_name} docker run --rm slicer/buildenv-${env_name} > $script chmod +x $script
Create a machinename.sh file that performs the build and use SlicerBuildEnvironment docker image
To use SlicerBuildEnvironment docker images, create a wrapper script like the following updating username and machinename to match your system:
FileName: /home/username/Dashboards/DashboardScripts/machinename.sh
export DISPLAY=:0.0 # Changing directory is required by "slicer-buildenv-qt5-centos7-latest" script cd /home/username/Dashboards # Download up-to-date slicer/buildenv-qt5-centos7:latest image /home/username/bin/slicer-buildenv-qt5-centos7-latest update # Slicer dashboard settings docker_args="-e run_ctest_with_disable_clean=${run_ctest_with_disable_clean-FALSE}" docker_args+=" -e run_ctest_with_update=${run_ctest_with_update-TRUE}" docker_args+=" -e run_ctest_with_test=${run_ctest_with_test-FALSE}" # XXX Re-enable testing after slicer/slicer-test images have been updated # Slicer 'Preview' release time /home/username/bin/slicer-buildenv-qt5-centos7-latest \ --args "${docker_args}" \ ctest -S /work/DashboardScripts/machinename-slicer_preview_nightly.cmake -VV -O /work/Logs/machinename-slicer_preview_nightly.log
Scheduling on macOS
macOS is deprecating the use of cron in favor of launchd.
Create a file $HOME/Library/LaunchAgents/org.slicer.NightlyDashboard.plist
This is Jim's plist file which has extra EnvironmentVariables to get through his firewall.
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>Label</key> <string>org.slicer.NightlyDashboard</string> <key>ProgramArguments</key> <array> <string>/usr/local/bin/ctest</string> <string>-S</string> <string>/Users/millerjv/Projects/Slicer4-Dashboard/SlicerDashboardScript.cmake</string> </array> <key>EnvironmentVariables</key> <dict> <key>ALL_PROXY</key> <string>proxy.research.ge.com:8080</string> <key>http_proxy</key> <string>proxy.research.ge.com:8080</string> <key>RSYNC_PROXY</key> <string>proxy.research.ge.com:8080</string> </dict> <key>Nice</key> <integer>1</integer> <key>StartCalendarInterval</key> <dict> <key>Hour</key> <integer>1</integer> <key>Minute</key> <integer>03</integer> </dict> <key>StandardErrorPath</key> <string>/tmp/org.slicer.NightlyDashboard.err</string> <key>StandardOutPath</key> <string>/tmp/org.slicer.NightlyDashboard.out</string> </dict> </plist>
You can manually load this agent using
$ launchctl load $HOME/Library/LaunchAgents/org.slicer.NightlyDashboard.plist
and the agent will be reloaded automatically on a reboot.
It seems I have to be logged onto the console for the launch agent to get installed and run. What I do is log on, then select "Login Window..." so that other people can log onto the console. This is suboptimal as I need to do this after each reboot of the system.
Scheduling on Windows
Create a .bat file that performs the build
For example, a clean nightly build can be performed by a batch file like this:
rmdir /S /Q "C:\D\P\Slicer-0-build" "c:\Program Files (x86)\CMake\bin\ctest.exe" -S "C:\D\DashboardScripts\overload-vs2015-slicer_preview_nightly.cmake" -C Release -VV -O C:\D\Logs\overload-vs2015-slicer_preview_nightly.txt
Create a new scheduled task that runs the .bat file
- Windows XP: Create a new scheduled task at Control Panel / Scheduled Tasks / Add Scheduled
- Windows 7: Task scheduler can be found at Start Menu / All Programs / Accessories / System Tools / Task Scheduler.
Setting up a continuous dashboard
Similarly if you want to setup a continuous dashboard,
$ cp <MachineName>_<OperatingSystem>-slicer_preview_nightly.cmake <MachineName>_<OperatingSystem>-slicer_preview_continuous.cmake
Edit <MachineName>_<OperatingSystem>-slicer_preview_continuous.cmake
and set
set(SCRIPT_MODE "*continuous*")
Then add the corresponding entry in crontab
so that it starts everyday at 6am:
0 6 * * * /path/to/cmake-3.13.4/bin/ctest -S /Users/JOE/DashboardScripts/<MachineName>_<OperatingSystem>-slicer_preview_continuous.cmake -VV -O /Users/JOE/Dashboards/Logs/<MachineName>_<OperatingSystem>-slicer_preview_nightly.log
The output of the build is contained in ctest_package_make_package_output.txt
in your slicer build directory, you can check this file if something went wrong.
Factory machines setup
Tool used for task scheduling is described here: Documentation/Nightly/Developers/Factory
Dashboard scripts used on the factory are available here: https://github.com/Slicer/DashboardScripts
More details available here:
Remarks
Setting up a dashboard on a Windows machine
Disabling TortoiseSVN Icon Overlay
If you're using an interface for version control such as TortoiseSVN, make sure in the Icon Overlay settings that the status cache is set to None.
Keep short path
Keep the paths to your Nightly, Continuous, and Experimental directories as terse as possible. A good standard is C:\D
for dashboards, and C:\D\<N,C,E> subdirectories for Nightly, Continuous, and Experimental. Otherwise, CMake will choke on the paths.
How can I prevent the “… has stopped working” window
As documented here, set registry key HKEY_LOCAL_MACHINE\Software\Microsoft\.NETFramework\DbgJITDebugLaunchSetting
value to 1.
Value 1 simply return control. This results in a stack dump, after which the process is terminated.
This has been documented while trying to prevent the dialog "Microsoft Visual Studio 8 has stopped working" from appearing when Slicer extensions dashboard were running. The details associated with the dialog were:
Problem signature: Problem Event Name: BEX Application Name: devenv.exe Application Version: 9.0.21022.8 Application Timestamp: 47317b3d Fault Module Name: VCProjectEngine.dll_unloaded Fault Module Version: 0.0.0.0 Fault Module Timestamp: 47318038 Exception Offset: 72dbdce0 Exception Code: c0000005 Exception Data: 00000008
Reference:
- http://stackoverflow.com/questions/2507464/how-can-i-prevent-the-has-stopped-working-window-on-my-server-2008-r2-dev
- http://msdn.microsoft.com/en-us/library/2ac5yxx6(VS.80).aspx
Lockhunter: Unlock directories or folders
Often on Windows, once a dashboard build is started, it is impossible to delete anything from the source or build directory. Windows will report that this file is currently in use by another program, or "locking" it. For this, we recommend installing Lockhunter.