Documentation/Labs/CI-and-NightlyPackagesGeneration

Overview

As of April 26th 2016, the build machines known as the Slicer factories have two roles:

• Continuous integration for Slicer
• Nightly packages generation and testing for Slicer application and associated extensions

There are currently two factories (factory.kitware and factory-south.kitware) in charge of both Nightly testing and Nightly generation of packages.

Current scripts to build extensions:

• template: https://github.com/Slicer/Slicer/blob/master/Extensions/CMake/SlicerExtensionsDashboardScript.TEMPLATE.cmake

• windows: https://github.com/Slicer/DashboardScripts/blob/master/factory-south-win7-vs2013-64bits_slicerextensions_release_continuous.cmake

Limitations of the current infrastructure

• Some extensions are consistently not built
• No access to the factory internals
• No well described procedure on how factory machine with packaging can be set up
• Andriy: Not possible to choose the factory in the extension manager.
  • Jc: It is already possible to set the extension server in the settings
• Jorge: Community want to help, but nobody know how to
  • Jc: Having more build bots doing Nightly test would be great. See Documentation/Nightly/Developers/Tutorials/DashboardSetup for more details.

Possible improvements

• Temporary: Is there a way to run a manual build process ?

• Sharing the workload for maintaining and troubleshooting the factory process

• Cloud based solutions (e.g. AWS)
  • Costs ?
  • Preliminary tests ?

AWS cost

Instance types: https://aws.amazon.com/ec2/instance-types/ Instances prices: https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/

G2.2xlarge

G2.2xlarge: This family includes G2 instances intended for graphics and general purpose GPU compute applications. Features include the following:

- These are High Frequency Intel Xeon E5-2670 (Sandy Bridge) Processors - High-performance NVIDIA GPU with 1,536 CUDA cores and 4GB of video memory. - Each G2.2xlarge has 8 CPUs, 15 gigs of memory and 60GB HD - Cost per dedicated instance per year is SXXX for Linux and $XXX for windows

EC2 data transfer

Pricing: http://aws.amazon.com/ec2/pricing/?sc_channel=PS&sc_campaign=acquisition_US&sc_publisher=google&sc_medium=ec2_b&sc_content=sitelink&sc_detail=ec2%20cost&sc_category=ec2&sc_segment=pricing&sc_matchtype=p&sc_country=US&s_kwcid=AL!4422!3!73821520242!p!!g!!ec2%20cost&ef_id=VqGs4wAABNT-5OY4:20160130001731:s

Data Transfer: Internet -> EC2

• Cost $0

Data Transfer: EC2 -> Internet

• Slicer package size (GB): 0.2
• Extension package size (GB): 0.02
• Extension count: 68

• Data (GB) to download from EC2 per month per Operating system: ~50GB (0.2 + 0.02 * 68 * 31)

• Cost / year / instance: ~$110