Difference between revisions of "Documentation/Labs/CI-and-NightlyPackagesGeneration"

From Slicer Wiki
Jump to: navigation, search
 
(2 intermediate revisions by the same user not shown)
Line 6: Line 6:
  
 
There are currently [[Documentation/Nightly/Developers/Factory|two factories]] (factory.kitware and factory-south.kitware) in charge of both Nightly testing and Nightly generation of packages.
 
There are currently [[Documentation/Nightly/Developers/Factory|two factories]] (factory.kitware and factory-south.kitware) in charge of both Nightly testing and Nightly generation of packages.
 +
 +
== ==
 +
 +
Current scripts to build extensions:
 +
 +
* template: https://github.com/Slicer/Slicer/blob/master/Extensions/CMake/SlicerExtensionsDashboardScript.TEMPLATE.cmake
 +
 +
* windows: https://github.com/Slicer/DashboardScripts/blob/master/factory-south-win7-vs2013-64bits_slicerextensions_release_continuous.cmake
 +
  
 
== Limitations of the current infrastructure ==
 
== Limitations of the current infrastructure ==
Line 33: Line 42:
 
Instances prices: https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/
 
Instances prices: https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/
  
==== G2.2xlarge ====
+
G2.2xlarge (linux): $3825 / yr
 
+
G2.2xlarge (windows): $4934 / yr
G2.2xlarge: This family includes G2 instances intended for graphics
 
and general purpose GPU compute applications. Features include the
 
following:
 
 
 
- These are High Frequency Intel Xeon E5-2670 (Sandy Bridge) Processors
 
- High-performance NVIDIA GPU with 1,536 CUDA cores and 4GB of
 
video memory.
 
- Each G2.2xlarge has 8 CPUs, 15 gigs of memory and 60GB HD
 
- Cost per dedicated instance per year is SXXX for Linux and $XXX for windows
 
 
 
==== EC2 data transfer ====
 
 
 
Pricing: http://aws.amazon.com/ec2/pricing/?sc_channel=PS&sc_campaign=acquisition_US&sc_publisher=google&sc_medium=ec2_b&sc_content=sitelink&sc_detail=ec2%20cost&sc_category=ec2&sc_segment=pricing&sc_matchtype=p&sc_country=US&s_kwcid=AL!4422!3!73821520242!p!!g!!ec2%20cost&ef_id=VqGs4wAABNT-5OY4:20160130001731:s
 
 
 
Data Transfer: Internet -> EC2
 
* Cost $0
 
 
 
Data Transfer: EC2 -> Internet
 
 
 
* Slicer package size (GB): 0.2
 
* Extension package size (GB): 0.02
 
* Extension count: 68
 
  
* Data (GB) to download from EC2 per month per Operating system: ~50GB    (0.2 + 0.02 * 68 * 31)
+
Data Transfer
 +
* Internet -> EC2: $0
 +
* EC2 -> Internet: $55 / yr / instance
  
* Cost / year / instance: ~$110
+
Details: https://docs.google.com/spreadsheets/d/1tTndErxk_71ZjbT85Kwo5hkd9Yf10qm7ToaM087vKg4/edit?usp=sharing

Latest revision as of 15:40, 26 April 2016

Home < Documentation < Labs < CI-and-NightlyPackagesGeneration

Overview

As of April 26th 2016, the build machines known as the Slicer factories have two roles:

  • Continuous integration for Slicer
  • Nightly packages generation and testing for Slicer application and associated extensions

There are currently two factories (factory.kitware and factory-south.kitware) in charge of both Nightly testing and Nightly generation of packages.

Current scripts to build extensions:


Limitations of the current infrastructure

  • Some extensions are consistently not built
  • No access to the factory internals
  • No well described procedure on how factory machine with packaging can be set up
  • Andriy: Not possible to choose the factory in the extension manager.
    • Jc: It is already possible to set the extension server in the settings
  • Jorge: Community want to help, but nobody know how to

Possible improvements

  • Temporary: Is there a way to run a manual build process ?
  • Sharing the workload for maintaining and troubleshooting the factory process
  • Cloud based solutions (e.g. AWS)
    • Costs ?
    • Preliminary tests ?


AWS cost

Instance types: https://aws.amazon.com/ec2/instance-types/ Instances prices: https://aws.amazon.com/ec2/purchasing-options/dedicated-instances/

G2.2xlarge (linux): $3825 / yr G2.2xlarge (windows): $4934 / yr

Data Transfer

  • Internet -> EC2: $0
  • EC2 -> Internet: $55 / yr / instance

Details: https://docs.google.com/spreadsheets/d/1tTndErxk_71ZjbT85Kwo5hkd9Yf10qm7ToaM087vKg4/edit?usp=sharing