Documentation/Labs/AutomaticUpdateAndInstallationFramework

Overview

This page serves as roadmap for the Slicer Automatic Update and Installation framework.

Use cases:

• New Slicer version installed locally should be updated with the list of extensions previously installed.
• Self update of Slicer

Terminology

• User Interface
  • Extensions Manager is a Slicer dialog allowing the user to Search, Install, Remove and disable extensions.
  • ExtensionWizard module: This is a Slicer module providing a graphical interface within Slicer to aid in the creation of Slicer extensions.

• Command line
  • ExtensionWizard is a command line tool to simplify the process of creating and contributing extensions.
    • See also Contribute Extension Description File using Extension Wizard

• API
  • ExtensionsManagerModel: This is the backend of the Extensions Manager displayed to the user
  • Utilities/Scripts/SlicerWizard: This is a python package used in the ExtensionWizard

Convention

Tasks listed in this page can be associated with any of these state:

• ToBeDone
• Completed
• In progress

Use case: Keep track of the installed extensions across version

The goal here is to simplify the number of steps allowing people to reinstall extensions that were used in a previous installation.

• Update: Mathias is currently working on this task (Previous Efforts by Dennis)
• Meta issue: 2779: Mechanism to keep track of the installed extensions across version

Idea

• After installing a new version of Slicer, the user can select from all previously installed extensions and batch-install them
• extensions installed in the last (previous) Slicer versions are marked and preselected (one click-solution for getting the previous setup)
• extensions that where previously installed but are not compatible for some reason, are shown but not selectable
• UI: a third tab ("Restore Extensions") should be added to the Extension Manager and provide at least a list view for selection / a button button for installing the selected extensions

Gathering information about previous extensions

• extensions information / subfolders containing the extensions are available in the "NA-MIC" folder
  • on Linux, that's in ~/.config/NA-MIC
  • on Windows, in AppData/Roaming
  • on OS X, seems to be a subdirectory of the application bundle
• folder contains and Slicer-[Revision].ini file that points to paths relevant for extensions (modules/libraries/dependencies)
• however, no explicit information about installed extensions are stored there, information about installed extension is gathered at runtime by parsing the extension subfolders
• PROPOSAL: added the extension IDs of all currently installed extensions in an additional setting information [Extensions]/InstalledExtension, thus, no folders need to be parsed and only the .ini files are needed

Changes

• Base/QTGUI/Resources/UI/qSlicerExtensionsManagerWidget.ui
  • a new tab ("Restore Extensions") was added
  • this tab contains a placeholder-widget for the qSlicerExtensionsRestoreWidget
• qSlicerExtensionsManagerModel
  • add a function that provides informations about previous extensions: getExtensionRestoreInformation()
  • add a callback for tracking the progress during non-update related updates: onInstallDownloadProgress (used in: downloadAndInstallExtension)
  • extend installExtension / scheduleExtensionForUninstall to add (remove) ids of installed extensions to a new parameter [Extensions]/InstalledExtension

Addition

• qSlicerExtensionsRestoreWidget
  • a new widget that renders a list of available extension that were previously installed
  • on initialization it calls getExtensionRestoreInformation() and builds a ListItemWidget
  • if "Install selected" is triggered, downloadAndInstallExtension is called sequentially for each selected extension
  • after installation of all extensions, the list is updated

Screenshot of the "restore extension" widget

Progress

• Adapt extensions manager model to provide neccessary information Completed
• Extend the UI of the extensions manager (new tab, new widget) Completed
• Add a widget that provides a selectable list of previously installed extensions Completed
• Install procedure (with feedback) for selected extensions Completed
• Discuss the proposed solution with JC Completed
• Adapt solution accordingly Completed
• Submit solution Completed

Patch

• First (not final) version: File:0001-ENH-working-version-of-extension-restore-functionali.zip

Questions

• Does the uninstall progress also remove the extension-subfolders? (yes)
• Is the proposed solution (accessing the .ini file) suitable for all OS? (yes)
• How could settings of previous Slicer installations be accessed under MacOS? (same way as for other platforms)

Feedback from JC on first version (12/08/2016)

• Generally agrees to current solution
• Solution should be compatible to all platforms (there is a central config/extension folder also on MacOS)
• Suggested Changes:
  • add a legend to explain color coding (green, gray, black, marked - unmarked) of the presented extensions
  • check for extension compatibility currently deactivated, because model functionality used local information to check compatibility
    • JC added patch to add server-side information to the compatibility check
    • commit d448b1c9634c71a61b3476a98bfccffba6bcedb5
  • would be better to track previously installed extensions in the central, not revision-dependent slicer.ini
    • thus, no (possibly error-prone) parsing of the file names is necessary
    • also, we would no long have to rely on a central saving location for all extension related information (currently the NA-MIC folder = Extension install path)
    • Structure could be:
      • [installedExtensionsHistory] → [Revision] → [ExtensionID]
    • To enable access to slicer.ini the SlicerCoreApp needs to be extened (e.g. getHistoryFilePath, see. line 319, qSlicerCoreApp.cxx)
• Additional Information:
  • Q: Why are extensions not removed after uninstalling Slicer?
  • A: related to NSIS installer setup, could be changed if installation setup is changed
  • Q: How could an semi-/automatic update be realized?
  • A:
    • Change of the underlying QT framework (would mean fundamental changes to the current setup)
    • or: use an approach similar to the installation of extensions, by having a central store for new slicer version
    • no auto-uninstall but auto-install / maybe just a popup on start, with a link to the new version
    • since extension information is saved in central slicer.ini the installation of extensions could be triggered on the first start of the new version
• General vision (this addition to the extension manager is the first step):
  • core Slicer application + profiles (e.g. using the same set of extensions / setup / layout that was used for a paper) :
  • pre-packed Slicer versions available for downloads
  • can be extended with downloadable profiles

Adaptations (March) according to input from JC (December) and Ron (January)

After the disclaimer is shown, the slicer.ini is parsed for previously installed extensions, if candidates are found (and the configuration is set to check on startup) a pop-up is shown for instantaneous installation:

Dialogs and progress dialogs are provided by the headless mode of the extension restore widget:

A restart is requested after installation. If the configuration is set to auto-restart and silent extension check on startup, you get an automatic installation of previous extensions after installing a new Slicer build as requested by Ron:

An improved widget interface gives detailed information about the state of previously installed extensions, including the auto selection of installation candidates. It also allows to de/activate check on startup and silent installation:

The extension history is now centrally tracked in the slicer settings file ( model->setExtensionsHistorySettingsFilePath(q->slicerUserSettingsFilePath());):

//SLICER.INI
...
[ExtensionsHistory]
Revisions\25844=ChangeTracker, SlicerToKiwiExporter, CleaverExtension
Revisions\25843=AnglePlanesExtension, ABC, CurveMaker, SlicerToKiwiExporter
Revisions\25842=AnglePlanesExtension, CurveMaker, CMFreg
Revisions\25895=SlicerToKiwiExporter
ScheduledForRemoval=
...

A set of new functions in the extension manager model help to reflect the current extension installation state (important for future installation):

//saves the currently installed extension to the [ExtensionsHistory] Revisions\XXXXX=ExtensionName setting
void saveExtensionToHistorySettings(const QString& extensionsHistorySettingsFile, const ExtensionMetadataType &extensionMetadata);
 
//track info that an extension should be removed (will be removed from history with respect to current slicer version - [ExtensionsHistory] ScheduledForRemoval=ExtensionName)
void scheduleExtensionHistorySettingRemoval(const QString& extensionsHistorySettingsFile, const ExtensionMetadataType &extensionMetadata);
 
//helper function to add a history setting to a specific setting task (scheduled for removal or adding)
void addExtensionHistorySetting(const QString& extensionsHistorySettingsFile, const ExtensionMetadataType &extensionMetadata, const QString& settingsPath);
 
//removes scheduled removal
void cancelExtensionHistorySettingRemoval(const QString& extensionsHistorySettingsFile, const QString& extensionName);
 
//remove the extensions from the history settings with respect to the current slicer revision
void removeScheduledExtensionHistorySettings(const QString& extensionsHistorySettingsFile);
 
//core function to parse the settings and create a list with extensionName keys --> extension_id, is_installed, is_compatible, last_revision, was_previously_installed
QVariantMap getExtensionsInfoFromPreviousInstallations(const QString& extensionsHistorySettingsFile);
 
//gathers information and emits extensionHistoryGatheredOnStartup
void gatherExtensionsHistoryInformationOnStartup();

//PUBLIC:
void gatherExtensionsHistoryInformationOnStartup();
QVariantMap getExtensionHistoryInformation();

the extension history information map (QVariantMap getExtensionHistoryInformation()) that can look like this (shortened - based on the settings shown above):

QMap(
("ABC",
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155791") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, false) )
( "UsedLastInRevision" ,  QVariant(QString, "25843") )
( "WasInstalledInLastRevision" ,  QVariant(bool, false) ) )  ) )
( "AnglePlanesExtension" , 
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155785") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, false) )
( "UsedLastInRevision" ,  QVariant(QString, "25843") )
( "WasInstalledInLastRevision" ,  QVariant(bool, false) ) )  ) )
( "CMFreg" , 
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155788") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, false) )
( "UsedLastInRevision" ,  QVariant(QString, "25842") )
( "WasInstalledInLastRevision" ,  QVariant(bool, false) ) )  ) )
( "ChangeTracker" , 
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155789") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, true) )
( "UsedLastInRevision" ,  QVariant(QString, "25895") )
( "WasInstalledInLastRevision" ,  QVariant(bool, true) ) )  ) )         //would be a preselected install candidate if it wasn't already installed
...

The compatibility is now checked correctly for non installed extensions and the current extension id is retrieved:

//SlicerExtensionsManagerModelPrivate::getExtensionsInfoFromPreviousInstallations

...
parameters["productname"] = extensionName; 
parameters["slicer_revision"] = q->slicerRevision();
parameters["os"] = q->slicerOs();
parameters["arch"] = q->slicerArch();
const ExtensionMetadataType& metaData = retrieveExtensionMetadata(parameters);
extensionId = metaData.value("extension_id").toString();     //retrieve updated extension id for not installed extensions
...
isCompatible = (this->isExtensionCompatible(metaData,
this->SlicerRevision, this->SlicerOs, this->SlicerArch).length() == 0);

Completed

NA

Use case: Self update of Slicer

Proposed Changes

• Display list of nightly builds in a web view
• Selecting a new version should automate:
  • download
  • auto-start installer, with extra information for the next steps
  • auto-start new Slicer
  • possibly, directly going to the above extensions-reinstallation tab (maybe not even showing the main window before)
• stuff that is there, or might be useful
  • cross-platform re-start functionality is there (as part of extension installation)
  • probably, at least on Win/Linux, "start Slicer after installation" is possible
  • unattended installation on any platform(s)?

Notes

2016.05.03

• See also Documentation/Labs/ExtensionsFrameworkRoadmap

• Ability to ask the installer (or Slicer) to install other extensions
  • Would it make sense to transition to http://doc.qt.io/qtinstallerframework/ ?

• Serialize what is installed in you current Slicer

• Update of installed extensions:
  • See https://www.slicer.org/slicerWiki/index.php/Documentation/4.5/SlicerApplication/ExtensionsManager#Updating_installed_extensions