View source for Documentation/Labs/AutomaticUpdateAndInstallationFramework

= Overview =

This page serves as roadmap for the Slicer Automatic Update and Installation framework.

Use cases:
* New Slicer version installed locally should be updated with the list of extensions previously installed.
* Self update of Slicer

= Terminology =

* User Interface
** [[Documentation/4.5/SlicerApplication/ExtensionsManager|Extensions Manager]] is a Slicer dialog allowing the user to Search, Install, Remove and disable extensions.
** [[Documentation/Nightly/Modules/ExtensionWizard|ExtensionWizard module]]: This is a Slicer module providing a graphical interface within Slicer to aid in the creation of Slicer extensions.


* Command line
** [[Documentation/Nightly/Developers/ExtensionWizard|ExtensionWizard]] is a command line tool to simplify the process of creating and contributing extensions.
*** See also [[Documentation/Nightly/Developers/Tutorials/Contribute_Extension_Description_File_using_Extension_Wizard|Contribute Extension Description File using Extension Wizard]]


* API
** [https://github.com/Slicer/Slicer/blob/master/Base/QTCore/qSlicerExtensionsManagerModel.h ExtensionsManagerModel]: This is the backend of the [[Documentation/4.5/SlicerApplication/ExtensionsManager|Extensions Manager]] displayed to the user
** [https://github.com/Slicer/Slicer/tree/master/Utilities/Scripts/SlicerWizard Utilities/Scripts/SlicerWizard]: This is a python package used in the ExtensionWizard

= Convention =

Tasks listed in this page can be associated with any of these state:
* '''ToBeDone'''
*  {{done}} '''Completed'''
* '''In progress'''

= Use case: Keep track of the installed extensions across version =

The goal here is to simplify the number of steps allowing people to reinstall extensions that were used in a previous installation.

* Update: [[User:Mneugebauer|Mathias]] is currently working on this task ([[{{FULLPAGENAME}}/PreviousEfforts|Previous Efforts by Dennis]])
* Meta issue: [http://www.na-mic.org/Bug/view.php?id=2779 2779]: Mechanism to keep track of the installed extensions across version

== Idea ==
* After installing a new version of Slicer, the user can select from all previously installed extensions and batch-install them
* extensions installed in the last (previous) Slicer versions are marked and preselected (one click-solution for getting the previous setup)
* extensions that where previously installed but are not compatible for some reason, are shown but not selectable
* UI: a third tab ("Restore Extensions") should be added to the Extension Manager and provide at least a list view for selection / a button button for installing the selected extensions

== Gathering information about previous extensions == 
* extensions information  / subfolders containing the extensions are available in the "NA-MIC" folder
** on Linux, that's in ~/.config/NA-MIC
** on Windows, in AppData/Roaming
** on OS X, seems to be a subdirectory of the application bundle
* folder contains and Slicer-[Revision].ini file that points to paths relevant for extensions (modules/libraries/dependencies)
* however, no explicit information about installed extensions are stored there, information about installed extension is gathered at runtime by parsing the extension subfolders
* '''PROPOSAL:''' added the extension IDs of all currently installed extensions in an additional setting information <code>[Extensions]/InstalledExtension</code>, thus, no folders need to be parsed and only the .ini files are needed

== Changes ==
* <code>Base/QTGUI/Resources/UI/qSlicerExtensionsManagerWidget.ui</code> 
** a new tab ("Restore Extensions") was added
** this tab contains a placeholder-widget for the <code>qSlicerExtensionsRestoreWidget</code>
* <code>qSlicerExtensionsManagerModel</code>
** add a function that provides informations about previous extensions: <code>getExtensionRestoreInformation()</code>
** add a callback for tracking the progress during non-update related updates: <code>onInstallDownloadProgress</code> (used in: <code>downloadAndInstallExtension</code>)
** extend <code>installExtension</code> / <code>scheduleExtensionForUninstall</code> to add (remove) ids of installed extensions to a new parameter <code>[Extensions]/InstalledExtension</code>

== Addition ==
* <code>qSlicerExtensionsRestoreWidget</code>
** a new widget that renders a list of available extension that were previously installed
** on initialization it calls <code>getExtensionRestoreInformation()</code> and builds a ListItemWidget
** if "Install selected" is triggered, <code>downloadAndInstallExtension</code> is called sequentially for each selected extension
** after installation of all extensions, the list is updated
[[File:Image2016-11-25 15-26-52.png|thumb|Screenshot of the "restore extension" widget]]

== Progress ==
* Adapt extensions manager model to provide neccessary information {{done}} '''Completed'''
* Extend the UI of the extensions manager (new tab, new widget) {{done}} '''Completed'''
* Add a widget that provides a selectable list of previously installed extensions {{done}} '''Completed'''
* Install procedure (with feedback) for selected extensions {{done}} '''Completed'''
* Discuss the proposed solution with JC {{done}} '''Completed'''
* Adapt solution accordingly {{done}} '''Completed'''
* Submit solution {{done}} '''Completed'''

== Patch ==
* First (not final) version: [[File:0001-ENH-working-version-of-extension-restore-functionali.zip|thumb|First patch]]

== Questions ==
* Does the uninstall progress also remove the extension-subfolders? (yes)
* Is the proposed solution (accessing the .ini file) suitable for all OS? (yes)
* How could settings of previous Slicer installations be accessed under MacOS? (same way as for other platforms)

== Feedback from JC on first version (12/08/2016) ==

* Generally agrees to current solution
* Solution should be compatible to all platforms (there is a central config/extension folder also on MacOS)
* Suggested Changes:
** add a legend to explain color coding (green, gray, black, marked - unmarked) of the presented extensions
** check for extension compatibility currently deactivated, because model functionality used local information to check compatibility
*** JC added patch to add server-side information to the compatibility check
*** commit d448b1c9634c71a61b3476a98bfccffba6bcedb5
** would be better to track previously installed extensions in the central, not revision-dependent slicer.ini
*** thus, no (possibly error-prone) parsing of the file names is necessary
*** also, we would no long have to rely on a central saving location for all extension related information (currently the NA-MIC folder = Extension install path)
*** Structure could be:
**** [installedExtensionsHistory] → [Revision] → [ExtensionID]
*** To enable access to slicer.ini the SlicerCoreApp needs to be extened (e.g. getHistoryFilePath, see. line 319, qSlicerCoreApp.cxx)
* Additional Information:
** Q: Why are extensions not removed after uninstalling Slicer?
** A: related to NSIS installer setup, could be changed if installation setup is changed
** Q: How could an semi-/automatic update be realized?
** A: 
*** Change of the underlying QT framework (would mean fundamental changes to the current setup)
*** or: use an approach similar to the installation of extensions, by having a central store for new slicer version
*** no auto-uninstall but auto-install / maybe just a popup on start, with a link to the new version
*** since extension information is saved in central slicer.ini the installation of extensions could be triggered on the first start of the new version 
* General vision (this addition to the extension manager is the first step):
** core Slicer application + profiles (e.g. using the same set of extensions / setup / layout that was used for a paper) :
** pre-packed Slicer versions available for downloads
** can be extended with downloadable profiles

== Adaptations (March) according to input from JC (December) and Ron (January) ==
After the disclaimer is shown, the slicer.ini is parsed for previously installed extensions, if candidates are found (and the configuration is set to check on startup) a pop-up is shown for instantaneous installation:

[[File:Extension check on startup.png|500px]]

Dialogs and progress dialogs are provided by the headless mode of the extension restore widget:

[[File:Extension installation progress startup.png|500px]]

A restart is requested after installation. If the configuration is set to auto-restart and silent extension check on startup, you get an automatic installation of previous extensions after installing a new Slicer build as requested by Ron:

[[File:Restart confirm after extension installation on startup.png|500px]]

An improved widget interface gives detailed information about the state of previously installed extensions, including the auto selection of installation candidates. It also allows to de/activate check on startup and silent installation:

[[File:Extension restore widget interface.png|500px]]

The extension history is now centrally tracked in the slicer settings file (<code> model->setExtensionsHistorySettingsFilePath(q->slicerUserSettingsFilePath());</code>):

<source>
//SLICER.INI
...
[ExtensionsHistory]
Revisions\25844=ChangeTracker, SlicerToKiwiExporter, CleaverExtension
Revisions\25843=AnglePlanesExtension, ABC, CurveMaker, SlicerToKiwiExporter
Revisions\25842=AnglePlanesExtension, CurveMaker, CMFreg
Revisions\25895=SlicerToKiwiExporter
ScheduledForRemoval=
...
</source>



A set of new functions in the extension manager model help to reflect the current extension installation state (important for future installation):

<source lang='c++'>
//saves the currently installed extension to the [ExtensionsHistory] Revisions\XXXXX=ExtensionName setting
void saveExtensionToHistorySettings(const QString& extensionsHistorySettingsFile, const ExtensionMetadataType &extensionMetadata);
 
//track info that an extension should be removed (will be removed from history with respect to current slicer version - [ExtensionsHistory] ScheduledForRemoval=ExtensionName)
void scheduleExtensionHistorySettingRemoval(const QString& extensionsHistorySettingsFile, const ExtensionMetadataType &extensionMetadata);
 
//helper function to add a history setting to a specific setting task (scheduled for removal or adding)
void addExtensionHistorySetting(const QString& extensionsHistorySettingsFile, const ExtensionMetadataType &extensionMetadata, const QString& settingsPath);
 
//removes scheduled removal
void cancelExtensionHistorySettingRemoval(const QString& extensionsHistorySettingsFile, const QString& extensionName);
 
//remove the extensions from the history settings with respect to the current slicer revision
void removeScheduledExtensionHistorySettings(const QString& extensionsHistorySettingsFile);
 
//core function to parse the settings and create a list with extensionName keys --> extension_id, is_installed, is_compatible, last_revision, was_previously_installed
QVariantMap getExtensionsInfoFromPreviousInstallations(const QString& extensionsHistorySettingsFile);
 
//gathers information and emits extensionHistoryGatheredOnStartup
void gatherExtensionsHistoryInformationOnStartup();

//PUBLIC:
void gatherExtensionsHistoryInformationOnStartup();
QVariantMap getExtensionHistoryInformation();

</source>

the extension history information map (<code>QVariantMap getExtensionHistoryInformation()</code>) that can look like this (shortened - based on the settings shown above):

<source>
QMap(
("ABC",
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155791") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, false) )
( "UsedLastInRevision" ,  QVariant(QString, "25843") )
( "WasInstalledInLastRevision" ,  QVariant(bool, false) ) )  ) )
( "AnglePlanesExtension" , 
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155785") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, false) )
( "UsedLastInRevision" ,  QVariant(QString, "25843") )
( "WasInstalledInLastRevision" ,  QVariant(bool, false) ) )  ) )
( "CMFreg" , 
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155788") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, false) )
( "UsedLastInRevision" ,  QVariant(QString, "25842") )
( "WasInstalledInLastRevision" ,  QVariant(bool, false) ) )  ) )
( "ChangeTracker" , 
QVariant(QVariantMap, QMap(
("ExtensionId", QVariant(QString, "155789") )
( "IsCompatible" ,  QVariant(bool, true) )
( "IsInstalled" ,  QVariant(bool, true) )
( "UsedLastInRevision" ,  QVariant(QString, "25895") )
( "WasInstalledInLastRevision" ,  QVariant(bool, true) ) )  ) )         //would be a preselected install candidate if it wasn't already installed
...
</source>

The compatibility is now checked correctly for non installed extensions and the current extension id is retrieved:
<source>
//SlicerExtensionsManagerModelPrivate::getExtensionsInfoFromPreviousInstallations

...
parameters["productname"] = extensionName; 
parameters["slicer_revision"] = q->slicerRevision();
parameters["os"] = q->slicerOs();
parameters["arch"] = q->slicerArch();
const ExtensionMetadataType& metaData = retrieveExtensionMetadata(parameters);
extensionId = metaData.value("extension_id").toString();     //retrieve updated extension id for not installed extensions
...
isCompatible = (this->isExtensionCompatible(metaData,
this->SlicerRevision, this->SlicerOs, this->SlicerArch).length() == 0);
</source>

== Completed ==
NA

= Use case: Self update of Slicer =

== Proposed Changes ==

* Display list of nightly builds in a web view
* Selecting a new version should automate:
** download
** auto-start installer, with extra information for the next steps
** auto-start new Slicer
** possibly, directly going to the above extensions-reinstallation tab (maybe not even showing the main window before)
* stuff that is there, or might be useful
** cross-platform re-start functionality is there (as part of extension installation)
** probably, at least on Win/Linux, "start Slicer after installation" is possible
** unattended installation on any platform(s)?

= Notes =

== 2016.05.03 ==

* See also [[Documentation/Labs/ExtensionsFrameworkRoadmap]]

* Ability to ask the installer (or Slicer) to install other extensions
** Would it make sense to transition to http://doc.qt.io/qtinstallerframework/ ?

* Serialize what is installed in you current Slicer

* Update of installed extensions:
** See https://www.slicer.org/slicerWiki/index.php/Documentation/4.5/SlicerApplication/ExtensionsManager#Updating_installed_extensions