Skip to content

Latest commit

 

History

History
288 lines (223 loc) · 27 KB

UPGRADING.md

File metadata and controls

288 lines (223 loc) · 27 KB
Raw

Upgrading

This document summarizes the environmental changes that need to be made when upgrading IRIDA that cannot be automated.

Unreleased

  • This upgrade deletes sequence files from the file system when they are removed from IRIDA. To clean up all previously removed sequence files, a script can be found under the src/main/resources/scripts/sequence-files folder in the IRIDA repo.

22.05 to 22.09

  • This upgrade switches the OAuth2 implementation from using spring-security-oauth to spring-security-oauth2-authorization-server and spring-security-oauth2-resource-server. Due to the dependency updates we have changed the format of the OAuth2 access tokens, they are now JWT Tokens (https://jwt.io/introduction) and are encrypted/decrypted using a certificate within a java keystore. No default java keystore is provided, so administrators will need to update their deployments to configure an appropriate java keystore. The same java keystore will need to be present on all servers which allow api access, otherwise access tokens generated on one server will not work on any other server.
    • The required java keystore needs to be generated using keytool(included with standard java installation), an example is below:
    keytool -genkeypair -alias JWK -keyalg RSA -keystore /etc/irida/jwk-key-store.jks -validity 3650 -storepass SECRET -keypass SECRET -storetype PKCS12
    • After generating the java keystore as above you will need to update /etc/irida/irida.conf to specify the keystore location and the keystore password:
    oauth2.jwk.key-store=/etc/irida/jwk-key-store.jks
oauth2.jwk.key-store-password=SECRET

22.03 to 22.05

  • This upgrade deprecates two pipelines, SISTR_TYPING and MLST_MENTALIST, and disables them from being executed. Any previously-run analysis results will still function as normal. If you wish to re-enable these pipelines you can set irida.workflow.types.disabled= (i.e., set the value to empty) in the /etc/irida/irida.conf file and restart IRIDA. If you wish to keep these pipelines disabled but include your own additional disabled pipelines you can set irida.workflow.types.disabled=SISTR_TYPING,MLST_MENTALIST and add your own pipelines to disable after this list.
    • For SISTR_TYPING, it is recommended to switch to using the version implemented as a plugin https://github.com/phac-nml/irida-plugin-sistr. For MLST_MENTALIST, we can no longer provide support for the installation of this software in IRIDA and Galaxy and have chosen to deprecate it.

22.01 to 22.03

  • It is recommended to stop the servlet container before deploying the new war file.

21.09 to 22.01

  • This upgrade converted the project from bare Spring to Spring Boot, which deprecated a number of properties relating to database connection and setup. These deprecated properties are mentioned in /etc/irida/irida.conf.
  • Due to an update in Spring you will need to revoke tokens for all OAuth clients, you can perform this through the UI or with the following sql:
USE IRIDA_DB_NAME;
truncate oauth_access_token;
truncate oauth_refresh_token;
  • This upgrade changed the way css and js assets are compiled and as such custom login pages will need to be updated.
    • Replace <link rel="stylesheet" th:href="@{/dist/css/login.bundle.css}" /> with <webpacker:css entry="login" />.
    • Replace <script th:src="@{/dist/js/login.bundle.js}"></script> with <webpacker:js entry="login" />.

21.09.1 to 21.09.2

  • This updates log4j from 1.2.17 to 2.17.0. Most users will not need to take any action, but log4j 2 uses a different (XML) syntax for configuration. If you have any custom log4j configurations these will need to be updated to match the log4j 2 syntax.

21.05 to 21.09

  • It is recommended to stop the servlet container before deploying the new war file.

21.05.1 to 21.05.2

  • The SNVPhyl pipeline has been upgraded to version 1.2.3. Please make sure to install the necessary tools in Galaxy.

21.01.3 to 21.05

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.

21.01 to 21.01.3

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.

20.09 to 21.01

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • IRIDA 21.01 includes changes to the analysis plugin system. Analysis pipelines must be updated to support this new change.
  • This upgrade includes changes to the sample-metadata database system. While implementing this change we encountered issues with the metadata auditing system, so we recommend additional backup steps before performing this upgrade. See https://phac-nml.github.io/irida-documentation/administrator/upgrades/#sample-metadata-audit-record-updates for more details.
  • A couple of new columns have been added to the announcements' table in the database. The new announcement title column cannot be empty and will be given a default value. There is a script available to populate the new title from the header of the existing announcements content, which can be found under the src/main/resources/scripts/announcements folder in the IRIDA repo.

20.05 to 20.09

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • The SNVPhyl pipeline has been upgraded. Please make sure to install the necessary tools in Galaxy.
  • The AssemblyAnnotation and AssemblyAnnotationCollection pipelines have been upgraded. Please make sure to install the necessary tools in Galaxy.
  • We have upgraded our Galaxy Docker image to Galaxy 20.05: docker pull phacnml/galaxy-irida-20.05. The default account credentials for this Galaxy instance have changed. The username remains as [email protected] but the password is password. The admin key has also changed to fakekey. Please keep these changes in mind (and make adjustments to the Galaxy credentials in /etc/irida/irida.conf) if you use the Docker image.
    • We were also unable to automatically install the MentaLiST tool when building the Docker image. It does seem to be working when installing via the Galaxy web UI (though you may have to attempt to reinstall it multiple times), so if you wish to use MentaLiST in the Galaxy Docker instance you will have to install the tool once the Docker container is started. Please see the MentaLiST install instructions for more details. Once installed you will have to run docker exec -it [CONTAINER ID] supervisorctl restart galaxy: to restart Galaxy before you can use the tool.
  • While testing out tool installation we were encountering some issues installing tools via the web UI for older versions of Galaxy (<20.01). If installation of the Galaxy tools is not working via the Galaxy UI for older Galaxy instances we recommend using the tool Ephemeris to install tools in Galaxy via the command-line. More information about using Ephemeris to install IRIDA tools can be found at https://phac-nml.github.io/irida-documentation/administrator/galaxy/setup/#automated-installation-of-tools.

20.01 to 20.05

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • This version changes the endpoint for creating sequencing runs to allow any type of sequencer. The legacy sequencingRun/miseq endpoint is maintained, but deprecated. See https://phac-nml.github.io/irida-documentation/developer/rest/#creating-sequencing-runs for more info.
  • Assemblies can now be uploaded to IRIDA rather than just created through an analysis pipeline. This requires a new filesystem directory configured for files to be stored. You must add assembly.file.base.directory to your /etc/irida/irida.conf file and create a new diretory for these files to be stored.

19.09 to 20.01

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • This upgrade changes the Java version to Java 11. To upgrade, follow the install instructions for your system in https://phac-nml.github.io/irida-documentation/administrator/web/#prerequisite-install-instructions.
  • Tomcat 8 (or another Servlet 3.1 compatible servlet container) is required for this IRIDA version. Systems using Tomcat 7 must be upgraded before deploying this update.
  • This upgrade adds a required field to OAuth2 clients using the authorization_code grant (that is external applications connecting to IRIDA via the web application). This includes other IRIDA installations synchronizing data via the Remote API system, and Galaxy importer clients. In order for these systems to continue working properly, administrators must register a redirect URI for all authorization_code clients. For more on this process, see https://phac-nml.github.io/irida-documentation/administrator/upgrades/#2001.
  • The configuration key hibernate.dialect=org.hibernate.dialect.MySQL55Dialect should be set in your /etc/irida/irida.conf file for this release. Note the change from MySQL5Dialect to MySQL55Dialect.

19.05 to 19.09

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.

19.01 to 19.05

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • This upgrade will remove FastQC resuts from the database and move them to the file system. It is strongly recommended to make a backup of your database before this upgrade. Before upgrading you should read more at https://phac-nml.github.io/irida-documentation/administrator/upgrades/#1905.

19.01 to 19.01.2

  • A new configuration value is available to control the number of threads used for communication with Galaxy when running pipelines. The default value is 4. To change, please set irida.workflow.analysis.threads in the /etc/irida/irida.conf file. This can help when running lots of pipelines in IRIDA.

0.22.0 to 19.01

0.21.0 to 0.22.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.

  • This upgrade changes the way the file processors handle uploaded files. File processing now takes place as a scheduled task rather than immediately after files are uploaded. For deployments with multiple IRIDA servers running against the same database, prossing may not be performed by the IRIDA server the files were uploaded to and will instead be balanced among all the available servers. If you want to disable file processing on an IRIDA server, set the following property in /etc/irida/irida.conf : file.processing.process=false.

  • A new pipeline, bio_hansel, has been included. You will have to make sure to install the necessary Galaxy tools listed in the documentation.

  • The MentaLiST pipeline has been upgraded. Please make sure to install the necessary tools in Galaxy.

  • The SISTR pipeline has been upgraded to make use of shovill for assembly. Please make sure to install the shovill Galaxy tool. Also, please make sure to follow the additional instructions in https://phac-nml.github.io/irida-documentation/administrator/galaxy/pipelines/sistr/#address-shovill-related-issues, which involves some modifications of the conda environment for shovill. In particular, you must:

    1. Install the proper ncurses and bzip2 packages from the conda-forge channel.

      # activate the Galaxy shovill conda env
source galaxy/deps/_conda/bin/activate galaxy/deps/_conda/envs/[email protected]
# install ncurses and bzip2 from conda-forge channel
conda install -c conda-forge ncurses bzip2

    2. Set the SHOVILL_RAM environment variable in the conda environment:

      cd galaxy/deps/_conda/envs/[email protected]
mkdir -p etc/conda/activate.d
mkdir -p etc/conda/deactivate.d

echo -e "export _OLD_SHOVILL_RAM=\$SHOVILL_RAM\nexport SHOVILL_RAM=8" >> etc/conda/activate.d/shovill-ram.sh
echo -e "export SHOVILL_RAM=\$_OLD_SHOVILL_RAM" >> etc/conda/deactivate.d/shovill-ram.sh

      Please change 8GB to what works for you for shovill (or setup based on the $GALAXY_MEMORY_MB environment variable, see the linked instructions for more details).

0.20.0 to 0.21.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • Two new pipelines, refseq_masher and MentaLiST, are included with this release. Additional Galaxy tools will need to be installed. Please see the linked installation details for each pipeline for more information.

0.19.0 to 0.20.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • This upgrade removes Dandelion framework from IRIDA. -Ddandelion.profile.active=prod" should be removed from Tomcat settings. Please see https://phac-nml.github.io/irida-documentation/administrator/web/#servlet-container-configuration.
  • You may configure the number of days a password is valid for before it needs reset. Add the security.password.expiry key to /etc/irida/irida.conf to configure. To disable password expiry, set to -1 (default). For example to set to 90 days, add the following:
security.password.expiry=90

0.18.0 to 0.19.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.
  • Any custom pipelines developed should follow the new Analysis scheme. An Analysis subclass is no longer required (but may be used) for custom pipelines, but an AnalysisType entry is required. See developer docs for more information.

0.17.0 to 0.18.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.

0.16.0 to 0.17.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file. Note this upgrade removes a number of deprecated database tables such as remote_related_proejct and remote_sequence_file. While these tables were likely empty it is strongly recommended you back up your database before this upgrade.
  • Remove snapshot.file.base.directory entry from /etc/irida/irida.conf. This directory is no longer used. If the directory pointed to by this config entry is empty it may be safely removed.
  • The SISTR pipeline has been upgraded to use sistr_cmd 1.0.2. This new version must be installed in Galaxy. Please see https://phac-nml.github.io/irida-documentation/administrator/galaxy/pipelines/sistr/.

0.15.0 to 0.16.0

0.14.0 to 0.15.0

0.13.0 to 0.14.0

  • This upgrade makes schema changes to the databases and cannot be parallel deployed. Servlet container must be stopped before deploying the new war file.

0.12.0 to 0.13.0

  • You may now configure the number of threads to be used for file processing on the web server. These threads perform tasks such as unzipping files and running FastQC. The following configuration keys can be set in /etc/irida/irida.conf (default values displayed):

      file.processing.core.size=4
  file.processing.max.size=8
  file.processing.queue.capacity=512

  • Deploy the new war file.

0.11.0 to 0.12.0

1.0.0 to 0.11.0

  • No special upgrade steps required, just deploy the new war file.

1.0.0-alpha10 to 1.0.0

  • You may now configure the link and text that gets rendered on web pages under the 'Help' menu for accessing an external help forum. You can configure the link and text by adding some keys to /etc/irida/web.conf:

      help.page.title=Your Help Page Title
  help.page.url=http://www.example.org/help

    These are optional settings. If they are not configured, no link will appear in the 'Help' menu.

  • You may now configure the e-mail address that gets rendered on web pages under the 'Help' menu. You can configure the e-mail address by adding a key to /etc/irida/web.conf:

      [email protected]

    This is an optional setting. If it is not configured, no 'Contact Us' e-mail address will appear in the 'Help' menu.

  • Deploy the new war file.

1.0.0-alpha9 to 1.0.0-alpha10

  • No special upgrade steps required, just deploy the new war file.

1.0.0-alpha8 to 1.0.0-alpha9

  • No special upgrade steps required, just deploy the new war file.

1.0.0-alpha8 to 1.0.0-alpha8.2

  • Add irida.scheduled.threads key to /etc/irida/irida.conf file with the size of your desired thread pool. Suggested size is 2.

1.0.0-alpha7 to 1.0.0-alpha8

1.0.0-alpha6 to 1.0.0-alpha7

1.0.0-alpha5 to 1.0.0-alpha6

  • No special upgrade steps required, just deploy the new war file.

1.0.0-alpha4 to 1.0.0-alpha5

  • A new version of SNVPhyl should be installed in Galaxy, you must install a new tool repository suite_snvphyl with revision 99463e5aef1b from the IRIDA Main Toolshed. Please see https://phac-nml.github.io/irida-documentation/administrator/galaxy/pipelines/phylogenomics/#irida-whole-genome-phylogenomics for more information.

  • You can configure the location of the announcements file by adding a key to /etc/irida/irida.conf with the name updates.file with a value of the location of the Markdown formatted file that will be displayed. The user running your serlvet container must be able to read this file. Example:

      updates.file=/etc/irida/announcments.md

  • Administrator notifications for filesystem errors during file uploads in the web interface can be configured by adding a key to /etc/irida/irida.conf with the name irida.administrative.notifications.email with a value of the e-mail address to which notifications can be sent. Notifications are sent when the IRIDA-managed filesystem is in an inconsistent state (trying to write over files that already exist), when there is no disk space left on the volume to which the files are written, or when some other, unknown filesystem-related exception takes place. Example:

      [email protected]

  • The schedule for sending out project digest e-mails is defaulted to be once per day at midnight. You can customize the frequency for the digest notifications by adding a key to /etc/irida/irida.conf with the name irida.scheduled.subscription.cron with a value of a cron-like expression. Please see http://docs.spring.io/spring/docs/4.0.6.RELEASE/javadoc-api/org/springframework/scheduling/annotation/Scheduled.html#cron-- for the format of the expression. Example:

      # send e-mails weekly (on Sunday at midnight) instead of daily
  irida.scheduled.subscription.cron=0 0 0 * * 0

1.0.0-alpha3 to 1.0.0-alpha4

  • No special upgrade steps required, just deploy the new war file.