Updating tool versions in a tutorial

Overview
Creative Commons License: CC-BY Questions:
  • How can I update the tool versions in a tutorial in the GTN?

  • What else do these updates impact, and how do I update that for consistency?

Objectives:
  • Implement tutorial tool version update on existing GTN material

Requirements:
Time estimation: 1 hour
Supporting Materials:
Published: Nov 29, 2024
Last modification: Nov 29, 2024
License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License. The GTN Framework is licensed under MIT
purl PURL: https://gxy.io/GTN:T00474
version Revision: 1

Here, we provide a clear set of instructions for updating a GTN tutorial that uses the Galaxy interface for performing analysis. The GTN, and Galaxy, have a number of features to make it as reproducible and shareable as possible, so navigating what needs to be done for an overall update may feel daunting - but not anymore!

We encourage you to pick a tutorial to try this on, so you can use this tutorial side-by-side.

Phase 1: Find a tutorial with an outdated workflow

Hands-on: Choose Your Own Tutorial

This is a "Choose Your Own Tutorial" section, where you can select between multiple paths. Click one of the buttons below to select how you want to follow the tutorial

Do you already have a tutorial and workflow you want to update?

Hands-on: Check for an outdated workflow
  1. Find a hands_on Hands-on tutorial in your training topic of interest (for example, Single-cell)
  2. Select the workflow workflow from the header
  3. Import the workflow to your favourite Galaxy server

    • Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
    • Click on galaxy-upload Import at the top-right of the screen
    • Provide your workflow
      • Option 1: Paste the URL of the workflow into the box labelled “Archived Workflow URL”
      • Option 2: Upload the workflow file in the box labelled “Archived Workflow File”
    • Click the Import workflow button

    Below is a short video demonstrating how to import a workflow from GitHub using this procedure:

    Video: Importing a workflow from URL

  4. Go to the Workflow menu and select galaxy-wf-edit Edit workflow
  5. Click through the tools in the workflow and check the tool-versions to see if the tools are outdated.

If the workflow has tools that are up to date (or very close!), great! That tutorial does not need updating! Try another one!

Hands-on: Import the workflow
  1. Navigate to your target hands_on Hands-on tutorial
  2. Select the workflow workflow from the header
  3. Import the workflow to your favourite Galaxy server

    • Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
    • Click on galaxy-upload Import at the top-right of the screen
    • Provide your workflow
      • Option 1: Paste the URL of the workflow into the box labelled “Archived Workflow URL”
      • Option 2: Upload the workflow file in the box labelled “Archived Workflow File”
    • Click the Import workflow button

    Below is a short video demonstrating how to import a workflow from GitHub using this procedure:

    Video: Importing a workflow from URL

  4. Go to the Workflow menu and select galaxy-wf-edit Edit workflow

Phase 2: Check that nobody else is working on this

It’s always a good idea to check, just in case!

You can:

Phase 3: Update the workflow

Now, you will update the workflow to using the latest tool-versions tool versions.

Hands-on: Update the workflow
  1. Select galaxy-wf-options Workflow Options from the Workflow Editor
  2. Select upgrade_workflow Upgrade workflow to automatically update the tools.
  3. Address any issues that may arise.
  4. Save the workflow.

Phase 4: Test & fix the workflow

It’s crucial to test the workflow, as often times the outputs will be different due to the new tool versions. It can also transpire that the newer tool versions lead to errors, either because they don’t work or because you need to change parameter settings that were previously unavailable or not required. Testing is key!

Hands-on: Test the workflow
  1. Import the input datasets from the tutorial you are upgrading (follow the instructions on the tutorial itself for this).
  2. Run your updated workflow on the input datasets.
  3. Address any issues that may arise, and NOTE DOWN all changes.
  4. Ensure the workflow meets workflow best practices using the galaxy-wf-best-practices Best Practices button and add your name as a contributor, if not there already.

    When you are editing a workflow, there are a number of additional steps you can take to ensure that it is a Best Practice workflow and will be more reusable.

    1. Open a workflow for editing
    2. In the workflow menu bar, you’ll find the galaxy-wf-options Workflow Options dropdown menu.
    3. Click on it and select galaxy-wf-best-practices Best Practices from the dropdown menu.

      screenshot showing the best practices menu item in the gear dropdown.

    4. This will take you to a new side panel, which allows you to investigate and correct any issues with your workflow.

      screenshot showing the best practices side panel. several issues are raised like a missing annotation with a link to add that, and non-optional inputs that are unconnected. Additionally several items already have green checks like the workflow defining creator information and a license.

    The Galaxy community also has a guide on best practices for maintaining workflows. This guide includes the best practices from the Galaxy workflow panel, plus:

    • adding tests to the workflow
    • publishing the workflow on GitHub, a public GitLab server, or another public version-controlled repository
    • registering the workflow with a workflow registry such as WorkflowHub or Dockstore

  5. Add yourself as an author of the workflow.
  6. Save the updated workflow.
  7. Run your updated workflow on the input datasets in a fresh history.
  8. Save this history as an answer key history & make your history publicly available in Published Histories.

    Sharing your history allows others to import and access the datasets, parameters, and steps of your history.

    Access the history sharing menu via the History Options dropdown (galaxy-history-options), and clicking “history-share Share or Publish”

    1. Share via link
      • Open the History Options galaxy-history-options menu at the top of your history panel and select “history-share Share or Publish”
        • galaxy-toggle Make History accessible
        • A Share Link will appear that you give to others
      • Anybody who has this link can view and copy your history
    2. Publish your history
      • galaxy-toggle Make History publicly available in Published Histories
      • Anybody on this Galaxy server will see your history listed under the Published Histories tab opened via the galaxy-histories-activity Histories activity
    3. Share only with another user.
      • Enter an email address for the user you want to share with in the Please specify user email input below Share History with Individual Users
      • Your history will be shared only with this user.
    4. Finding histories others have shared with me
      • Click on the galaxy-histories-activity Histories activity in the activity bar on the left
      • Click the Shared with me tab
      • Here you will see all the histories others have shared with you directly

    Note: If you want to make changes to your history without affecting the shared version, make a copy by going to History Options galaxy-history-options icon in your history and clicking Copy this History

Phase 5: Update the tutorial

Warning: If you run out of time

You have already completed a large chunk of work (updated workflow+ answer key history) to get here, and we don’t want to lose it! Updating the tutorial to match the updated workflow can be a separate contribution. So if you get to this point and run out of time, please:

  1. Create an issue on the GTN Github Repository and include shareable links to your updated workflow & answer key history
  2. Message on the GTN Matrix channel with a link to your issue and explaining which tutorial you updated. If, however, you are able to finish the task yourself, please read on!

Note that you will need to have done either the Contributing to the GTN Github tutorial using command line or the Contributing to the GTN Github tutorial using Github Desktop. It’s helpful as well to understand the folder structure in the GTN, particularly how images go in an image folder either in a tutorial or in the parent folder. Each topic has roughly the following structure:

Each topic has the following structure:

├── README.md
├── metadata.yaml
├── images
├── docker
│   ├── Dockerfile
├── slides
│   ├── index.html
├── tutorials
│   ├── tutorial1
│   │   ├── tutorial.md
│   │   ├── images
│   │   ├── slides.html
│   │   ├── data-library.yaml
│   │   ├── workflows
│   │   │   ├── workflow.ga

The tutorial.md is what you’ll be editing, however you will also at the end upload a workflow.ga file and likely some image files.

Hands-on: Update the tools in the tutorial
  1. Update tool versions: Each hands_on Hands-on step in the tutorial will likely have tool (s) with versions in the text. Update these versions / links to be equivalent to your updated workflow.
  2. Update tool instructions: (Simultaneously) update the text to address any differences arising during the update, i.e. new parameters to set or other changes.
  3. Update images: Wherever images are used to show tool outputs, these will need updating. Use your final answer key history for this.
  4. Update text: Wherever results are referenced in the tutorial text, update these numbers (and possibly interpretation) to reflect the new answer key history.
  5. Update header: In the metadata at the beginning of the tutorial, there are likely answer key history links. Update this by adding the link to your answer key history.
  6. Update workflow file: In the tutorial folder in the training-material repository, there is a subfolder titled workflows. Download your updated workflow as a file and deposit that file there.
  7. Check if there are any links to workflows/histories in the tutorial text, and if so, update them.
  8. Add your name as a contributor to the tutorial as an editor in the metadata.

You may find this tutorial to be a helpful reference in the Markdown content of a GTN tutorial: Creating content in Markdown

Phase 6: Make a pull request and (optional) update workflow testing

At this point, you’re welcome to make your Pull Request for the updated tutorial. However, it is likely to fail linting if the workflow testing is not also updated. This can be tricky, so we’d rather you make the Pull Request with all the work you did than let this stop you.

But if you can make the workflow tests, that would be amazing! Here’s the tutorial: Creating workflow tests using Planemo

congratulations Congratulations! You’ve made it to the end! Thank you so much for contributing to the sustainability of Galaxy tutorials!