Updating tool versions in a tutorial
Author(s) | Wendi Bacon |
Editor(s) | Julia Jakiela Bérénice Batut |
OverviewQuestions:Objectives:
How can I update the tool versions in a tutorial in the GTN?
What else do these updates impact, and how do I update that for consistency?
Requirements:
Implement tutorial tool version update on existing GTN material
Time estimation: 1 hourSupporting Materials:
Published: Nov 29, 2024Last modification: Nov 29, 2024License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License. The GTN Framework is licensed under MITpurl PURL: https://gxy.io/GTN:T00474version Revision: 1
Here, we provide a clear set of instructions for updating a GTN tutorial that uses the Galaxy interface for performing analysis. The GTN, and Galaxy, have a number of features to make it as reproducible and shareable as possible, so navigating what needs to be done for an overall update may feel daunting - but not anymore!
We encourage you to pick a tutorial to try this on, so you can use this tutorial side-by-side.
Phase 1: Find a tutorial with an outdated workflow
Hands-on: Choose Your Own TutorialThis is a "Choose Your Own Tutorial" section, where you can select between multiple paths. Click one of the buttons below to select how you want to follow the tutorial
Do you already have a tutorial and workflow you want to update?
Hands-on: Check for an outdated workflow
- Find a hands_on Hands-on tutorial in your training topic of interest (for example, Single-cell)
- Select the workflow workflow from the header
Import the workflow to your favourite Galaxy server
- Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
- Click on galaxy-upload Import at the top-right of the screen
- Provide your workflow
- Option 1: Paste the URL of the workflow into the box labelled “Archived Workflow URL”
- Option 2: Upload the workflow file in the box labelled “Archived Workflow File”
- Click the Import workflow button
Below is a short video demonstrating how to import a workflow from GitHub using this procedure:
- Go to the Workflow menu and select galaxy-wf-edit Edit workflow
- Click through the tools in the workflow and check the tool-versions to see if the tools are outdated.
If the workflow has tools that are up to date (or very close!), great! That tutorial does not need updating! Try another one!
Hands-on: Import the workflow
- Navigate to your target hands_on Hands-on tutorial
- Select the workflow workflow from the header
Import the workflow to your favourite Galaxy server
- Click on Workflow on the top menu bar of Galaxy. You will see a list of all your workflows.
- Click on galaxy-upload Import at the top-right of the screen
- Provide your workflow
- Option 1: Paste the URL of the workflow into the box labelled “Archived Workflow URL”
- Option 2: Upload the workflow file in the box labelled “Archived Workflow File”
- Click the Import workflow button
Below is a short video demonstrating how to import a workflow from GitHub using this procedure:
- Go to the Workflow menu and select galaxy-wf-edit Edit workflow
Phase 2: Check that nobody else is working on this
It’s always a good idea to check, just in case!
You can:
- Send a message to the GTN Matrix Channel is your quickest way forward.
- Search through the GTN Github Repository for existing draft Pull Requests.
- Check with individual communities, who may have their own method of tracking. For example, the 🖖🏾Single-cell & sPatial Omics Community have a shared Click-Up board at the time of writing.
Phase 3: Update the workflow
Now, you will update the workflow to using the latest tool-versions tool versions.
Hands-on: Update the workflow
- Select galaxy-wf-options Workflow Options from the Workflow Editor
- Select upgrade_workflow Upgrade workflow to automatically update the tools.
- Address any issues that may arise.
- Save the workflow.
Phase 4: Test & fix the workflow
It’s crucial to test the workflow, as often times the outputs will be different due to the new tool versions. It can also transpire that the newer tool versions lead to errors, either because they don’t work or because you need to change parameter settings that were previously unavailable or not required. Testing is key!
Hands-on: Test the workflow
- Import the input datasets from the tutorial you are upgrading (follow the instructions on the tutorial itself for this).
- Run your updated workflow on the input datasets.
- Address any issues that may arise, and NOTE DOWN all changes.
Ensure the workflow meets workflow best practices using the galaxy-wf-best-practices Best Practices button and add your name as a contributor, if not there already.
When you are editing a workflow, there are a number of additional steps you can take to ensure that it is a Best Practice workflow and will be more reusable.
- Open a workflow for editing
- In the workflow menu bar, you’ll find the galaxy-wf-options Workflow Options dropdown menu.
Click on it and select galaxy-wf-best-practices Best Practices from the dropdown menu.
This will take you to a new side panel, which allows you to investigate and correct any issues with your workflow.
The Galaxy community also has a guide on best practices for maintaining workflows. This guide includes the best practices from the Galaxy workflow panel, plus:
- adding tests to the workflow
- publishing the workflow on GitHub, a public GitLab server, or another public version-controlled repository
- registering the workflow with a workflow registry such as WorkflowHub or Dockstore
- Add yourself as an author of the workflow.
- Save the
updated workflow
.- Run your
updated workflow
on the input datasets in a fresh history.Save this history as an
answer key history
& make your history publicly available in Published Histories.Sharing your history allows others to import and access the datasets, parameters, and steps of your history.
Access the history sharing menu via the History Options dropdown (galaxy-history-options), and clicking “history-share Share or Publish”
- Share via link
- Open the History Options galaxy-history-options menu at the top of your history panel and select “history-share Share or Publish”
- galaxy-toggle Make History accessible
- A Share Link will appear that you give to others
- Anybody who has this link can view and copy your history
- Publish your history
- galaxy-toggle Make History publicly available in Published Histories
- Anybody on this Galaxy server will see your history listed under the Published Histories tab opened via the galaxy-histories-activity Histories activity
- Share only with another user.
- Enter an email address for the user you want to share with in the Please specify user email input below Share History with Individual Users
- Your history will be shared only with this user.
- Finding histories others have shared with me
- Click on the galaxy-histories-activity Histories activity in the activity bar on the left
- Click the Shared with me tab
- Here you will see all the histories others have shared with you directly
Note: If you want to make changes to your history without affecting the shared version, make a copy by going to History Options galaxy-history-options icon in your history and clicking Copy this History
Phase 5: Update the tutorial
Warning: If you run out of timeYou have already completed a large chunk of work (
updated workflow
+answer key history
) to get here, and we don’t want to lose it! Updating the tutorial to match theupdated workflow
can be a separate contribution. So if you get to this point and run out of time, please:
- Create an issue on the GTN Github Repository and include shareable links to your
updated workflow
&answer key history
- Message on the GTN Matrix channel with a link to your issue and explaining which tutorial you updated. If, however, you are able to finish the task yourself, please read on!
Note that you will need to have done either the Contributing to the GTN Github tutorial using command line or the Contributing to the GTN Github tutorial using Github Desktop. It’s helpful as well to understand the folder structure in the GTN, particularly how images go in an image folder either in a tutorial or in the parent folder. Each topic
has roughly the following structure:
Each topic has the following structure:
├── README.md
├── metadata.yaml
├── images
├── docker
│ ├── Dockerfile
├── slides
│ ├── index.html
├── tutorials
│ ├── tutorial1
│ │ ├── tutorial.md
│ │ ├── images
│ │ ├── slides.html
│ │ ├── data-library.yaml
│ │ ├── workflows
│ │ │ ├── workflow.ga
The tutorial.md
is what you’ll be editing, however you will also at the end upload a workflow.ga
file and likely some image files.
Hands-on: Update the tools in the tutorial
- Update tool versions: Each hands_on Hands-on step in the tutorial will likely have tool (s) with versions in the text. Update these versions / links to be equivalent to your
updated workflow
.- Update tool instructions: (Simultaneously) update the text to address any differences arising during the update, i.e. new parameters to set or other changes.
- Update images: Wherever images are used to show tool outputs, these will need updating. Use your final
answer key history
for this.- Update text: Wherever results are referenced in the tutorial text, update these numbers (and possibly interpretation) to reflect the new
answer key history
.- Update header: In the metadata at the beginning of the tutorial, there are likely
answer key history
links. Update this by adding the link to youranswer key history
.- Update workflow file: In the tutorial folder in the
training-material
repository, there is a subfolder titledworkflows
. Download yourupdated workflow
as a file and deposit that file there.- Check if there are any links to workflows/histories in the tutorial text, and if so, update them.
- Add your name as a contributor to the tutorial as an editor in the metadata.
You may find this tutorial to be a helpful reference in the Markdown content of a GTN tutorial: Creating content in Markdown
Phase 6: Make a pull request and (optional) update workflow testing
At this point, you’re welcome to make your Pull Request for the updated tutorial. However, it is likely to fail linting if the workflow testing is not also updated. This can be tricky, so we’d rather you make the Pull Request with all the work you did than let this stop you.
But if you can make the workflow tests, that would be amazing! Here’s the tutorial: Creating workflow tests using Planemo
congratulations Congratulations! You’ve made it to the end! Thank you so much for contributing to the sustainability of Galaxy tutorials!