UseGalaxy.eu's Training Infrastructure as a Service

Overview

Questions
  • Is this service appropriate for my event?

Objectives
  • Identify if it is appropriate

  • Interact with the UseGalaxy.eu admins to arrange for infrastructure

Time estimation: 10 minutes
Last modification: Jan 20, 2020
License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License The GTN Framework is MIT

Introduction

TIaaS Logo

UseGalaxy.eu has developed Training Infrastructure as a Service (TIaaS for short) which allows you to use UseGalaxy.eu with a private queue for your training event. Your trainees’ jobs won’t wait in the main queue, and can be processed much more quickly than they might be otherwise. This can provide the experience of a local, private Galaxy combined with a public Galaxy that you are not responsible for maintaining. Additionally if something goes wrong, you can conveniently blame the UseGalaxy.eu admins, rather than feeling the stress of debugging and fixing your private Galaxy.

Agenda

In this tutorial, we will see:

  1. Identify if TIaaS is Appropriate For Your Training
  2. How TIaaS Works
  3. The Application Process
  4. The Student’s Process
  5. The Status Dashboard
  6. Before Your Training
  7. During Your Training

Identify if TIaaS is Appropriate For Your Training

First consider the requirements for your training to see if TIaaS from UseGalaxy.eu is a good fit for you:

  • Do you need really special tools that are not already available on UseGalaxy.eu?
    • The UseGalaxy.eu server supports most training workflows and has one of the largest toolboxes
    • But we only install publicly available and open tools: usegalaxy.eu tool repository
    • If you need something special you might want to contact the admins, and they can potentially accomodate your needs.
  • Do you need extra guarantees that the server will be online?
    • The UseGalaxy.eu server has good uptime for a server run by an academic group, but it cannot make promises regarding availability.
    • Additionally this server can experience occasional slow downs due to usage by other groups and users. If you need more guarantees, please find an alternative.

How TIaaS Works

We have several groups of virtual machines (VMs) attached to UseGalaxy.eu that run user jobs. For trainings we attach a new group of VMs that is specially labelled for that training. When normal users run tools on our server, these jobs are instructed to avoid the training pools by default.

When your users join a training, using a special URL provided to you, they then are placed in a special training group. Their jobs will then preferentially run on a training machine, and, in the event there is no more capacity, they will run on the main queue. If a spot on a training VM opens up first, they will run there rather than continuing to wait in the main queue.

The Application Process

hands_on Hands-on: Apply for TIaaS Training

  1. Fill out the form

  2. The UseGalaxy.eu admin team will review the request and get in touch with you as needed to identify the compute resources you need for your training

  3. The UseGalaxy.eu admin team will inform you of the URL you should provide to your participants during the training. They can open this URL and they will be added to the special training group

The Student’s Process

We have a “test” TIaaS training setup which will never have compute resources associated with it, but you can use it to test the process of signing up, if you wish:

hands_on Hands-on: Join a TIaaS Training

  1. Click on this link: https://usegalaxy.eu/join-training/test

  2. In the background you are added to a group in Galaxy. If you were to run jobs they would be tagged with this special queue.

That’s it, you’re now in the “test” TIaaS group. It’s really that easy for students.

The Status Dashboard

Once your students are registered, and you’re running your training, a common question instructors ask are “Are you all done?” and students are often not as vocal or repsonsive as we would like. So the TIaaS service now has a dashboard you can view which shows you the queue status for everyone in your training group. This works by finding all of the members of that training group, and checking all jobs that were created in the last 3 hours. For all of those jobs, this is displayed as a simple dashboard with the status of these jobs:

It shows:

  • Overview of queue (how many are in state new/queued/ok/error)
  • Overview split by tools (how many people are done running Fastqc?)
  • A full listing of the queue
TIaaS Queue Status
Figure 1: The TIaaS Status dashboard gives you an overview of all jobs states (are they ok or not), as well as a breakdown by tool. This is useful for finding out if everyone is finished running FastQC this morning and if they mostly worked OK. Finally it gives you a detailed breakdown, shown in the order they were submitted. This can give you a more detailed feeling for how the students are progressing through the tutorial.

hands_on Hands-on: View the dashboard

  1. The status dashboard is just the same URL as to join the group, with /status at the end: https://usegalaxy.eu/join-training/test/status

Before Your Training

  1. Run through your planned trainings on the UseGalaxy.eu server to ensure everything is available including data libraries and tools.
  2. If anything is missing, contact the admins via Gitter or Email and they will work to resolve the missing training material resources.
  3. Follow the rest of the training material’s guide to preparing a workshop

During Your Training

  1. Check out your status dashboard
  2. Watch for problems and contact the admins via Gitter
    • Jobs spending abnormally long in the queue
    • Jobs failing unexpectedly
  3. Remind your participants to submit bug reports if they experience any tool errors
  4. Follow the rest of the training material’s guide to running a workshop

Conclusion

Key points

  • Infrastructure is available for running Galaxy trainings for free from UseGalaxy.eu

  • This can be easier than setting up a local Galaxy and may have more resources available

  • But have a backup plan

Frequently Asked Questions

Have questions about this tutorial? Check out the FAQ page for the Teaching and Hosting Galaxy training topic to see if your question is listed there. If not, please ask your question on the GTN Gitter Channel or the Galaxy Help Forum

Feedback

Did you use this material as an instructor? Feel free to give us feedback on how it went.

Click here to load Google feedback frame

Citing this Tutorial

  1. Helena Rasche, 2020 UseGalaxy.eu's Training Infrastructure as a Service (Galaxy Training Materials). https://training.galaxyproject.org/archive/2021-09-01/topics/instructors/tutorials/setup-tiaas-for-training/tutorial.html Online; accessed TODAY
  2. Batut et al., 2018 Community-Driven Data Analysis Training for Biology Cell Systems 10.1016/j.cels.2018.05.012

details BibTeX

@misc{instructors-setup-tiaas-for-training,
author = "Helena Rasche",
title = "UseGalaxy.eu's Training Infrastructure as a Service (Galaxy Training Materials)",
year = "2020",
month = "01",
day = "20"
url = "\url{https://training.galaxyproject.org/archive/2021-09-01/topics/instructors/tutorials/setup-tiaas-for-training/tutorial.html}",
note = "[Online; accessed TODAY]"
}
@article{Batut_2018,
    doi = {10.1016/j.cels.2018.05.012},
    url = {https://doi.org/10.1016%2Fj.cels.2018.05.012},
    year = 2018,
    month = {jun},
    publisher = {Elsevier {BV}},
    volume = {6},
    number = {6},
    pages = {752--758.e1},
    author = {B{\'{e}}r{\'{e}}nice Batut and Saskia Hiltemann and Andrea Bagnacani and Dannon Baker and Vivek Bhardwaj and Clemens Blank and Anthony Bretaudeau and Loraine Brillet-Gu{\'{e}}guen and Martin {\v{C}}ech and John Chilton and Dave Clements and Olivia Doppelt-Azeroual and Anika Erxleben and Mallory Ann Freeberg and Simon Gladman and Youri Hoogstrate and Hans-Rudolf Hotz and Torsten Houwaart and Pratik Jagtap and Delphine Larivi{\`{e}}re and Gildas Le Corguill{\'{e}} and Thomas Manke and Fabien Mareuil and Fidel Ram{\'{\i}}rez and Devon Ryan and Florian Christoph Sigloch and Nicola Soranzo and Joachim Wolff and Pavankumar Videm and Markus Wolfien and Aisanjiang Wubuli and Dilmurat Yusuf and James Taylor and Rolf Backofen and Anton Nekrutenko and Björn Grüning},
    title = {Community-Driven Data Analysis Training for Biology},
    journal = {Cell Systems}
}
                

Congratulations on successfully completing this tutorial!