Admin Training Course
purlPURL: https://gxy.io/GTN:P00000Comment: What is a Learning Pathway?
We recommend you follow the tutorials in the order presented on this page. They have been selected to fit together and build up your knowledge step by step. If a lesson has both slides and a tutorial, we recommend you start with the slides, then proceed with the tutorial.
Learn how to setup, configure, and maintain your own Galaxy server. This learning pathway will guide you through all the steps required to setup your own Galaxy server with Ansible, configuring it, and making it production ready.
This learning path covers all the topics usually taught during the regular 5-day admin training workshops. Each module represents what is usually roughly taught in one day during the events.
Monday: Setting up Galaxy with Ansible
This module covers getting a Galaxy server setup with Ansible, a server you will develop furhter in the rest of the modules
Time estimation: 3 hours 50 minutes
Learning Objectives
- Learn about different options about Galaxy deployment.
- Make an educated decision about your preferred deployment model.
- Learn Ansible basics
- Write a simple role
- Install a role from Ansible Galaxy (repository unrelated to the Galaxy Project)
- Have an understanding of how Galaxy's Ansible roles are structured and interact with one another
- Be able to use an Ansible playbook to install different flavors of Galaxy for different purposes
| Lesson | Slides | Hands-on | Recordings |
|---|---|---|---|
|
Galaxy from an administrator's point of view
|
|||
| Ansible | |||
| Galaxy Installation with Ansible |
Tuesday: Making the server useful
Here we pivot to focus on making our server useful; adding tools and data, configuring quotas and authentication
Time estimation: 7 hours
Learning Objectives
- Learn about different maintenance steps
- Setup postgres backups
- Setup cleanups
- Learn what to back up and how to recover
- Have an understanding of what CVMFS is and how it works
- Install and configure the CVMFS client on a linux machine and mount the Galaxy reference data repository
- Configure your Galaxy to use these reference genomes and indices
- Use an Ansible playbook for all of the above.
- Have an understanding of the way in which Galaxy stores and uses reference data
- Be able to download and use data managers to add a reference genome and its pre-calculated indices into the Galaxy reference data system
- Use an Ansible playbook for all of the above
- Configure your Galaxy to use Apptainer and BioContainers for running jobs
- Learn about Ephemeris
- Extract a list of tools from a workflow
- Install these tools on a given Galaxy
- Learn the Galaxy user/group management and assign Quotas.
- Understand the Role Based Access Control (RBAC) of Galaxy.
- Setup a data library
- Manage permissions
- Import data from disk
- Interact with Galaxy via BioBlend.
Wednesday: Clusters
Time estimation: 4 hours
Learning Objectives
- Be familiar with the basics of installing, configuring, and using Slurm
- Understand all components of the Galaxy job running stack
- Understand how the job conf controls Galaxy's jobs subsystem
- Have a strong understanding of Galaxy job destinations
- Understand the purpose and function of Galaxy job metrics
- Know how to map tools to job destinations
- Be able to use the dynamic job runner to make arbitrary destination mappings
- Understand the job resource selector config and dynamic rule creation
- The various ways in which tools can be mapped to destinations, both statically and dynamically
- How to write a dynamic tool destination (DTD)
- How to write a dynamic python function destination
- How to use the job resource parameter selection feature
- Have an understanding of what Pulsar is and how it works
- Install and configure a RabbitMQ message queueing server
- Install and configure a Pulsar server on a remote linux machine
- Be able to get Galaxy to send jobs to a remote Pulsar server
| Lesson | Slides | Hands-on | Recordings |
|---|---|---|---|
| Connecting Galaxy to a compute cluster | |||
|
job_conf.xml external-link
|
|||
| Mapping Jobs to Destinations using TPV | |||
| Running Jobs on Remote Resources with Pulsar |
Thursday: Expanding
Time estimation: 5 hours 30 minutes
Learning Objectives
- Have an understanding of what Celery is and how it works
- Install Redis
- Configure and start Celery workers
- Install Flower to the Galaxy venv and configure it
- Use an Ansible playbook for all of the above.
- Monitor a Celery task using the Flower dashboard
- Setup TIaaS
- Request and manage trainings
- Join a training
- Setup and start the Galaxy reports app.
- Learn gxadmin basics
- See some queries and learn how they help debug production issues
- Setup InfluxDB
- Setup Telegraf
- Setup Grafana
- Create several charts
- Have an understanding of Sentry
- Install Sentry
- Configure Galaxy and Pulsar to send errors to Sentry
- Monitor performance with Sentry
Friday: Grab bag
Here we have some additional topics, some of which are not admin related. Please feel free to pick and choose the tutorials that are interesting for you.
Time estimation: 9 hours 5 minutes
Learning Objectives
- Learn what a tool is and its structure
- Use the Planemo utilities to develop a tool
- Deal with the dependencies
- Write functional tests
- Make a tool ready for publishing in a ToolShed
- Understand and master dataset collections
- Learn about the Rule Based Uploader
- Create a tutorial from scratch
- Link a tutorial to a topic
- Create hands-on
- Add technical support for a tutorial
- Understand what Galaxy Interactive Tools are and how they work
- Be aware of the security implications of Interactive Tools
- Have a basic understanding of the Interactive Tools (GxIT/GIE) Proxy, its purpose, and configuration
- Be familiar with wildcard SSL certificates and how to get them from Let's Encrypt
- Configure your Galaxy to serve Interactive Tools using an Ansible Playbook
- Start, run, and use an Interactive Tool
| Lesson | Slides | Hands-on | Recordings |
|---|---|---|---|
|
Galaxy Troubleshooting
|
|||
|
Tool development and integration into Galaxy
|
|||
| Using dataset collections | |||
| Rule Based Uploader | |||
|
Creating a new tutorial
|
|||
| Galaxy Interactive Tools |
Editorial Board
This material is reviewed by our Editorial Board:
Helena Rasche
Nate Coraor
Simon Gladman