Architecture 01 - Galaxy Ecosystem and Projects
Contributors
John Chilton
Björn GrüningQuestions
What projects make up the Galaxy ecosystem?
How do different Galaxy projects interact?
What tools are available for developers vs administrators?
Objectives
Identify the major projects in the Galaxy ecosystem
Understand the difference between user-facing and developer tools
Learn about Galaxy communication channels
layout: introduction_slides topic_name: Galaxy Architecture
Architecture 01 - Galaxy Ecosystem and Projects
The architecture of the ecosystem.
layout: true left-aligned class: left, middle — layout: true class: center, middle
Please Interrupt!
We’re here to answer your questions about Galaxy architecture!
class: enlarge200
Chat: galaxyproject Lobby (via Element)
GitHub: github.com/galaxyproject
Bluesky: bsky.app/profile/galaxyproject.bsky.social
Mastodon: @galaxyproject
LinkedIn: Galaxy Project
The /galaxyproject projects
The architecture of the ecosystem.
Speaker Notes
Too many to cover all but we’ll cover some of the big ones in a few big categories. User-facing applications, projects for Galaxy plugin developers, and projects for Galaxy administrators.
Matrix Community with Element
Access via Element client or any Matrix client at https://matrix.to/#/#galaxyproject_Lobby:gitter.im
class: center
More links we’ll mention as we go through the slides:
- https://matrix.to/#/#galaxyproject_Lobby:gitter.im
- https://matrix.to/#/#galaxyproject_dev:gitter.im
- https://matrix.to/#/#galaxyproject_admins:gitter.im
- https://matrix.to/#/#galaxyproject_FederatedGalaxy:gitter.im
- https://matrix.to/#/#galaxyproject_bioblend:gitter.im
- https://matrix.to/#/#galaxyproject_ephemeris:gitter.im
- https://matrix.to/#/#usegalaxy-eu_Lobby:gitter.im
- https://matrix.to/#/#galaxy-iuc_iuc:gitter.im
- https://matrix.to/#/#bgruening_docker-galaxy-stable:gitter.im
- https://matrix.to/#/#Galaxy-Training-Network_Lobby:gitter.im
- https://matrix.to/#/#biocontainers_Lobby:gitter.im
- https://matrix.to/#/#bioconda_Lobby:gitter.im
Working group chats linked at https://galaxyproject.org/community/wg/.
class: enlarge200
User-Facing Applications
class: enlarge150
The main Galaxy application.
Web interface, database model, job running, etc…
Also includes other web applications including the ToolShed.
class: enlarge150
galaxyproject/training-material
Galaxy training material for scientists, developers, and admins. Powers https://training.galaxyproject.org/.
class: enlarge150
The Galaxy Hub is the community and documentation hub for the Galaxy Project. It is maintained by the community through this GitHub repository. It is a static website built using the metalsmith static site generator.
Powers https://galaxyproject.org/.
class: enlarge150
Content distribution system for publishing to multiple social media platforms. Post once, share across Mastodon, Bluesky, Matrix, Slack, and LinkedIn with automatic formatting.
class: enlarge150
Community resource repository for Galaxy users and contributors. Provides access to the Galaxy Community Catalog featuring tools, training materials, and workflows. Galaxy Labs for creating customized community subdomain pages.
class: enlarge150
Official Python client for the Galaxy, ToolShed, and CloudMan APIs.
Best documented path to scripting the Galaxy API.
A standalone client library for the Galaxy API, built using the type definitions from the main Galaxy client.
https://www.npmjs.com/package/@galaxyproject/galaxy-api-client
Galaxy API bindings for other languages, less actively maintained.
class: enlarge150
bgruening/docker-galaxy-stable
High quality Docker containers for stable Galaxy environments.
Releases corresponding to each new version of Galaxy.
Many flavors available.
class: white
class: enlarge200
For Plugin Developers
class: center
Galaxy tools maintained by the IUC (“Intergalactic Utilities Commission”).
A variety of tools, generally of high quality including many of the core tools for Galaxy main.
Demonstrates current tool development best practices - development on github and then deployed to test/main ToolSheds
Many older tools appearing on usegalaxy.org.
class: center
Tools Aside - More Repositories
Other repositories with high quality tools:
- Björn Grüning’s repo
- Peter Cock’s repos:
- ENCODE tools
- Biopython repo
- Galaxy Proteomics repo
- Greg von Kuster’s repo
- TGAC repo
- AAFC-MBB Canada repo
- Mark Einon’s repo
Intergalactic Workflow Commission. Hosting workflows and defining best practices for publishing workflows.
https://matrix.to/#/#galaxyproject_iwc:gitter.im
class: enlarge150
Command-line utilities to assist in the development of Galaxy tools and workflows. Linting, testing, deploying to ToolSheds…
The best practice approach for Galaxy tool development!
Builds Galaxy environments for Galaxy tool development including Docker container, virtual machines, Google compute images
galaxyproject/galaxy-language-server
Language server implementation for Galaxy tools. Visual Studio Code extension for tool development.
Test execution, code completion, best practices, documentation tooltips, etc..
class: enlarge150
Build Galaxy framework dependencies as Python wheels when needed.
class: enlarge150
Provides stable URLs and caching for application links, etc.. An important layer for reproducibility but largely transparent.
class: enlarge200
For Deployers and Admins
class: enlarge150
galaxyproject/{ansible-*, *-playbook}
usegalaxy-eu/{ansible-*, *-playbook}
Ansible components to automate almost every aspect of Galaxy installation and maintenance.
Ansible is an advanced configuration management system
These playbooks are used to maintain Galaxy main, cloud and Docker images, virtual machines, …
class: enlarge150
A process manager (supervisor) and management tools for Galaxy servers.
galaxyctl which is used to manage the starting, stopping, and logging of Galaxy’s various processes.
galaxy which can be used to run a Galaxy server in the foreground.
class: enlarge150
Kubernetes helm chart for deploying Galaxy. Leveraged by cloudlaunch and CloudMan but usable standalone.
class: enlarge150
Distributed job execution engine for Galaxy.
Stages data, scripts, configuration.
Can run jobs on Windows machines.
Can act as its own queuing system or access an existing cluster DRM.
class: enlarge150
Library and CLI for managing Galaxy plugins - tools, index data, and workflows.
Layer on top of BioBlend building useful utilities for working with the Galaxy API from an administrator perspective.
class: enlarge150
Handy command-line utility for Galaxy administrators.
class: enlarge150
ephemeris vs gxadmin
Ephemeris generally talks to the Galaxy API and is a pure Python project, gxadmin talks directly to the Galaxy database and relevant files.
galaxyproject/total-perspective-vortex
TotalPerspectiveVortex (TPV) provides an installable set of dynamic rules for the Galaxy application that can route entities (Tools, Users, Roles) to appropriate job destinations based on a configurable yaml file.
class: enlarge150
Django-based infrastructure for creating pools of users, etc.. for training events and connecting them to Galaxy.
Some (out of many) friends of the project
Check out dev training materials “Tool Dependencies and Conda” and “Tool Dependencies and Containers” for more context.
Putting it all together
Speaker Notes
TODO: cover gxformat2
class: enlarge200
The rest of the slides will focus on the core repository.
.footnote[Next: Galaxy Project Management and Contribution]
Key Points
- Multiple user-facing applications (Galaxy, Training Material)
- Developer libraries for plugin developers
- Admin tools for deployment
- Active community on Matrix (via Element client)
Thank you!
This material is the result of a collaborative work. Thanks to the Galaxy Training Network and all the contributors!
Tutorial Content is licensed under
Creative Commons Attribution 4.0 International License.