Architecture 08 - Galaxy Application Components: Models, Managers, and Services

</span></div>

</span></div>

---

# Architecture 08 - Galaxy Application Components: Models, Managers, and Services

<div class="contributors-line">
		<a href="/training-material/hall-of-fame/jmchilton/" class="contributor-badge contributor-jmchilton"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/jmchilton?s=36" alt="John Chilton avatar" width="36" class="avatar" />
    John Chilton</a>
	</div>

</div>

<div class="footnote" style="bottom: 8em;">
  <i class="far fa-calendar" aria-hidden="true"></i><span class="visually-hidden">last_modification</span> Updated:   
  
</div>

<div class="footnote" style="bottom: 5em;">

<i class="fas fa-file-alt" aria-hidden="true"></i><span class="visually-hidden">text-document</span><a href="slides-plain.html"> Plain-text slides</a> |

</div>

<div class="footnote" style="bottom: 2em;">
    <strong>Tip: </strong>press <kbd>P</kbd> to view the presenter notes
    | <i class="fa fa-arrows" aria-hidden="true"></i><span class="visually-hidden">arrow-keys</span> Use arrow keys to move between slides

</div>

???
Presenter notes contain extra information which might be useful if you intend to use these slides for teaching.

Press `P` again to switch presenter notes off

Press `C` to create a new window where the same presentation will be displayed.
This window is linked to the main window. Changing slides on one will cause the
slide to change on the other.

Useful when presenting.

---

### <i class="far fa-question-circle" aria-hidden="true"></i><span class="visually-hidden">question</span> Questions

- How is business logic organized in Galaxy?

- What are models, managers, and services?

- How does the database layer work?

---

### <i class="fas fa-bullseye" aria-hidden="true"></i><span class="visually-hidden">objectives</span> Objectives

- Understand the three-layer architecture

- Learn about SQLAlchemy and the ORM

- Understand database migrations with Alembic

- Navigate the Galaxy data model

---

# Architecture 08 - Galaxy Application Components: Models, Managers, and Services

*The architecture surrounding Galaxy components.*

---

layout: true
name: left-aligned
class: left, middle
---
layout: true
class: center, middle

---

![This section will talk about that manager layer and what lies below](../../images/asgi_app.plantuml.svg)

---

There are many ways to describe and visualize the Galaxy server architecture,
one is to imagine the Galaxy database as the ultimate source for Galaxy "stuff"
and the API controllers as the ultimate sink.

In this architecture imagining of Galaxy, managers are the layer meant to
mediate all controller interactions (and isolate the backend from the web
framework) while the model layer is meant to mediate all database interactions
(and isolate the backend from database internals).

---

![Models and Managers](../../images/core_models_managers.plantuml.svg)

---

### Services

Handle API and web processing details of requests and responses at a high-level.

Thin layer below the controllers to shield applciation logic from FastAPI internals.

In practice, it is totally fine to skip this layer and have FastAPI controllers talk directly
to managers. Also in practice, there are many places where the controller or service layers
are thicker than they should be - and these are anti-patterns that shouldn't be followed.

---

### Managers

High-level business logic that ties all of these components together.

Controllers should ideally be thin wrappers around actions defined in managers.

Whenever a model requires more than just the database, the operation should be defined
in a manager instead of in the model.

---

### Managers - Some Key Files

![Key Managers](../../images/core_files_managers.mindmap.plantuml.svg)

---

### Managers - Some Helpers

![Manager Helpers](../../images/core_files_managers_helpers.mindmap.plantuml.svg)

---

### Galaxy Models

- Database interactions powered by SQLAlchemy - https://www.sqlalchemy.org/.
- Galaxy doesn't think in terms of "rows" but "objects".
- Classes for Galaxy model objects defined in `lib/galaxy/model/__init__.py`.
- Classes mapped to database objects in same module via "declarative mapping".
  - Classes/attributes mapped to tables/columns
  - Associations between classes mapped to relationships between tables

---

![SQLAlchemy Architecture](../../images/sqla_arch_small.png)

---

### Galaxy Database Schema Migrations

- Automated execution of incremental, reversible changes to the database schema.
- Performed to update or revert the database schema to a newer or older version.
- Powered by Alembic - https://alembic.sqlalchemy.org/.
  - (as of 22.05; prior to that by SQLAlchemy Migrate)
- Each file in `lib/galaxy/model/migrations/alembic/versions_gxy` represents a migration description
  - `e7b6dcb09efd_create_gxy_branch.py`
  - `6a67bf27e6a6_deferred_data_tables.py`
  - `b182f655505f_add_workflow_source_metadata_column.py`

---

### More on Schema Migrations

- Great documentation in code README - `lib/galaxy/model/migrations/README.md`
  - Admin perspective on how to migrate databases forward and revert on problems.
  - Developer persepctive on how to add new revisions.
- Galaxy's data model is split into the galaxy model and the legacy install model:
  - Persisted in one combined database or two separate databases
  - Represented by 2 migration branches: "gxy" and "tsi"
- Schema changes defined in revision modules:
  - `lib/galaxy/model/migrations/alembic/versions_gxy` (gxy branch: galaxy model)
  - `lib/galaxy/model/migrations/alembic/versions_tsi` (tsi branch: legacy install model)

---

### Database Diagram

![Galaxy Schema](../../images/galaxy_schema.png)

https://galaxyproject.org/admin/internals/data-model/

---

![HDA foor bar...](../../images/hda.svg)

---

![HDA Dataset](../../images/hda_dataset.plantuml.svg)

---

### Dataset Metadata

- Typed key-value pairs attached to HDA.
- Keys and types defined at the datatype level.
- Can be used by tools to dynamically control the tool form.

???

---

![HDAs and HDCAs](../../images/hda_hdca.plantuml.svg)

---

![Workflows](../../images/workflow_definition.svg)

---

![Workflow Running](../../images/workflow_run.svg)

---

![Libraries](../../images/libraries.svg)

---

![Library Permissions](../../images/library_permissions.svg)

.footnote[Previous: [Galaxy Task Management with Celery](/training-material/topics/dev/tutorials/architecture-tasks/slides.html) | Next: [Galaxy Plugin Architecture](/training-material/topics/dev/tutorials/architecture-plugins/slides.html)]

---
### <i class="fas fa-key" aria-hidden="true"></i><span class="visually-hidden">keypoints</span> Key points

- Services handle high-level API processing

- Managers contain business logic

- Models mediate database interactions

- SQLAlchemy provides ORM

- Alembic handles schema migrations

---

## Thank You!

This material is the result of a collaborative work. Thanks to the [Galaxy Training Network](https://training.galaxyproject.org) and all the contributors!

</div>

</div>

Tutorial Content is licensed under <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.<br/>