Architecture 15 - Galaxy Production Deployment

</span></div>

</span></div>

---

# Architecture 15 - Galaxy Production Deployment

<div class="contributors-line">
		<a href="/training-material/hall-of-fame/jmchilton/" class="contributor-badge contributor-jmchilton"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/jmchilton?s=36" alt="John Chilton avatar" width="36" class="avatar" />
    John Chilton</a>
	</div>

</div>

<div class="footnote" style="bottom: 8em;">
  <i class="far fa-calendar" aria-hidden="true"></i><span class="visually-hidden">last_modification</span> Updated:   
  
</div>

<div class="footnote" style="bottom: 5em;">

<i class="fas fa-file-alt" aria-hidden="true"></i><span class="visually-hidden">text-document</span><a href="slides-plain.html"> Plain-text slides</a> |

</div>

<div class="footnote" style="bottom: 2em;">
    <strong>Tip: </strong>press <kbd>P</kbd> to view the presenter notes
    | <i class="fa fa-arrows" aria-hidden="true"></i><span class="visually-hidden">arrow-keys</span> Use arrow keys to move between slides

</div>

???
Presenter notes contain extra information which might be useful if you intend to use these slides for teaching.

Press `P` again to switch presenter notes off

Press `C` to create a new window where the same presentation will be displayed.
This window is linked to the main window. Changing slides on one will cause the
slide to change on the other.

Useful when presenting.

---

### <i class="far fa-question-circle" aria-hidden="true"></i><span class="visually-hidden">question</span> Questions

- How is Galaxy deployed in production?

- What is the difference between development and production setups?

- How does usegalaxy.org work?

---

### <i class="fas fa-bullseye" aria-hidden="true"></i><span class="visually-hidden">objectives</span> Objectives

- Understand production deployment architecture

- Learn about PostgreSQL, nginx, and uWSGI

- Understand multi-process and multi-host setups

- Learn about usegalaxy.org infrastructure

---

# Architecture 15 - Galaxy Production Deployment

*The architecture of Galaxy's production deployment.*

---

layout: true
name: left-aligned
class: left, middle
---
layout: true
class: center, middle

---

SQLite

gunicorn all-in-one

Single process

Single host

Local jobs

]

PostgreSQL

gunicorn for web process + webless workers + nginx proxy

Multiple processes

Multiple hosts

Jobs across many clusters

https://usegalaxy.org/production
]

---

### PostgreSQL

- Database server can scale way beyond default sqlite
- Supports concurrent connections from multiple Galaxy processes
- Better performance for production workloads
- https://www.postgresql.org/
- Configuration: `github.com/galaxyproject/usegalaxy-playbook` → `roles/galaxyprojectdotorg.postgresql`

---

### nginx (or Apache)

- Optimized servers for serving static content
- Reverse proxy to Galaxy application servers
- Load balancing across multiple Galaxy processes
- https://www.nginx.com/resources/wiki/
- https://docs.galaxyproject.org/en/master/admin/nginx.html#proxying-galaxy-with-nginx
- Configuration: `github.com/galaxyproject/usegalaxy-playbook` -> `templates/nginx/usegalaxy.j2`

---

### Webless

- Galaxy typically runs in Gunicorn - a production-grade ASGI server
- This is a great tool for both development and production but in production typically job running and workflow scheduling should happen outside a webserver
- https://docs.galaxyproject.org/en/master/admin/scaling.html#gunicorn-for-web-serving-and-webless-galaxy-applications-as-job-handlers
- https://training.galaxyproject.org/training-material/topics/admin/tutorials/ansible-galaxy/tutorial.html

---

### Multi-processes

Threads in Python are limited by the [GIL](https://wiki.python.org/moin/GlobalInterpreterLock).

Running multiple processes of Galaxy and separate processes for web handling
and job processing works around this.

This used to be an important detail - but gravity + gunicorn make things a lot easier.

---

### Cluster Support

![Cluster Support](../../images/cluster_support.svg)

Galaxy can submit jobs to various cluster managers (Slurm, PBS, SGE, etc.)

https://docs.galaxyproject.org/en/master/admin/cluster.html

https://training.galaxyproject.org/training-material/topics/admin/tutorials/connect-to-compute-cluster/tutorial.html

---

### usegalaxy.org Web Architecture

![usegalaxy.org web servers](../../images/usegalaxy_webservers.svg)

---

### Complete usegalaxy.org Infrastructure

![usegalaxy.org servers](../../images/usegalaxyorg.svg)

Multiple web servers, job handlers, and compute clusters working together

---

### Key Production Considerations

- **Database**: Use PostgreSQL for production
- **Web Server**: Use nginx or Apache as reverse proxy
- **Processes**: Run multiple Galaxy processes
- **Job Handling**: Separate job handlers from web workers
- **Storage**: Use scalable object storage solutions
- **Monitoring**: Implement logging and monitoring
- **Backups**: Regular database and file backups

---

### Production Deployment Resources

- **Admin Training**: https://training.galaxyproject.org/topics/admin/
- **Galaxy Admin Docs**: https://docs.galaxyproject.org/en/master/admin/
- **Ansible Playbooks**: https://github.com/galaxyproject/usegalaxy-playbook
- **Community Support**: https://help.galaxyproject.org/

.footnote[Previous: [Galaxy Startup Process](/training-material/topics/dev/tutorials/architecture-startup/slides.html) | Next: [Galaxy Testing](/training-material/topics/dev/tutorials/architecture-tests/slides.html)]

---
### <i class="fas fa-key" aria-hidden="true"></i><span class="visually-hidden">keypoints</span> Key points

- Production uses PostgreSQL instead of SQLite

- nginx/Apache serve static content, uWSGI runs Galaxy

- Multiple processes and hosts for scalability

- usegalaxy.org runs across multiple servers and clusters

---

## Thank You!

This material is the result of a collaborative work. Thanks to the [Galaxy Training Network](https://training.galaxyproject.org) and all the contributors!

</div>

</div>

Tutorial Content is licensed under <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.<br/>