Galaxy Installation with Ansible

name: inverse
layout: true
class: center, middle, inverse

</span></div>

</span></div>

---

# Galaxy Installation with Ansible

<div class="contributors-line">
		
	
<ul class="text-list">
			
			<li>
				
  <a href="/training-material/hall-of-fame/slugger70/" class="contributor-badge contributor-slugger70"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/slugger70?s=36" alt="Simon Gladman avatar" width="36" class="avatar" />
    Simon Gladman</a>
  
			<li>
				
  <a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
    Nate Coraor</a>
  </li>
</ul>

</div>

<div class="footnote" style="bottom: 8em;">
  <i class="far fa-calendar" aria-hidden="true"></i><span class="visually-hidden">last_modification</span> Updated:   
  <i class="fas fa-fingerprint" aria-hidden="true"></i><span class="visually-hidden">purl</span><abbr title="Persistent URL">PURL</abbr>: <a href="https://gxy.io/GTN:S00002">gxy.io/GTN:S00002</a>
</div>

<div class="footnote" style="bottom: 5em;">

<i class="far fa-play-circle" aria-hidden="true"></i><span class="visually-hidden">video-slides</span> <a href="/training-material/videos/watch.html?v=/admin/tutorials/ansible-galaxy/slides">Video slides</a> |

<i class="fas fa-file-alt" aria-hidden="true"></i><span class="visually-hidden">text-document</span><a href="slides-plain.html"> Plain-text slides</a> |

</div>

<div class="footnote" style="bottom: 2em;">
    <strong>Tip: </strong>press <kbd>P</kbd> to view the presenter notes
    | <i class="fa fa-arrows" aria-hidden="true"></i><span class="visually-hidden">arrow-keys</span> Use arrow keys to move between slides

</div>

???
Presenter notes contain extra information which might be useful if you intend to use these slides for teaching.

Press `P` again to switch presenter notes off

Press `C` to create a new window where the same presentation will be displayed.
This window is linked to the main window. Changing slides on one will cause the
slide to change on the other.

Useful when presenting.

---

### <i class="far fa-question-circle" aria-hidden="true"></i><span class="visually-hidden">question</span> Questions

- How does it all connect?

- What steps will we go through?

---

### <i class="fas fa-bullseye" aria-hidden="true"></i><span class="visually-hidden">objectives</span> Objectives

- Get a high-level overview of a Galaxy server setup

---

## Install PostgreSQL & Galaxy extensions

![Graph showing a Postgres DB in a minimal deployment (one node)](../../images/ansible-galaxy-0.png)

???

- The first step of a Galaxy deployment is the database.
- This is the foundation of everything.

---

## Install Galaxy & Attach Storage

![Galaxy is now attached to the DB. Storage is connected to Galaxy](../../images/ansible-galaxy-1.png)

???

- Galaxy is deployed, and attached to the database.
- Storage must be available for Galaxy

---

## Setup Gunicorn

![Gunicorn is attached to Galaxy](../../images/ansible-galaxy-2.png)

???

- Next, Gunicorn is setup to run the Galaxy app.

---

## Configure NGINX

![NGINX is added to proxy Gunicorn](../../images/ansible-galaxy-3.png)

???

- Next, nginx is attached to UWSGI to proxy connections and speed up access.

---

## Configure Backup

![A backup location area is added, with a node DB backup inside connected to the PostgreSQL db](../../images/ansible-galaxy-4.png)

???

- A backup location is a very important part of a Galaxy deployment.
- We only backup the database, as the Galaxy configuration is stored in your playbooks
- You probably also want to backup the user data storage.

---

## Configure CVMFS

![A cvmfs cache is added connected to Galaxy, next to storage](../../images/ansible-galaxy-5.png)

???

- CVMFS provides reference data and can be attached to your storage.
- Galaxy is configured to read data from CVMFS.
- Compute is configured to access it as well for jobs that need reference data.

---

## Configure Job Handlers

![Job Handlers are added, connected to Galaxy. This connects to a Slurm deployment which is connected to a compute node which points to storage and cvmfs](../../images/ansible-galaxy-6.png)

???

- Job handlers are configured and deployed with the app.
- These connect to the compute and manage jobs.
- Slurm is a much more intelligent resource manager than Galaxy.
- The job handlers are configured to connect to Slurm.
- Slurm deployment is explained in a separate tutorial.

---

## Setup Remote Compute

![Remote compute site, with Pulsar and compute is attached to the job handlers](../../images/ansible-galaxy-7.png)

???

- Lastly, we can scale Galaxy further with remote compute.
- Pulsar connected at a remote site will handle this.

---

## Major Initial Decisions

* Where to install Galaxy
* Where to store Galaxy datasets
* Database location

???

- These are the major initial decisions you will face.
- Where to install Galaxy, what servers or VMs do you have available?
- Where to store the data?
- Do you have enough space for your users?
- Where to reliably store the database?

---
## Where to install Galaxy

* Must be at same path on cluster - more on this in cluster sessions

???

- Galaxy should be installed somewhere that is available across the cluster.
- We'll cover this in detail in the lesson.

---
## Where to store Galaxy datasets

* Must be at same path on cluster
* Consider future scalability

???

- Where should data be stored?
- Do you have network-attached storage available?
- It must be available to the entire cluster where compute happens.

---
## Database location

* Fast local, reliable storage
* Consider future scalability

???

- The database server should be very reliable.
- It does not need so much disk space, but consider future scalability.

---
## Basic best practices

* Run as an **unprivileged user**
* When possible, separate *code* from *data* and *configs*
* Write protect code and configs

.left[All of these practices are supported in the [galaxyproject.galaxy][galaxy-role] Ansible role and covered in the [Galaxy Installation with Ansible][ansible-galaxy-tutorial] tutorial!]

[galaxy-role]: https://galaxy.ansible.com/galaxyproject/galaxy
[ansible-galaxy-tutorial]: /training-material/topics/admin/tutorials/ansible-galaxy/tutorial.html

???

- Here are the basic best practices.
- Run without privileges so if someone gains access they are limited in what they can do.
- Ensure the code and configuration are separate.
- If someone manages to act as the galaxy user, this will prevent them from changing galaxy's behaviour.
- All of these best practices are built into the ansible role.

---

## Example "Advanced" UseGalaxy.* Deployment

![The same deployment as before, but now they are segmented differently. Postgres and Rabbit MQ on their own hosts, storage on an NFS server, slurm and compute on a Cluster, and then nginx, gunicorn, galaxy, and job handlers on a Head Node](../../images/ansible-galaxy-usegalaxy.png)

???

- Here we can see what a UseGalaxy.* deployment looks like
- This is roughly representative of UseGalaxy.eu, but all Galaxies are slightly different.
- For instance some Galaxies have multiple head nodes to balance the load.

---
### <i class="fas fa-key" aria-hidden="true"></i><span class="visually-hidden">keypoints</span> Key points

- Everything can be accomplished with Ansible roles from Galaxy

- You can easily deploy a base Galaxy, or one with more features.

---

## Thank You!

This material is the result of a collaborative work. Thanks to the [Galaxy Training Network](https://training.galaxyproject.org) and all the contributors!

<div class="contributors-line">
		
<table class="contributions">
	
	<tr>
		<td><abbr title="These people wrote the bulk of the tutorial, they may have done the analysis, built the workflow, and wrote the text themselves.">Author(s)</abbr></td>
		<td>
			
  <a href="/training-material/hall-of-fame/slugger70/" class="contributor-badge contributor-slugger70"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/slugger70?s=36" alt="Simon Gladman avatar" width="36" class="avatar" />
    Simon Gladman</a>
  
  <a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
    Nate Coraor</a>
  
		</td>
	</tr>

<tr>
		<td><abbr title="These people edited the text, either for spelling and grammar, flow, GTN-fit, or other similar editing categories">Editor(s)</abbr></td>
		<td>
			
  <a href="/training-material/hall-of-fame/hexylena/" class="contributor-badge contributor-hexylena"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/hexylena?s=36" alt="Helena Rasche avatar" width="36" class="avatar" />
    Helena Rasche</a>
  </td>
	</tr>

<tr class="reviewers">
		<td><abbr title="These people reviewed this material for accuracy and correctness">Reviewers</abbr></td>
		<td>
			
  <a href="/training-material/hall-of-fame/slugger70/" class="contributor-badge contributor-slugger70">
    <img src="https://avatars.githubusercontent.com/slugger70?s=36" alt="Simon Gladman avatar" width="36" class="avatar" />
  </a>
  
  <a href="/training-material/hall-of-fame/hexylena/" class="contributor-badge contributor-hexylena">
    <img src="https://avatars.githubusercontent.com/hexylena?s=36" alt="Helena Rasche avatar" width="36" class="avatar" />
  </a>
  
  <a href="/training-material/hall-of-fame/lldelisle/" class="contributor-badge contributor-lldelisle">
    <img src="https://avatars.githubusercontent.com/lldelisle?s=36" alt="Lucille Delisle avatar" width="36" class="avatar" />
  </a>
  
  <a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo">
    <img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
  </a>
  
  <a href="/training-material/hall-of-fame/nsoranzo/" class="contributor-badge contributor-nsoranzo">
    <img src="https://avatars.githubusercontent.com/nsoranzo?s=36" alt="Nicola Soranzo avatar" width="36" class="avatar" />
  </a>
  
  <a href="/training-material/hall-of-fame/shiltemann/" class="contributor-badge contributor-shiltemann">
    <img src="https://avatars.githubusercontent.com/shiltemann?s=36" alt="Saskia Hiltemann avatar" width="36" class="avatar" />
  </a>
  
  <a href="/training-material/hall-of-fame/martenson/" class="contributor-badge contributor-martenson">
    <img src="https://avatars.githubusercontent.com/martenson?s=36" alt="Martin Čech avatar" width="36" class="avatar" />
  </a>
  </td>
	</tr>

</table>

</div>

</div>

Tutorial Content is licensed under <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.<br/>