Regression in Machine Learning

name: inverse
layout: true
class: center, middle, inverse

</span></div>

</span></div>

---

# Regression in Machine Learning

<div class="contributors-line">
		
	
<ul class="text-list">
			
			<li>
				<a href="/training-material/hall-of-fame/anuprulez/" class="contributor-badge contributor-anuprulez"><img src="https://avatars.githubusercontent.com/anuprulez?s=36" alt="Anup Kumar avatar" width="36" class="avatar" />
    Anup Kumar</a></li>
</ul>

</div>

<div class="footnote" style="bottom: 8em;">
  <i class="far fa-calendar" aria-hidden="true"></i><span class="visually-hidden">last_modification</span> Updated:   
  <i class="fas fa-fingerprint" aria-hidden="true"></i><span class="visually-hidden">purl</span><abbr title="Persistent URL">PURL</abbr>: <a href="https://gxy.io/GTN:S00137">gxy.io/GTN:S00137</a>
</div>

<div class="footnote" style="bottom: 5em;">

<i class="fas fa-file-alt" aria-hidden="true"></i><span class="visually-hidden">text-document</span><a href="slides-plain.html"> Plain-text slides</a> |

</div>

<div class="footnote" style="bottom: 2em;">
    <strong>Tip: </strong>press <kbd>P</kbd> to view the presenter notes
    | <i class="fa fa-arrows" aria-hidden="true"></i><span class="visually-hidden">arrow-keys</span> Use arrow keys to move between slides

</div>

???
Presenter notes contain extra information which might be useful if you intend to use these slides for teaching.

Press `P` again to switch presenter notes off

Press `C` to create a new window where the same presentation will be displayed.
This window is linked to the main window. Changing slides on one will cause the
slide to change on the other.

Useful when presenting.

---

## Requirements

Before diving into this slide deck, we recommend you to have a look at:

- [Introduction to Galaxy Analyses](/training-material/topics/introduction)

- [Statistics and machine learning](/training-material/topics/statistics)
    - Basics of machine learning: [<i class="fab fa-slideshare" aria-hidden="true"></i><span class="visually-hidden">slides</span> slides](/training-material/topics/statistics/tutorials/machinelearning/slides.html) - [<i class="fas fa-laptop" aria-hidden="true"></i><span class="visually-hidden">tutorial</span> hands-on](/training-material/topics/statistics/tutorials/machinelearning/tutorial.html)

---

### <i class="far fa-question-circle" aria-hidden="true"></i><span class="visually-hidden">question</span> Questions

- How to use regression techniques to create predictive models from biological datasets?

---

### <i class="fas fa-bullseye" aria-hidden="true"></i><span class="visually-hidden">objectives</span> Objectives

- Learn regression background

- Apply regression based machine learning algorithms

- Learn ageing biomarkers and predict age from DNA methylation datasets

- Learn how visualizations can be used to analyze predictions

---

# Regression

.pull-left[
- Supervised learning
- Real valued targets
- Cost/error/loss functions
- Algorithms
    - Linear model
    - Support vectors
    - Tree and Ensemble
- Used for
    - [Predict gene expression pattern](https://www.frontiersin.org/articles/10.3389/fgene.2019.00120/full)
    - [Estimate DNA copy number](https://academic.oup.com/bioinformatics/article/28/18/2357/252846)
    - [Identify drug response](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2881366/)
]

.pull-right[ ![Ensemble model](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r0.png)]

---

# Cost function

.pull-left[
- Mathematical functions
- Error = True - Predicted
- Examples
    - Mean squared error
    - Mean absolute error
    - Coefficient of determination (R2)
    - ...
]

.pull-right[ ![Ensemble model](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r0_1.png)]

---

# Linear models

.pull-left[
- Learn weight/coefficient for each feature
- y (predicted target) = w0 + w1 x Feature 1 + w2 x Feature 2 + .. +  wN x Feature N
- w (weights) = [w0, w1, w2, …, wN]
- X (input features) = [Feature1, Feature2, …, FeatureN]
- Examples
    - Linear regression
    - Ridge regression
    - ElasticNet
- Different variants of the minimisation equation
- Advantage: Simple and fast
- Disadvantage: Problems in learning non-linear relations
]

.pull-right[ ![Linear model](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r1.png)]

---

# Support vector machines

.pull-left[
- Linear and non-linear variants
- Support vectors
- Advantages
    - High-dimensional data 
    - Number of samples << number of dimensions
    - Memory efficient - uses only support vectors
- Disadvantages
    - Large runtime
    - Scale invariant
- Examples: SVR, NuSVR, LinearSVR
]

.pull-right[ ![Support vector machines](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r2.png)]

---

# K nearest neighbors

.pull-left[
- Prediction based on the nearest neighbours
- Examples
    - K Nearest neighbours: based on k neighbours
    - Radius neighbours: neighbours within r radius
- Advantages
    - Simple to understand
    - Non-parametric
- Disadvantages
    - Runtime increases with data
    - High memory requirements
    - Insensitive to outliers
]

.pull-right[  ![K nearest neighbours](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r3.png)]

---

# Decision tree

.pull-left[
- Decision rules and paths
- Advantages
    - Easy to interpret
    - Logarithmic cost for prediction
- Disadvantages
    - Sensitive to variations in data
    - Prone to overfit
    - Need to balance dataset
]

.pull-right[ ![Decision tree](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r4.png)]

---

# Ensemble models

.pull-left[
- Combination of multiple trees
- Bagging
    - Build independent multiple trees 
    - Average prediction
    - Examples
        - Random Forest
        - Extremely Randomised Trees
- Boosting
    - Improve tree models sequentially
    - Combine weak models to robust ensemble
    - Examples
        - AdaBoost
        - GradientBoosting
]

.pull-right[ ![Ensemble model](/training-material/topics/statistics/tutorials/regression_machinelearning/images/r5.png)]

---

# References

- Linear models - https://scikit-learn.org/stable/modules/linear_model.html
- Support vector machines - https://www.sciencedirect.com/science/article/pii/S0022169406000473
- Nearest neighbours - https://scikit-learn.org/stable/modules/neighbors.html
- Decision tree - https://gdcoder.com/decision-tree-regressor-explained-in-depth/
- Ensemble models - https://www.geosci-model-dev.net/12/1209/2019/

---

# For additional references, please see tutorial's References section

---

- Galaxy Training Materials ([training.galaxyproject.org](https://training.galaxyproject.org))

![Screenshot of the gtn stats page with 21 topics, 170 tutorials, 159 contributors, 16 scientific topics, and a growing community](/training-material/topics/introduction/images/gtn_stats.png)

???

- If you would like to learn more about Galaxy, there are a large number of tutorials available.
- These tutorials cover a wide range of scientific domains.

---

# Getting Help

- **Help Forum** ([help.galaxyproject.org](https://help.galaxyproject.org))

![Galaxy Help](/training-material/topics/introduction/images/galaxy_help.png)

- **Gitter Chat**
    - [Main Chat](https://gitter.im/galaxyproject/Lobby)
    - [Galaxy Training Chat](https://gitter.im/Galaxy-Training-Network/Lobby)
    - Many more channels (scientific domains, developers, admins)

???

- If you get stuck, there are ways to get help.
- You can ask your questions on the help forum.
- Or you can chat with the community on Gitter.

---

# Join an event

- Many Galaxy events across the globe
- Event Horizon: [galaxyproject.org/events](https://galaxyproject.org/events)

![Event schedule](/training-material/topics/introduction/images/event_horizon.png)

???

- There are frequent Galaxy events all around the world.
- You can find upcoming events on the Galaxy Event Horizon.

---
### <i class="fas fa-key" aria-hidden="true"></i><span class="visually-hidden">keypoints</span> Key points

- Regression is a supervised approach in machine learning.

- For regression tasks, data is divided into training and test sets.

- Using regression, the samples are learned using the training set and predicted using the test set.

- For each regression algorithm, the parameters should be optimised based on the dataset.

- Regression methods can be applied to, for example, RNA-seq gene expression to predict biological age.

---

## Thank You!

This material is the result of a collaborative work. Thanks to the [Galaxy Training Network](https://training.galaxyproject.org) and all the contributors!

<div class="contributors-line">
		
<table class="contributions">
	
	<tr>
		<td><abbr title="These people wrote the bulk of the tutorial, they may have done the analysis, built the workflow, and wrote the text themselves.">Author(s)</abbr></td>
		<td>
			<a href="/training-material/hall-of-fame/anuprulez/" class="contributor-badge contributor-anuprulez"><img src="https://avatars.githubusercontent.com/anuprulez?s=36" alt="Anup Kumar avatar" width="36" class="avatar" />
    Anup Kumar</a>
		</td>
	</tr>

<tr class="reviewers">
		<td><abbr title="These people reviewed this material for accuracy and correctness">Reviewers</abbr></td>
		<td>
			<a href="/training-material/hall-of-fame/bgruening/" class="contributor-badge contributor-badge-small contributor-bgruening"><img src="https://avatars.githubusercontent.com/bgruening?s=36" alt="Björn Grüning avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/teresa-m/" class="contributor-badge contributor-badge-small contributor-teresa-m"><img src="https://avatars.githubusercontent.com/teresa-m?s=36" alt="Teresa Müller avatar" width="36" class="avatar" /></a></td>
	</tr>

</table>

</div>

</div>

Tutorial Content is licensed under <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.<br/>