Storage Management

name: inverse
layout: true
class: center, middle, inverse

</span></div>

</span></div>

---

# Storage Management

<div class="contributors-line">
		
	
<ul class="text-list">
			
			<li>
				<a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
    Nate Coraor</a>
			<li>
				<a href="/training-material/hall-of-fame/martenson/" class="contributor-badge contributor-martenson"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/martenson?s=36" alt="Martin Čech avatar" width="36" class="avatar" />
    Martin Čech</a>
			<li>
				<a href="/training-material/hall-of-fame/hexylena/" class="contributor-badge contributor-hexylena"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/hexylena?s=36" alt="Helena Rasche avatar" width="36" class="avatar" />
    Helena Rasche</a>
			<li>
				<a href="/training-material/hall-of-fame/shiltemann/" class="contributor-badge contributor-shiltemann"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/shiltemann?s=36" alt="Saskia Hiltemann avatar" width="36" class="avatar" />
    Saskia Hiltemann</a></li>
</ul>

</div>

<div class="footnote" style="bottom: 8em;">
  <i class="far fa-calendar" aria-hidden="true"></i><span class="visually-hidden">last_modification</span> Updated:   
  <i class="fas fa-fingerprint" aria-hidden="true"></i><span class="visually-hidden">purl</span><abbr title="Persistent URL">PURL</abbr>: <a href="https://gxy.io/GTN:S00007">gxy.io/GTN:S00007</a>
</div>

<div class="footnote" style="bottom: 5em;">

<i class="far fa-play-circle" aria-hidden="true"></i><span class="visually-hidden">video-slides</span> <a href="/training-material/videos/watch.html?v=/admin/tutorials/data-library/slides">Video slides</a> |

<i class="fas fa-file-alt" aria-hidden="true"></i><span class="visually-hidden">text-document</span><a href="slides-plain.html"> Plain-text slides</a> |

</div>

<div class="footnote" style="bottom: 2em;">
    <strong>Tip: </strong>press <kbd>P</kbd> to view the presenter notes
    | <i class="fa fa-arrows" aria-hidden="true"></i><span class="visually-hidden">arrow-keys</span> Use arrow keys to move between slides

</div>

???
Presenter notes contain extra information which might be useful if you intend to use these slides for teaching.

Press `P` again to switch presenter notes off

Press `C` to create a new window where the same presentation will be displayed.
This window is linked to the main window. Changing slides on one will cause the
slide to change on the other.

Useful when presenting.

---

### <i class="far fa-question-circle" aria-hidden="true"></i><span class="visually-hidden">question</span> Questions

- How does Galaxy locate data?

- How can I have Galaxy use multiple storage locations?

---

### <i class="fas fa-bullseye" aria-hidden="true"></i><span class="visually-hidden">objectives</span> Objectives

- Setup Galaxy with both the Hierarachical and Distributed Object Storages

---

# Data Libraries

- Provide a convenient way to share datasets with users
- Great for commonly used datasets (e.g. reference data, GTN tutorial data)

.image-75[![data libraries screenshot](/training-material/topics/admin/images/data/data-library-screenshot.png)]

???

- Data libraries provide a convenient way for Galaxy administrators to share datasets with users.
- This is ideal for commonly used datasets such as reference data, or data for GTN tutorials.

---

# Data Libraries

- Access to library datasets:
  - **Shared Data** menu, browse data and import into history
  - Directly from tool form

.pull-left[
.image-50[![data library access from tool form](/training-material/topics/admin/images/data/select-from-lib0.png)]

]

.pull-right[

.image-90[![data library access from tool form](/training-material/topics/admin/images/data/select-from-lib3.png)]

]

???

- Users can browse these data libraries and import datasets directly into their histories.
- Additionally, these datasets can also be selected directly from the tool form.

---

# Advantages of data libraries

- **Avoid duplication** of data
- Does not count towards user's **quota**
- Libraries can be shared with all users, or specific groups
- **Manage permissions** on library/dataset level using roles and groups.
  - Admins can *create* libraries.
  - Ordinary users can be granted permission to *manage* libraries

![library permissions](/training-material/topics/admin/images/data/library-permissions.png)

???

- Every dataset in the library is stored only once, no matter how many users are using it in their histories.
- The data in data libraries does not count against user quotas.
- Management of libraries can be delegated to users.
- And lastly, libraries can be public, restricted to individuals, or to groups.

---

# Importing Data

- There are multiple ways to add data to libraries:
  - From history
  - From user directory
  - From import directory and/or path (admins only)
  - From remote source (via API/BioBlend/ephemeris)

![import menu](/training-material/topics/admin/images/data/import-menu.png)

???

- Galaxy provides many options for importing data.
- You can import data from a history, or from a server directory.
- Importing data from a server directory is convenient, as Galaxy can recreate the folder structure that is on disk.
- Additionally Galaxy can store library data as a symlink.
- This prevents needing to copy large shared datasets into Galaxy's own data store.

---

# Configuration

In `galaxy.yml`:

* `user_library_import_dir`
  * Allows authorized non-administrators to upload a directory of files.
  * Directory must contain sub-directories named the same as user's email.
  * Works well in combination with `ftp_upload_dir`.
* `library_import_dir`
  * Allows administrators to upload a directory of files.
* `allow_path_paste`
  * Admin-only, allows importing from any path that the Galaxy's user has access to.

???

- The path of the server directories that users and admins can use to import files can be configured in galaxy.yml .
- For admins, it is also possible to enable import from any server path.

---
### <i class="fas fa-key" aria-hidden="true"></i><span class="visually-hidden">keypoints</span> Key points

- The distributed object store configuration allows you to easily expand that storage that is attached to your Galaxy.

- You can move data around without affecting users.

---

## Thank You!

This material is the result of a collaborative work. Thanks to the [Galaxy Training Network](https://training.galaxyproject.org) and all the contributors!

<div class="contributors-line">
		
<table class="contributions">
	
	<tr>
		<td><abbr title="These people wrote the bulk of the tutorial, they may have done the analysis, built the workflow, and wrote the text themselves.">Author(s)</abbr></td>
		<td>
			<a href="/training-material/hall-of-fame/natefoo/" class="contributor-badge contributor-natefoo"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/natefoo?s=36" alt="Nate Coraor avatar" width="36" class="avatar" />
    Nate Coraor</a><a href="/training-material/hall-of-fame/martenson/" class="contributor-badge contributor-martenson"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/martenson?s=36" alt="Martin Čech avatar" width="36" class="avatar" />
    Martin Čech</a><a href="/training-material/hall-of-fame/hexylena/" class="contributor-badge contributor-hexylena"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/hexylena?s=36" alt="Helena Rasche avatar" width="36" class="avatar" />
    Helena Rasche</a><a href="/training-material/hall-of-fame/shiltemann/" class="contributor-badge contributor-shiltemann"><img src="/training-material/assets/images/orcid.png" alt="orcid logo" width="36" height="36"/><img src="https://avatars.githubusercontent.com/shiltemann?s=36" alt="Saskia Hiltemann avatar" width="36" class="avatar" />
    Saskia Hiltemann</a>
		</td>
	</tr>

<tr class="reviewers">
		<td><abbr title="These people reviewed this material for accuracy and correctness">Reviewers</abbr></td>
		<td>
			<a href="/training-material/hall-of-fame/hexylena/" class="contributor-badge contributor-badge-small contributor-hexylena"><img src="https://avatars.githubusercontent.com/hexylena?s=36" alt="Helena Rasche avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/nsoranzo/" class="contributor-badge contributor-badge-small contributor-nsoranzo"><img src="https://avatars.githubusercontent.com/nsoranzo?s=36" alt="Nicola Soranzo avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/shiltemann/" class="contributor-badge contributor-badge-small contributor-shiltemann"><img src="https://avatars.githubusercontent.com/shiltemann?s=36" alt="Saskia Hiltemann avatar" width="36" class="avatar" /></a><a href="/training-material/hall-of-fame/abretaud/" class="contributor-badge contributor-badge-small contributor-abretaud"><img src="https://avatars.githubusercontent.com/abretaud?s=36" alt="Anthony Bretaudeau avatar" width="36" class="avatar" /></a></td>
	</tr>

</table>

</div>

</div>

Tutorial Content is licensed under <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.<br/>