Galaxy usage on SURF Research Cloud

Author(s)

Overview
Questions:

How do I start a Galaxy instance on SURF Research Cloud?

How do I link to external storage from my Galaxy workspace?

How do I install tools?

Objectives:

Understand how to work with a Galaxy on-demand instance in SURF Research Cloud

Requirements:

Access to the SURF Research Cloud

SRAM authentication

An SSH key connected to your account

Have a basic understanding of how Galaxy works

Time estimation: 30 minutes

Level: Introductory Introductory

Supporting Materials:

Published: Dec 18, 2024

Last modification: Dec 19, 2024

License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License. The GTN Framework is licensed under MIT

version Revision: 3

Using Galaxy via the SURF Research Cloud (SRC) allows researchers to start Galaxy instances on-demand and analyze their data in a secure environment following the General Data Protection Regulations (GDPR). The instance provides secure authentication, where users must have a SURF Research account prior to this tutorial, have set the SURF Research Access Management (SRAM) authentication method, and connect an SSH key to their accounts. In case you are not familiar with SRC and need help in setting up your accounts, please follow the instructions on the SURF Knowledge Base

In this training the aim is to focus on:

Understanding how to start Galaxy on SRC
Attaching external volumes for storage

Galaxy instances can be started and stopped on demand, depending on personal cases and requirements. Inside the SRC members should have access to all publicly available catalog items. If you are not able to create a catalog item, please contact SURF servicedesk.

Agenda

Prerequisites

Starting a Galaxy instance inside SRC step-by-step

Create an external storage

Create a new workspace

Going Further with Pulsar?

Prerequisites

This tutorial assumes you are member of a Collaborative Organisation (CO) in SRAM that has access to SRC and a wallet with budget in SRC with enough sources to create Galaxy and Pulsar catalog items. (For more information please refer to the SURF Knowledge Base.

You must have previous experience working with data inside Galaxy.

Starting a Galaxy instance inside SRC step-by-step

Hands-on: Access the SRC

The first and most important step is to have access to the SURF Research Cloud.

You will need to login to the portal.

Assuming you are a Dutch researcher, you should expect to see your educational institution listed on the authentication provider screen.

Log in with your institute credentials and follow the SRAM authentication on your phone. Inside the environment, you can see if there are any active virtual machines, the options to create a workspace, new storage or request new wallet for your projects.

With this, you’re ready to get started launching resources within SRC.

Create an external storage

The purpose of the storage is to attach it to your workspace, where you will save your data. This way, if the machine is deleted, your data will not be lost. We strongly advise anyone using the SRC to create storages and attach them to their machines!

Hands-on: Create a Storage Volume

Locate the “Create new storage” option

Click on “Create new”.

Select the collaborative organisation you want the storage to be part of.

Select the desired cloud provider and the size of the volume you want to use.

If you are unsure about the size of the storage you need, contact your administrator. Consider also consulting Galaxy from an Administrator’s Point of View

Name your storage however you like and press on submit.

You will be redirected to the main page, where you will be able to track the status of the storage creation.

Once the storage is deployed, you can attach it to any of your machines! 🎉

Be aware, that in order to attach an external storage to an existing machine, you would need to pause the workspace first! (If you are not sure how to do that please refer to External storage volumes)

Create a new workspace

Depending on user’s rights, you can create a new workspace for your collaboration.

Workspaces may also be known as Virtual Machines/VMs or Instances in other cloud providers.

Hands-on: Create a Workspace

On the right side under the “Workspaces” click on the “Add” button.

Then, you will be redirected to a new page, where you must first choose the collaborative organisation in which you want to create your workspace (in case you are a member of multiple organisations). Once chosen, you a new page will be loaded, where you will have access to all available catalog items.

Use the magnifying glass on the right side of the panel and search for Galaxy. Two catalog items will appear - the Galaxy instance designed for SURF and a Galaxy Pulsar node that can be connected to the instance. Select “Galaxy at SURF”.

SURF Research Cloud allows researchers to host their catalog items on different cloud providers. The Galaxy catalog item is currently supported only on the HPC Cloud and for Ubuntu 22.04. On this page, users can select the number of cores and RAM that they want on their machine. More sizes can be added in the future, and on request. Choose wisely and in case you are not certain contact your administrator!

Select the storage you have created earlier so it is attached to the new workspace.

Lastly, before the workspace is deployed, you need to choose for how long the machine will run.

The standard life-time of the VM is 5 days. If you need it for longer, this option can be changed once the machine is running. Note, that once the machine is expired and deleted it cannot be restored! Plan accordingly and migrate your data in time to prevent data loss!

This is an incredibly useful feature as it saves you from forgetting to destroy a VM. Especially for GPU nodes it can help you ensure that they disappear after your computation is complete.

In this form, you can also select how to name your workspace, add a description (if you want) and specify the hostname. Then, scroll down and submit the workspace creation.

With that, your workspace will start being created.

A galaxy instance named galaxy demo is in status creating. — **Figure 1**: Creating the workspace can take some time. You will see on the main page the status of your workspace.

While you are waiting, you can create your Galaxy Pulsar node simultaneously to save time.

Once the workspace is running, press on the down arrow to check any information such as the ID, the owner of the workspace, the collaboration it is part of, when it is created and when it expires and the wallet it is using. Additionally, you can pause the workspace when you are not using it. Pressing on the storage option will display the storage that has been attached to the workspace.

To access the Galaxy instance, press on the “Access” button. This will redirect you to a new webpage. The SRAM login will be displayed again, and after authenticating, you will be redirected to the Galaxy at SURF Research Cloud page!

Congratulations! You have started a Galaxy instance inside the SURF Research Cloud! 🏄‍♀️🎉

If you have administrator rights, you can download tools that can be used by the members of the collaborative organisation. To obtain administrator rights you need to be part of the src_co_admin group inside the collaborative organisation. In case you are not sure how to download tools as a Galaxy administrator, please refer to Installing Tools into Galaxy.

Going Further with Pulsar?

Another important step when starting a local Galaxy instance on SRC is connecting it to a Galaxy Pulsar Node, which you can do in this Pulsar SRC tutorial. The Pulsar is an useful component that is used to execute your tasks on remote servers/clusters. In this case, the Pulsar is started as a new workspace, allowing users to also run interactive tools.

In SRC, this can also enable you to save costs. By choosing a smaller Galaxy instance, and attaching a large Pulsar node when you need to do computations, you can be sure to optimise your budget expenditures.

You've Finished the Tutorial

Key points

With SRC you can start your own Galaxy on-demand instance in a secure environment

You can log in and use Galaxy via the SRC with your university credentials

Frequently Asked Questions

Have questions about this tutorial? Check out the FAQ page for the Galaxy Server administration topic to see if your question is listed there. If not, please ask your question on the GTN Gitter Channel or the Galaxy Help Forum

Glossary

CO: Collaborative Organisation
GDPR: General Data Protection Regulations
SRAM: SURF Research Access Management
SRC: SURF Research Cloud

Feedback

Did you use this material as an instructor? Feel free to give us feedback on how it went.
Did you use this material as a learner or student? Click the form below to leave feedback.

Citing this Tutorial

Mirela Minkova, Helena Rasche, Dawa Ometto, Galaxy usage on SURF Research Cloud (Galaxy Training Materials). https://training.galaxyproject.org/training-material/topics/admin/tutorials/surf-research-cloud-galaxy/tutorial.html Online; accessed TODAY
Hiltemann, Saskia, Rasche, Helena et al., 2023 Galaxy Training: A Powerful Framework for Teaching! PLOS Computational Biology 10.1371/journal.pcbi.1010752
Batut et al., 2018 Community-Driven Data Analysis Training for Biology Cell Systems 10.1016/j.cels.2018.05.012

@misc{admin-surf-research-cloud-galaxy,
author = "Mirela Minkova and Helena Rasche and Dawa Ometto",
	title = "Galaxy usage on SURF Research Cloud (Galaxy Training Materials)",
	year = "",
	month = "",
	day = "",
	url = "\url{https://training.galaxyproject.org/training-material/topics/admin/tutorials/surf-research-cloud-galaxy/tutorial.html}",
	note = "[Online; accessed TODAY]"
}
@article{Hiltemann_2023,
	doi = {10.1371/journal.pcbi.1010752},
	url = {https://doi.org/10.1371%2Fjournal.pcbi.1010752},
	year = 2023,
	month = {jan},
	publisher = {Public Library of Science ({PLoS})},
	volume = {19},
	number = {1},
	pages = {e1010752},
	author = {Saskia Hiltemann and Helena Rasche and Simon Gladman and Hans-Rudolf Hotz and Delphine Larivi{\`{e}}re and Daniel Blankenberg and Pratik D. Jagtap and Thomas Wollmann and Anthony Bretaudeau and Nadia Gou{\'{e}} and Timothy J. Griffin and Coline Royaux and Yvan Le Bras and Subina Mehta and Anna Syme and Frederik Coppens and Bert Droesbeke and Nicola Soranzo and Wendi Bacon and Fotis Psomopoulos and Crist{\'{o}}bal Gallardo-Alba and John Davis and Melanie Christine Föll and Matthias Fahrner and Maria A. Doyle and Beatriz Serrano-Solano and Anne Claire Fouilloux and Peter van Heusden and Wolfgang Maier and Dave Clements and Florian Heyl and Björn Grüning and B{\'{e}}r{\'{e}}nice Batut and},
	editor = {Francis Ouellette},
	title = {Galaxy Training: A powerful framework for teaching!},
	journal = {PLoS Comput Biol}
}

                   

Funding

These individuals or organisations provided funding support for the development of this resource

SURF

Congratulations on successfully completing this tutorial!

Go Further
Do you want to extend your knowledge? Follow one of our recommended follow-up trainings:

tutorial Hands-on: Pulsar usage on SURF Research Cloud

You can use Ephemeris's shed-tools install command to install the tools used in this tutorial.
shed-tools install [-g GALAXY] [-a API_KEY] -t <(curl https://training.galaxyproject.org/training-material/api/topics/admin/tutorials/surf-research-cloud-galaxy/tutorial.json | jq .admin_install_yaml -r)
Alternatively you can copy and paste the following YAML
---
install_tool_dependencies: true
install_repository_dependencies: true
install_resolver_dependencies: true
tools: []

No feedback has been recieved yet for this training. Be the first one by filling in the feedback form from above.

Want to learn more with a live instructor? Check out these upcoming events:

Date Event Location Contact

January 28, 2025 Galaxy at SURF Research Cloud workshop
Half-day workshop demonstrating the local Galaxy instance built for users of SURF Research Cloud. **Registration**: To sign up for the event, please send an email to [Mirela Minkova](mailto:mirela.minkova@surf.nl?subject=Registering%20SRC%20Galaxy%20Workshop)
SURF Amsterdam, Amsterdam, The Netherlands Mirela Minkova