Galaxy Monitoring with Reports
Nate Coraor
Björn Grüning
Simon Gladman
Helena RascheOverviewQuestions:
Objectives:
How to monitor a Galaxy service with the Reports application?
Requirements:
Setup and start the Galaxy reports app.
- slides Slides: Ansible
- tutorial Hands-on: Ansible
- slides Slides: Galaxy Installation with Ansible
- tutorial Hands-on: Galaxy Installation with Ansible
Time estimation: 30 minutesSupporting Materials:Published: Jan 31, 2019Last modification: Jun 25, 2024License: Tutorial Content is licensed under Creative Commons Attribution 4.0 International License. The GTN Framework is licensed under MITpurl PURL: https://gxy.io/GTN:T00018rating Rating: 5.0 (0 recent ratings, 3 all time)version Revision: 41
The reports application gives some pre-configured analytics screens. These are very easy to setup and can help with debugging issues in Galaxy.
Warning: Currently Broken, Requires Separate DomainReports does not work, under a path prefix (the default setup that most people will use.) It is completely broken and the developers have no plans to fix it in the near term. See galaxyproject/galaxy#15966 for more details.
However, it should still function with a separate domain, if that is possible for your setup. Otherwise, it will not work. If you wish to follow this tutorial, please be aware of this.
Agenda
Comment: Galaxy Admin Training PathThe yearly Galaxy Admin Training follows a specific ordering of tutorials. Use this timeline to help keep track of where you are in Galaxy Admin Training.
Setting up Reports
The reports application is included with the Galaxy codebase and this tutorial assumes you’ve already done all of the setup required for Galaxy, systemd, uWSGI, and NGINX.
Hands-on: Setup Reports
First we add a basic configuration of the Reports app to the playbook templates. Create
templates/galaxy/config/folder, if it doesn’t exist, and createtemplates/galaxy/config/reports.ymlwith the following contents:--- /dev/null +++ b/templates/galaxy/config/reports.yml @@ -0,0 +1,4 @@ +reports: + database_connection: "{{ galaxy_config.galaxy.database_connection }}" + file_path: "{{ galaxy_config.galaxy.file_path }}" + template_cache_path: "{{ galaxy_mutable_data_dir }}/compiled_templates/reports/"If you haven’t worked with diffs before, this can be something quite new or different.
If we have two files, let’s say a grocery list, in two files. We’ll call them ‘a’ and ‘b’.
Input: Old$ cat old
🍎
🍐
🍊
🍋
🍒
🥑Output: New$ cat new
🍎
🍐
🍊
🍋
🍍
🥑We can see that they have some different entries. We’ve removed 🍒 because they’re awful, and replaced them with an 🍍
Diff lets us compare these files
$ diff old new
5c5
< 🍒
---
> 🍍Here we see that 🍒 is only in a, and 🍍 is only in b. But otherwise the files are identical.
There are a couple different formats to diffs, one is the ‘unified diff’
$ diff -U2 old new
--- old 2022-02-16 14:06:19.697132568 +0100
+++ new 2022-02-16 14:06:36.340962616 +0100
@@ -3,4 +3,4 @@
🍊
🍋
-🍒
+🍍
🥑This is basically what you see in the training materials which gives you a lot of context about the changes:
--- oldis the ‘old’ file in our view+++ newis the ‘new’ file- @@ these lines tell us where the change occurs and how many lines are added or removed.
- Lines starting with a - are removed from our ‘new’ file
- Lines with a + have been added.
So when you go to apply these diffs to your files in the training:
- Ignore the header
- Remove lines starting with - from your file
- Add lines starting with + to your file
The other lines (🍊/🍋 and 🥑) above just provide “context”, they help you know where a change belongs in a file, but should not be edited when you’re making the above change. Given the above diff, you would find a line with a 🍒, and replace it with a 🍍
Added & Removed Lines
Removals are very easy to spot, we just have removed lines
--- old 2022-02-16 14:06:19.697132568 +0100
+++ new 2022-02-16 14:10:14.370722802 +0100
@@ -4,3 +4,2 @@
🍋
🍒
-🥑And additions likewise are very easy, just add a new line, between the other lines in your file.
--- old 2022-02-16 14:06:19.697132568 +0100
+++ new 2022-02-16 14:11:11.422135393 +0100
@@ -1,3 +1,4 @@
🍎
+🍍
🍐
🍊Completely new files
Completely new files look a bit different, there the “old” file is
/dev/null, the empty file in a Linux machine.$ diff -U2 /dev/null old
--- /dev/null 2022-02-15 11:47:16.100000270 +0100
+++ old 2022-02-16 14:06:19.697132568 +0100
@@ -0,0 +1,6 @@
+🍎
+🍐
+🍊
+🍋
+🍒
+🥑And removed files are similar, except with the new file being /dev/null
--- old 2022-02-16 14:06:19.697132568 +0100
+++ /dev/null 2022-02-15 11:47:16.100000270 +0100
@@ -1,6 +0,0 @@
-🍎
-🍐
-🍊
-🍋
-🍒
-🥑In your
galaxyserversgroup variables file, tell the playbook to deploy the reports configuration file, and gravity to manage reports:--- a/group_vars/galaxyservers.yml +++ b/group_vars/galaxyservers.yml @@ -151,6 +151,11 @@ galaxy_config: pools: - job-handlers - workflow-schedulers + reports: + enable: true + url_prefix: /reports + bind: "unix:{{ galaxy_mutable_config_dir }}/reports.sock" + config_file: "{{ galaxy_config_dir }}/reports.yml" galaxy_job_config_file: "{{ galaxy_config_dir }}/galaxy.yml" @@ -171,6 +176,8 @@ galaxy_config_templates: dest: "{{ galaxy_config.galaxy.dependency_resolvers_config_file }}" - src: templates/galaxy/config/job_resource_params_conf.xml.j2 dest: "{{ galaxy_config.galaxy.job_resource_params_file }}" + - src: templates/galaxy/config/reports.yml + dest: "{{ galaxy_config.gravity.reports.config_file }}" galaxy_extra_dirs: - /dataThen we need to tell NGINX it should serve our Reports app under
<server_url>/reportsurl. Edit yourtemplates/nginx/galaxy.j2file, and within the server block, add a block for proxying the reports application. It should look like:--- a/templates/nginx/galaxy.j2 +++ b/templates/nginx/galaxy.j2 @@ -103,4 +103,9 @@ server { proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; } + + location /reports/ { + proxy_pass http://{{ galaxy_config.gravity.reports.bind }}:/; + } + }Run the playbook:
Input: Bashansible-playbook galaxy.ymlThe reports application should be available, under
/reports
1.sh
Comment: Insecure!But notice that your Reports server is not secured! Check out the External Authentication tutorial for information on securing Reports.
Hands-on: Time to git commitIt’s time to commit your work! Check the status with
git statusAdd your changed files with
git add ... # any files you see that are changedAnd then commit it!
git commit -m 'Finished Galaxy Monitoring with Reports'
Comment: Got lost along the way?If you missed any steps, you can compare against the reference files, or see what changed since the previous tutorial.
If you’re using
gitto track your progress, remember to add your changes and commit with a good commit message!
Comment: Galaxy Admin Training PathThe yearly Galaxy Admin Training follows a specific ordering of tutorials. Use this timeline to help keep track of where you are in Galaxy Admin Training.