This tutorial will take you from the multiple AnnData outputs of the previous tutorial to a single, combined AnnData object, ready for all the fun downstream processing. We will also look at how to add in metadata (for instance, SEX or GENOTYPE) for analysis later on.

Details: Where am I?

You are in one of the four tutorials associated with a Case Study, which replicates and expands on the analysis performed in a manuscript Bacon et al. 2018.

Open image in new tab

Agenda

In this tutorial, we will cover:

1. Get Data
2. Important tips for easier analysis
3. Concatenating objects
4. Adding cell metadata
   1. Sex & Genotype metadata
   2. Add batch metadata
5. Conclusion

Get Data

The sample data is a subset of the reads from each of the seven samples in a mouse dataset of fetal growth restriction Bacon et al. 2018 (see the study in Single Cell Expression Atlas and the project submission).

Details: Fun facts about the dataset

We are particularly keen for learners to be able to go from raw FASTQ files all the way through analysis. We aren’t handing you a curated dataset that we specially modified in order for this tutorial to work.

• This tutorial’s input dataset is the full dataset generated from the level previous tutorial in this case study.
• The only difference is that in that previous tutorial, we only analysed one dataset. Our input data, however, is the result of applying that workflow to all seven datasets.
• You can find this dataset in the study in Single Cell Expression Atlas and the project submission.

Hands-on: Choose Your Own Tutorial

This is a "Choose Your Own Tutorial" (CYOT) section (also known as "Choose Your Own Analysis" (CYOA)), where you can select between multiple paths. Click one of the buttons below to select how you want to follow the tutorial

If you’re on the EU or ORG/USA server, then the quickest way to Get the Data for this tutorial is via importing a history. Otherwise, you can also import from Zenodo - it just might take a moment longer if you’re in a live course and everyone is importing the same dataset at the same time!

Import History Zenodo

galaxy-eye Inspect the param-file Experimental Design text file. This shows you how each N70X corresponds to a sample, and whether that sample was from a male or female. This will be important metadata to add to our sample, which we will add very similarly to how you added the gene_name and mito metadata previously!

Important tips for easier analysis

Tip: Using tutorial mode

Tools are frequently updated to new versions. Your Galaxy may have multiple versions of the same tool available. By default, you will be shown the latest version of the tool. This may NOT be the same tool used in the tutorial you are accessing. Furthermore, if you use a newer tool in one step, and try using an older tool in the next step… this may fail! To ensure you use the same tool versions of a given tutorial, use the Tutorial mode feature.

• Open your Galaxy server
• Click on the curriculum icon on the top menu, this will open the GTN inside Galaxy.
• Navigate to your tutorial
• Tool names in tutorials will be blue buttons that open the correct tool for you
• Note: this does not work for all tutorials (yet)
• You can click anywhere in the grey-ed out area outside of the tutorial box to return back to the Galaxy analytical interface

Warning: Not all browsers work!

• We’ve had some issues with Tutorial mode on Safari for Mac users.
• Try a different browser if you aren’t seeing the button.

Tip: Use our Single Cell Omics Lab

Did you know we have a unique Single Cell Omics Lab with all our single cell tools highlighted to make it easier to use on Galaxy? We recommend this site for all your single cell analysis needs, particularly for newer users.

The Single Cell Omics Lab is a different view of the underlying Galaxy server that organises tools and resources better for single-cell users! It also provides a platform for communities to engage and connect; distribute more targeted news and events; and highlight community-specific funding sources.

Try it out!

• subdomain Europe: Single Cell Omics Lab
• subdomain USA: Single Cell Omics Lab
• subdomain Australia: Single Cell Omics Lab

Tip: Troubleshooting errors

When something goes wrong in Galaxy, there are a number of things you can do to find out what it was. Error messages can help you figure out whether it was a problem with one of the settings of the tool, or with the input data, or maybe there is a bug in the tool itself and the problem should be reported. Below are the steps you can follow to troubleshoot your Galaxy errors.

1. Expand the red history dataset by clicking on it.
   • Sometimes you can already see an error message here

2. View the error message by clicking on the bug icon galaxy-bug

3. Check the logs. Output (stdout) and error logs (stderr) of the tool are available:
   • Expand the history item
   • Click on the details icon
   • Scroll down to the Job Information section to view the 2 logs:
     • Tool Standard Output
     • Tool Standard Error
   • For more information about specific tool errors, please see the Troubleshooting section
4. Submit a bug report! If you are still unsure what the problem is.
   • Click on the bug icon galaxy-bug
   • Write down any information you think might help solve the problem
     • See this FAQ on how to write good bug reports
   • Click galaxy-bug Report button
5. Ask for help!
   • Where?
     • In the User community chatspace in Slack in our #single-cell-users channel
     • In the GTN Matrix Channel
     • In the Galaxy Matrix Channel
     • Browse the Galaxy Help Forum to see if others have encountered the same problem before (or post your question).
   • When asking for help, it is useful to share a link to your history

Tip: Using Answer Key Histories

If you get stuck, you can first check your history against an galaxy-history-answer Answer Key history found in the header of (some) tutorials.

First, import the target history.

Tip: Importing a history

1. Open the link to the shared history
2. Click on the Import this history button on the top left
3. Enter a title for the new history
4. Click on Copy History

Next, compare the answer key history with your own history.

Tip: View histories side-by-side

You can view multiple Galaxy histories at once. This allows to better understand your analyses and also makes it possible to drag datasets between histories. This is called “History multiview”. The multiview can be enabled either view History menu or via the Activity Bar:

1. Enabling Multiview via History menu is done by first clicking on the galaxy-history-options “History options” drop-down and selecting galaxy-multihistory “Show Histories Side-by-Side option”:

   

2. Clicking the galaxy-multihistory “History Multiview” button within the Activity Bar:

   

You can compare there, or if you’re really stuck, you can also click and drag a given dataset to your history to continue the tutorial from there.

Tip: Copy a dataset between histories

There 3 ways to copy datasets between histories

1. From the original history

   1. Click on the galaxy-gear icon which is on the top of the list of datasets in the history panel
   2. Click on Copy Datasets

   3. Select the desired files

   4. Give a relevant name to the “New history”

   5. Validate by ‘Copy History Items’
   6. Click on the new history name in the green box that have just appear to switch to this history

2. Using the galaxy-columns Show Histories Side-by-Side

   1. Click on the galaxy-dropdown dropdown arrow top right of the history panel (History options)
   2. Click on galaxy-columns Show Histories Side-by-Side
   3. If your target history is not present
      1. Click on ‘Select histories’
      2. Click on your target history
      3. Validate by ‘Change Selected’
   4. Drag the dataset to copy from its original history
   5. Drop it in the target history

3. From the target history

   1. Click on User in the top bar
   2. Click on Datasets
   3. Search for the dataset to copy
   4. Click on its name
   5. Click on Copy to current History

You can also use our handy troubleshooting guide.

Tip: Troubleshooting errors

When something goes wrong in Galaxy, there are a number of things you can do to find out what it was. Error messages can help you figure out whether it was a problem with one of the settings of the tool, or with the input data, or maybe there is a bug in the tool itself and the problem should be reported. Below are the steps you can follow to troubleshoot your Galaxy errors.

1. Expand the red history dataset by clicking on it.
   • Sometimes you can already see an error message here

2. View the error message by clicking on the bug icon galaxy-bug

3. Check the logs. Output (stdout) and error logs (stderr) of the tool are available:
   • Expand the history item
   • Click on the details icon
   • Scroll down to the Job Information section to view the 2 logs:
     • Tool Standard Output
     • Tool Standard Error
   • For more information about specific tool errors, please see the Troubleshooting section
4. Submit a bug report! If you are still unsure what the problem is.
   • Click on the bug icon galaxy-bug
   • Write down any information you think might help solve the problem
     • See this FAQ on how to write good bug reports
   • Click galaxy-bug Report button
5. Ask for help!
   • Where?
     • In the GTN Matrix Channel
     • In the Galaxy Matrix Channel
     • Browse the Galaxy Help Forum to see if others have encountered the same problem before (or post your question).
   • When asking for help, it is useful to share a link to your history

Concatenating objects

Hands On: Concatenating AnnData objects

1. Manipulate AnnData ( Galaxy version 0.10.9+galaxy1) with the following parameters:
   • param-file “Annotated data matrix”: N701-400k
   • “Function to manipulate the object”: Concatenate along the observations axis
   • param-file “Annotated data matrix to add”: Select all the other matrix files from bottom to top, N702 to N707

   Comment

   If you imported files from Zenodo instead of using the input history, yours might not be in the same order as ours. Since the files will be concatenated in the order that you click, it will be helpful if you click them in the same order, from N702 to N707. This will ensure your samples are given the same batch numbers as we got in this tutorial, which will help when we’re adding in metadata later!

   Warning: Don't add N701!

   You are adding files to N701, so do not add N701 to itself!

   • “Join method”: Intersection of variables
   • “Key to add the batch annotation to obs”: batch
   • “Separator to join the existing index names with the batch category”: -
2. Rename galaxy-pencil output Combined_Object

Now let’s look at what we’ve done! You can peek at the AnnData object in your galaxy-history history. You can also inspect the dataset for further details using a tool.

Hands On: Inspecting AnnData Objects

1. Inspect AnnData ( Galaxy version 0.10.9+galaxy1) with the following parameters:
   • param-file “Annotated data matrix”: Combined_object
   • “What to inspect?”: General information about the object
2. Inspect AnnData ( Galaxy version 0.10.9+galaxy1) with the following parameters:
   • param-file “Annotated data matrix”: Combined_object
   • “What to inspect?”: Key-indexed observations annotation (obs)
3. Inspect AnnData ( Galaxy version 0.10.9+galaxy1) with the following parameters:
   • param-file “Annotated data matrix”: Combined_object
   • “What to inspect?”: Key-indexed annotation of variables/features (var)

Now have a look at the three tool Inspect AnnData outputs.

Question

1. How many cells do you have now?
2. Where is batch information stored?

Solution

1. If you peek at your dataset, or look at the General information tool output, you will find there are 316 cells, as the matrix is now 316 cells (n_obs) x 35734 genes (n_var). You will also find these numbers in the obs tool (cells) and var tool (genes) file sizes.
2. Batch information is stored under Key-indexed observations annotation (obs). Different versions of the Manipulate tool might put the batch columns in different locations. The tool version in this tutorial puts batch in the 8th column. Batch refers to the order in which the matrices were added. The files are added from the bottom of the history upwards, so be careful how you set up your histories when running this (i.e. if your first dataset is N703 and the second is N701, the batch will call N703 0 and N701 1!)

Adding cell metadata

I set up the example history with the earliest indices at the bottom.

Open image in new tab

{ width=400px }

Therefore, when it is all concatenated together, the batch appears as follows:

If you used Zenodo to import files, they may not have imported in order (i.e. N701 to N707, ascending). In that case, you will need to tweak the parameters of the next tools appropriately to label your batches correctly!

The two critical pieces of metadata in this experiment are sex and genotype. I will later want to color my cell plots by these parameters, so I want to add them in now!

Sex & Genotype metadata

Hands On: Labelling sex

1. Replace Text ( Galaxy version 9.5+galaxy0) with the following parameters:
   • param-file “File to process”: output of Inspect AnnData: Key-indexed observations annotation (obs) tool)

   • “1. Replacement”

     • “in column”: Column: 8 - or whichever column batch is in
     • “Find pattern”: 0|1|3|4|5|6
     • “Replace with”: male
   • + Insert Replacement

   • “2. Replacement”

     • “in column”: Column: 8
     • “Find pattern”: 2
     • “Replace with”: female
   • + Insert Replacement

   • “3. Replacement”

     • “in column”: Column: 8
     • “Find pattern”: batch
     • “Replace with”: sex

   The output of the param-file Replace Text tool tool has many columns. However, we want only the column containing the sex information, in order by cell barcode, to add to our AnnData later.

2. Cut columns from a table with the following parameters:
   • “Cut columns”: c8
   • “Delimited by”: Tab
   • param-file “From”: output of Replace text tool
3. Rename galaxy-pencil output Sex_metadata

That was so fun, let’s do it all again but for genotype!

Hands On: Labelling genotype

1. Replace Text in a specific column ( Galaxy version 9.5+galaxy0) with the following parameters:
   • param-file “File to process”: output of Inspect AnnData: Key-indexed observations annotation (obs) tool

   • “1. Replacement”

     • “in column”: Column: 8
     • “Find pattern”: 0|3|4|5
     • “Replace with”: wildtype
   • + Insert Replacement

   • “2. Replacement”

     • “in column”: Column: 8
     • “Find pattern”: 1|2|6
     • “Replace with”: knockout
   • + Insert Replacement

   • “3. Replacement”

     • “in column”: Column: 8
     • “Find pattern”: batch
     • “Replace with”: genotype

   Now we want only the column containing the genotype information - we will ultimately add this into the cell annotation in the AnnData object.

2. Cut columns from a table with the following parameters:
   • “Cut columns”: c8
   • “Delimited by”: Tab
   • param-file “From”: output of Replace text tool
3. Rename galaxy-pencil output Genotype_metadata

You might want to do this with all sorts of different metadata - which labs handled the samples, which days they were run, etc. Once you’ve created and cut all your metadata columns, we can paste them together before adding them into the AnnData object itself.

Hands On: Combining metadata columns

1. Paste two files side by side with the following parameters:
   • param-file “Paste”: Genotype_metadata
   • param-file “and”: Sex_metadata
   • “Delimit by”: Tab
2. Rename galaxy-pencil output Cell_Metadata

Let’s add this metadata to the AnnData object!

Hands On: Adding metadata to AnnData object

1. Manipulate AnnData ( Galaxy version 0.10.9+galaxy1) with the following parameters:
   • param-file “Annotated data matrix”: Combined_object
   • “Function to manipulate the object”: Add new annotation(s) for observations or variables
   • “What to annotate?”: Observations (obs)`
   • param-file “Table with new annotations”: Cell_Metadata

Woohoo! We’re there! You can peek at your new AnnData object in your history galaxy-history to see the additional Obs categories to make sure this worked. You should now find a sex and genotype in the Obs listing.

Add batch metadata

I want to clean up this AnnData object just a bit more first. It would be a lot nicer if ‘batch’ meant something, rather than ‘the order in which the Manipulate AnnData tool added my datasets’.

Hands On: Labelling batches

1. Manipulate AnnData ( Galaxy version 0.10.9+galaxy1) with the following parameters:
   • param-file “Annotated data matrix”: output of Manipulate AnnData - Add new annotations tool
   • “Function to manipulate the object”: Rename categories of annotation
   • “Key for observations or variables annotation”: batch
   • “Comma-separated list of new categories”: N701,N702,N703,N704,N705,N706,N707
2. Rename galaxy-pencil output Batched_Object

congratulations Well done! Peek at your final param-file Batched_Object to see the wealth of information that has been added. You are now ready to move along to further filtering!

Conclusion

You might find the galaxy-history-answer Answer Key Histories helpful to check or compare with:

• UseGalaxy.eu

  UseGalaxy.eu-ARCHIVE2

  UseGalaxy.eu-ARCHIVE1

  All total samples - processed after Alevin into single object (UseGalaxy.eu)

You can also run this entire tutorial via a galaxy-workflows-activity Workflow, after performing the Get data step initially.

• Tutorial Workflow

Finally, you may remember that the datasets you analysed in this tutorial were downsampled. The idential analysis can be performed on whole samples, which you can find in the example histories below.

• warning Remember that if you are in a course, time for exploring these histories will not be factored into the schedule. Explore these outside of course time!

Tip: How can I talk with other users?

feedback To discuss with like-minded scientists, join our Galaxy Training Network chatspace in Slack and discuss with fellow users of Galaxy single cell analysis tools on #single-cell-users

We also post new tutorials / workflows there from time to time, as well as any other news.

point-right If you’d like to contribute ideas, requests or feedback as part of the wider community building single-cell and spatial resources within Galaxy, you can also join our Single cell & sPatial Omics Community of Practice.

tool You can request tools here on our Single Cell and Spatial Omics Community Tool Request Spreadsheet

You've Finished the Tutorial

Please also consider filling out the Feedback Form as well!

Key points

• Create a single scanpy-accessible AnnData object from multiple AnnData files, including relevant cell metadata according to the study design

• Retreive partially analysed data from a public repository

Frequently Asked Questions

• Tutorial FAQs
• Topic FAQs
• Galaxy FAQs
• GTN Matrix Chat
• Galaxy Help Forum

Useful literature

Further information, including links to documentation and original publications, regarding the tools, analysis techniques and the interpretation of results described in this tutorial can be found here.

References

1. Bacon, W. A., R. S. Hamilton, Z. Yu, J. Kieckbusch, D. Hawkes et al., 2018 Single-Cell Analysis Identifies Thymic Maturation Delay in Growth-Restricted Neonatal Mice. Frontiers in Immunology 9: 10.3389/fimmu.2018.02523

Feedback

Did you use this material as an instructor? Feel free to give us feedback on how it went.
Did you use this material as a learner or student? Click the form below to leave feedback.

Citing this Tutorial

1. Wendi Bacon, Combining single cell datasets after pre-processing (Galaxy Training Materials). https://training.galaxyproject.org/training-material/topics/single-cell/tutorials/scrna-case_alevin-combine-datasets/tutorial.html Online; accessed TODAY
2. Hiltemann, Saskia, Rasche, Helena et al., 2023 Galaxy Training: A Powerful Framework for Teaching! PLOS Computational Biology 10.1371/journal.pcbi.1010752
3. Batut et al., 2018 Community-Driven Data Analysis Training for Biology Cell Systems 10.1016/j.cels.2018.05.012

BibTeX

@misc{single-cell-scrna-case_alevin-combine-datasets,
author = "Wendi Bacon",
	title = "Combining single cell datasets after pre-processing (Galaxy Training Materials)",
	year = "",
	month = "",
	day = "",
	url = "\url{https://training.galaxyproject.org/training-material/topics/single-cell/tutorials/scrna-case_alevin-combine-datasets/tutorial.html}",
	note = "[Online; accessed TODAY]"
}
@article{Hiltemann_2023,
	doi = {10.1371/journal.pcbi.1010752},
	url = {https://doi.org/10.1371%2Fjournal.pcbi.1010752},
	year = 2023,
	month = {jan},
	publisher = {Public Library of Science ({PLoS})},
	volume = {19},
	number = {1},
	pages = {e1010752},
	author = {Saskia Hiltemann and Helena Rasche and Simon Gladman and Hans-Rudolf Hotz and Delphine Larivi{\`{e}}re and Daniel Blankenberg and Pratik D. Jagtap and Thomas Wollmann and Anthony Bretaudeau and Nadia Gou{\'{e}} and Timothy J. Griffin and Coline Royaux and Yvan Le Bras and Subina Mehta and Anna Syme and Frederik Coppens and Bert Droesbeke and Nicola Soranzo and Wendi Bacon and Fotis Psomopoulos and Crist{\'{o}}bal Gallardo-Alba and John Davis and Melanie Christine Föll and Matthias Fahrner and Maria A. Doyle and Beatriz Serrano-Solano and Anne Claire Fouilloux and Peter van Heusden and Wolfgang Maier and Dave Clements and Florian Heyl and Björn Grüning and B{\'{e}}r{\'{e}}nice Batut and},
	editor = {Francis Ouellette},
	title = {Galaxy Training: A powerful framework for teaching!},
	journal = {PLoS Comput Biol}
}

                   

Do you want to extend your knowledge?

Follow one of our recommended follow-up trainings:

• tutorial Hands-on: Filter, plot and explore single-cell RNA-seq data with Scanpy

Galaxy Administrators: Install the missing tools

You can use Ephemeris's shed-tools install command to install the tools used in this tutorial.

shed-tools install [-g GALAXY] [-a API_KEY] -t <(curl https://training.galaxyproject.org/training-material/api/topics/single-cell/tutorials/scrna-case_alevin-combine-datasets/tutorial.json | jq .admin_install_yaml -r)

Alternatively you can copy and paste the following YAML

---
install_tool_dependencies: true
install_repository_dependencies: true
install_resolver_dependencies: true
tools:
- name: text_processing
  owner: bgruening
  revisions: 12615d397df7
  tool_panel_section_label: Text Manipulation
  tool_shed_url: https://toolshed.g2.bx.psu.edu/
- name: text_processing
  owner: bgruening
  revisions: 6073bb457ec0
  tool_panel_section_label: Text Manipulation
  tool_shed_url: https://toolshed.g2.bx.psu.edu/
- name: anndata_ops
  owner: ebi-gxa
  revisions: a4774b7b2e85
  tool_panel_section_label: Single-cell
  tool_shed_url: https://toolshed.g2.bx.psu.edu/
- name: anndata_inspect
  owner: iuc
  revisions: 6f0d0c784f09
  tool_panel_section_label: Single-cell
  tool_shed_url: https://toolshed.g2.bx.psu.edu/
- name: anndata_inspect
  owner: iuc
  revisions: afe044e6bc93
  tool_panel_section_label: Single-cell
  tool_shed_url: https://toolshed.g2.bx.psu.edu/
- name: anndata_manipulate
  owner: iuc
  revisions: ed4996a16f7f
  tool_panel_section_label: Single-cell
  tool_shed_url: https://toolshed.g2.bx.psu.edu/
- name: anndata_manipulate
  owner: iuc
  revisions: be1fdce914c6
  tool_panel_section_label: Single-cell
  tool_shed_url: https://toolshed.g2.bx.psu.edu/

Feedback

t{ hist[0] | to_stars }}	1

September 2024

• 4 stars: Liked: Well-described and easy to follow