Start Your ARC Workshop Series

Session 03 - Annotate your data
June 14th, 2023


Dominik Brilhaus - CEPLAS Data Science

Goals

  • Get familiar with ISA metadata and Swate
  • Annotate data in your ARC

Legend

💻 = Locally (on your machine)

🌐 = Remote (in the DataHUB)

Info in <brackets> are placeholders - please replace with proper info

Part 0: Check-in

Open the online notepad

https://pad.hhu.de/u44vMavDQLa2e4hvjx1Oyg

Screen-sharing during the workshop

https://hhu.webex.com/hhu-en/j.php?MTID=me5501c7ea1607b9ee3e1edfe401598f3

Check Swate installation

☑️ Make sure Swate is installed:

  1. Open Excel (online or Desktop)
  2. Go to the Insert tab: Click the arrow next to "My Add-ins". There you should be able to select Swate.
  3. Go to the Data tab: you should see the Swate (Core) add-in.

💡 Alternatively, you can use Swate standalone
(⚠️ this is however work in progress and likely to change)

Have a simple text editor ready

  • Windows Notepad
  • MacOS TextEdit

Recommended text editors with code highlighting:

Download the demo data

git clone "https://demo-user:5ehDYeHcqP2MqVXsNNPu@git.nfdi4plants.org/teaching/demo-arc_level1.git"

Where we left off last time

👩‍💻 Initiated an ARC
📂 Structured and ...
🌐 Shared with collaborators


Today we want to

... annotate the experimental data

Part 1: Intro ISA

ARC builds on ISA

https://isa-tools.org/format/specification.html

ARC builds on ISA

isa.<>.xlsx files within ARCs

Study and assay files are registered in the investigation file

The output of a study or assay file can function as input for a new isa.assay.xlsx

Output building blocks:

  • Sample Name
  • Raw Data File
  • Derived Data File

Annotation Building Block types

  • Source Name (Input)
  • Protocol Columns
    • Protocol Type, Protocol Ref
  • Characteristic
  • Parameter
  • Factor
  • Component
  • Output Columns
    • Sample Name, Raw Data File, Derived Data File

💡 For details, check out the Annotation Principles.

Part 2: Swate hands-on with demo data

Swate Overview

Major areas of the Swate user interface.

Let's annotate the plant samples first

  1. Navigate to the demo ARC.
  2. Open the lab notes studies/talinum_drought/protocols/plant_material.txt in a text editor.
  3. Open the empty studies/talinum_drought/isa.study.xlsx workbook in Excel.

Create an annotation table


Create a Swate annotation table via the create annotation table button in the yellow pop-up box OR click the Create Annotation Table quick access button.


💡 Each table is by default created with one input (Source Name) and one output (Sample Name) column

💡 Only one annotation table can be added per Excel sheet

Add a building block

  1. Navigate to the Building Blocks tab via the navbar. Here you can add Building Blocks to the table.
  2. Instead of Parameter select Characteristic from the drop-down menu (A)
  3. Search for organism in the search bar (B). This search looks for suitable Terms in our Ontology database.
  4. Select the Term with the id OBI:0100026 and,
  5. Click Add building block.

💡 This adds three columns to your table, one visible and two hidden.

Insert values to annotate your data

  1. Navigate to the Terms tab in the Navbar
  2. In the annotation table, select any number of cells below Characteristic [organism]
  3. Click into the search field in Swate.

💡 You should see organism showing in a field in front of the search field
💡 The search will now yield results related to organism

  1. In the search field, search for "Talinum fruticosum"
  2. Select the first hit and click Fill selected cells with this term

Add a building block with a unit

  1. In the Building Blocks tab, select Parameter, search for light intensity exposure and select the term with id PECO:0007224.
  2. Check the box for This Parameter has a unit and search for microeinstein per square meter per second in the adjacent search bar.
  3. Select UO:0000160.
  4. Click Add building block.

💡 This adds four columns to your table, one visible and three hidden.

Insert unit-values to annotate your data

In the annotation table, select any cell below Parameter [light intensity exposure] and add "425" as light intensity.

💡 You can see the numbers being complemented with the chosen unit, e.g. 425.00 microeinstein per square meter per second

Showing ontology reference columns

Hold Ctrl and click the Autoformat Table quick access button to adjust column widths and un-hide all hidden columns.

💡 You can see that your organism of choice was added with id and source Ontology in the reference (hidden) columns.
⚠️ This feature is currently not supported on MacOS

Update ontology reference columns

Click the Update Ontology Terms quick access buttons.

💡 This updates all reference columns according to the main column. In this case the reference columns for Parameter [light intensity exposure] are updated with the id and source ontology of the microeinstein per square meter per second unit.

Your ISA table is growing

At this point. Your table should look similar to this:

Hiding ontology reference columns

Click the Autoformat Table quick access button without holding Ctrl to hide all reference columns.

Exercise 📝

Try to add suitable building blocks for other pieces of metadata from the plant growth protocol (studies/talinum_drought/protocols/plant_material.txt).

Let's annotate the RNA Seq data

  1. Navigate to the demo ARC.
  2. Open the lab notes assays/rnaseq/protocols/RNA_extraction.txt in a text editor.
  3. Open the empty assays/rnaseq/isa.assay.xlsx) workbook in Excel.

Use a template

  1. Navigate to Templates in the Navbar and click Browse database in the first function block.

💡 Here you can find community created workflow annotation templates

  1. Search for RNA extraction and click select
    • You will see a preview of all building blocks which are part of this template.
  2. Click Add template to add all Building Blocks from the template to your table, which do not exist yet.

Adding / Updating unit references

Sometimes you need to add or update the unit of an existing building block.

  1. Select any number of rows of the Parameter [biosource amount] building block to mark it for the next steps.
  2. Open the Building Blocks tab
  3. In the bottom panel "Add/Update unit reference to existing building block", search for the unit "milligram". Select the unit term and click Update unit for cells.
    💡 If you already had values in the main column they will be updated automatically.
  4. Click the Update Ontology Terms quick access button, to update the reference columns.

Remove building blocks

If there are any Building Blocks which do not fit your experiment you can use the Remove Building Block quick access button to remove it including all related (hidden) reference columns.

⚠️ Due to the hidden reference columns, we recommend not to delete table columns via usual Excel functions.

New process, new worksheet

  1. Add a new sheet to the assays/rnaseq/isa.assay.xlsx) workbook.
  2. Add the template "RNASeq Assay"

Exercise 📝

Try to fill the two sheets with the protocol details:

  • assays/rnaseq/protocols/RNA_extraction.txt and
  • assays/rnaseq/protocols/Illumina_libraries.txt

Your ISA table is ready 🎉

Go ahead, adjust the Building Blocks you want to use to describe your experiment as you see fit.
Insert values using Swate Term search and add input and output.

A small detour on "Excel Tables"

Swate uses Excel's "table" feature to annotate workflows. Each table represents one process from input (e.g. plant leaf material) to output (e.g. leaf extract).

Example workflows with three processes each:

  • Plant growth → sampling → extraction
  • Measured data files → statistical analysis → result files

💡 Excel tables allow to group data that belongs together inside one sheet. This is not to be confused with a (work)sheet or workbook.

workbook              (e.g. "isa.assay.xlsx")
 └─── worksheet       (e.g. "plant_growth")
          └─── table  (e.g. "annotationTable")

Contributors

Slides presented here include contributions by

Title slide to class

Source to slide(s)

../../bricks/ARC_ISAmodel.md

Source to slide(s)

../../bricks/ARC_ISA-tab.md

Source to slide(s)

../../bricks/ARC_ISAxlsx01.md

Source to slide(s)

../../bricks/ARC_ISAxlsx02.md

Source to slide(s)

../../bricks/ARC_ISAxlsx03.md

Source to slide(s)

../../bricks/ARC_ISAxlsx04.md

Source to slide(s)

../../bricks/lesson_Swate_DataPLANT-Annotation_Building_Block_types.md