Annotation editor

Document area

The text is displayed in the document area. There you can read and annotate text.

Text annotations

Once a piece of text is annotated, it becomes an entity. In tagtog you can operate with entities and do things as normalize them, relate them, etc.

The background color of each annotation depends on the color picked for the Entity Type. The font color changes based on the background color so the contrast is appropriate to read.

In green the gene names, in red the mutations. Font color change depending on the entity background color

Create new text annotations

A new text annotation is created by highlighting text with the mouse. Position the cursor at the beginning of the text you want to highlight. Press and hold your primary mouse button (commonly the left-button). While holding the mouse button, drag the cursor to the end of the text and let go of the mouse button. Once completed, all the text from the beginning to the end should be highlighted using the same Entity Type used in the previous text annotation. Currently the only way to change the entity type used for new annotations is by first changing the entity type of existing annotation.

Tips & tricks:

• If you double-click, you annotate the word clicked.
• If you try to annotate a word that starts or ends in space, the space won't be annotated.
• Press and hold the ctrl or command key to annotate a whole paragraph. Simply hold the key and click on the paragraph to annotate it.

Overlapping text annotations

Just create a new annotation that is contained within the span of existing one or that only overlaps part of it. Overlapping text annotations are recognizable at a glance while not disturbing you from reading the text.

Example of contained annotation, the car make Toyota is contained in the model Toyota Corolla

Three entities annotated, two annotations are overlapping

Sample of customer feedback. Two annotations (first in pink, second in yellow) within the same span representing a vehicle part and the failing part.

Pre-selections

Automatic annotations created upon the manual creation or removal of other equal annotation (same entity type and same text). These type of annotations increase annotator's efficiency as potential candidates for new/to-remove annotations are automatically identified.

Pre-selections also work in the PDF annotation tool.

Type	Description
Pre-selection	Equal entities that are annotated upon manual annotation. E.g. if you annotate HER2 as Entity Type Gene, all occurrences of the string "HER2" will be annotated as Entity Type Gene. Pre-selections are visualized with a yellow border and the background color of the Entity Type. If you click on one of these pre-annotations, the pre-annotation will turn into a regular annotation. In addition, if you add/remove a normalization or entity label for this entity, this change will propagate to all pre-annotated occurrences.
Pre-deselection	Equal entities that are removed upon manual removal, e.g. if you remove an existing annotation with the text "HER2" and Entity Type Gene, all annotations with the text "HER2" with the same Entity Type will be pre-deselected. Pre-deselections are visualized with a yellow border and white background color. If you click on one of these pre-deselections, the annotation will be removed.

You can choose whether pre-selections are case sensitive or not. As other properties from pre-selections, this setting can be changed both directly in the document toolbar and/or at the project level.

Annotation Menu

By clicking on the primary mouse button (commonly the left-button) on a text annotation, you display the annotation menu.

These are the actions you can perform:

Toolbar

The toolbar is located on the top of the document area. From it you can perform these actions:

Original source

In case the document or text comes from a known provider, clicking this link you access the original source.

For example, if you upload a PubMed document by PubMedId (PMID), tagtog understands the source. Clicking on this button you will go to the article in Pubmed.

Annotations from other users

Each document has a master version and one separate annotation version per project member.

Click on the user list to show the different versions available. Click on any of these annotation versions to load it in the document editor area.

Depending on your permissions you can/cannot edit the different annotation versions. A locker icon indicates that your permissions for that version are read-only.

For each version listed, there is a checkmark indicating if that version is confirmed (green) or not. If you hover over the username, the role is displayed.

If Task Distribution is active, the list of versions is split into two groups: members assigned to the document (this document was distributed to this member), and members not assigned to the document (“Other”). If Task Distribution is not active, all suitable members’ versions are listed.

More information on multi-user annotation

Manage annotation versions

You might want to start from the annotations of other user or replace the master version with the annotations of a specific user. If different users have annotated the same documents, tagtog also supports an automatic adjudication process to compose the final version of the annotations based on the agreement among users. For such cases, you can use the options available in the toolbar.

If you click on that option, a list of actions shows up:

The availability of these options depends on the role permissions. More information on multi-user annotation

Pre-selections

Here you can turn on/off pre-selections or pre-deselections. You can also turn on/off case sensitivity.

Each time you load a new document, the default settings from your project will apply. The changes in this menu won’t change the default values. That is, the changes here only affect the current document.

Save a document

Each time a change is made in the document (e.g. new annotation or relation added), the Save button will turn into green to indicate there are changes to save. Click the button to save the changes.

Confirm a document

Usually users confirm the document once the annotations has been reviewed. This is used to indicate that this document can be used as training data for AI, or simply that all annotations has been reviewed by a human. There are different annotation flows you can use for your project.

To confirm a document click on the button with the icon

Once you have confirmed a document, many actions are disabled. You can undo the Confirm action by clicking again the button. It is a toggle button.

Download / View / Export

Here you can select which way you want to display or export the annotated document.

Annotated documents can be exported into various formats: output formats.

The tagtog Editor refers to the visualization of the annotated document in the annotation editor.

Remove annotations

Click on the button with the icon and select the option Remove annotationsto delete all the annotations in the current document. This won't remove the document.

Remove document

Click on the button with the icon and select the option Remove documentto remove the document from the folder.

Document navigator

Each button with an arrow pointing to left and right. If you click on the button with the left arrow, the previous document in the folder will be loaded. If you click on the button with the right arrow, the next document in the folder will be loaded.

Under the document navigator, you can find the document index. This will give you an idea of where you are and how much work is left.

Page navigator

If the document is paginated, the toolbar has a controller to navigate across the pages. For example, in PDF documents. More information.

Sidebar

The sidebar appearance changes depending on how you configured your project. It will only display those actionable items for those entity types, entity labels, document labels or relations used in the project.

You can show/hide the sidebar by clicking on the arrow near the top of the sidebar.

These are the components you can find in the sidebar:

Document labels

If you have any document label configured at Settings > Document Labels they will appear in this section in the side bar. Here the user can define the value of a document label for the current document. Once a change is made, you can save the document as usual.

The list of document labels is sorted alphabetically.

Clicking on the icon you reset the label to the default value ?

Entity tally

The entity tally displays a summary for the entities annotated in your current document.

On the top of this section you find some statistics with the number entities annotated and the entities not normalized. E.g. . Below the header, you can find a summary for the entities annotated in the current document. For example:

Entities are classified under Entity types. For each type some statistics are displayed: number of entities, manual annotated entities, automatic annotated entities, normalized entities

If you click on an entity, the entity is highlighted in the text and the Document Review starts with this entity. If you click again on the entity, you get out the Document Review mode

To quickly digest the status of the annotated entities, you can:

Group entities

Filter entities

Group entities

Classify entities in different groups.

Filter entities

You can display entities that meet certain criteria.

You decide whether you want to apply the filter only to the entity tally or extend the scope to the document. To control this functionality, there is an Apply to document switch available:

Off: show the resulting entities only in the entity tally. Only those entities meeting the filter criteria are listed.

On: show the resulting entities in the entity tally, and in the document. In the entity tally, only those entities meeting the filter criteria are listed. In the document, these entities are visible, the rest are hidden (visually almost transparent).

Example of a filter applied to a document. Notice that the entities filtered out are hidden (visually almost transparent). In this example, the document is styled using tagtog blocks.

You can filter entities by the following properties:

To reset the filter to the default values, there are two options:

Reset each filter property independently: click on a icon near the property.

Reset all filter properties: if the filter is active, an option to reset the whole filter shows up on the top of the filter.

Relation tally

It keeps the count of the relations defined in the current document. In this section you can remove existing relations, clicking on the button .

This tally only appears if you have relation types defined at Settings > Relations.

Document Review

If you want to review a set of entities from your document, you can use the Document Review mode:

1Using the filter, select the entities you want to review. For example: review all the entities under 0.7 probability threshold.

2Press the hotkey r to start the Document Review from the first entity. Alternatively, you can click on an entity in the Entity tally to start from that entity. Use ↑ and ↓ to navigate across the matching entities. For each entity, using hotkeys, you can perform the following actions:

Click on an entity in the text or press ESC to exit the Document Review mode.

Document Review. In this example, we first filter the entities by text, and next, we review the resulting items performing some actions with the keyboard (change entity labels, creating a relationship, and removing the last entity)

Folders

Folders can be accessed from the document pool or the web editor. You can find the folders panel of the left side of the editor. More information about folders.