Task 4 - Hand Annotation

"Covers Multiple Objects"

We use "Covers Multiple Objects" when:

1. There are instances left after masking 25 individual instances.

2. When it is not possible to cover only one occurrence of an object

Multiple Objects

A mask should only cover one object at a time (same as Task 1).

E.g. The mask for the label "people" should contain only one person

Collections

Each instance of a collection is considered one object and is to be masked separately (same as Task 1).

Number of Objects

For a label that has a specific number of objects, the mask should contain the exact number of objects stated.

E.g. The mask for the label "3 cars" should contain 3 cars.

Masking too time-intensive

Reject a job using “Masking too time-intensive” when:

1. It is not possible to cover all with instance masks and masks covering groups.

2. It is too difficult to generate each mask:  if a mask takes more than 3 minutes to annotate (especially consistently across instances in the image), it should be rejected.

Object Only Refers to Text

We reject using this reason if the phrase refers to text in the image.

Examples of labels/phrases to reject with “Object only refers to text”

Reject using the “Object only refers to text” if you see one of these labels:

• Numbers

• Text

• Letter

• Lettering

• Writing

• Font

• Serif

Flag Object Phrase

Reject using this reason if you come across a label that should be flagged (same as Task 1)

Examples of labels/phrases to reject with “Flag object phrase”

Reject if the label/phrase is either one or the following:

• PII

• Ethnicity/Race/Nationality

• Socio-economic status

• Medical Condition/Disabilities

• Derogatory terms

• Facial Features

This remains the same as Task 1

What is an “Accept” mask?

An “accept” mask is a mask that comes pre-drawn with a job.

When do we modify an “Accept” mask?

An “Accept” mask should be modified if:

1. The mask needs to be fixed (imperfections, large holes, covering more than one instance, etc.)

2. The mask is incorrect (covering the wrong object)

Using the SAM tool

To use the SAM tool;

1. Click the ⚡symbol on the annotation pane

2. Draw a box around the object to be masked.

In case the mask drawn is not of good quality,

1. Right-click to add a mask

2. Left-click to remove part of the mask.

The label is “10 people”, what decision should be made?

Annotate or Reject?

Reject

Reason: Phrase is not in the image.

The label calls for 10 people, however, only 7 people are in the image.

What decision will you make on this image?
Label: A BMW car

Annotate or Reject?

Reject

Reason: Whole Image/Phrase is Image-Level

What decision will you make on this image?
Label: Number

Annotate or Reject?

Reject

Reason: Object refers to text

What decision will you make on this image?
Label: friends

Annotate or Reject?

Reject

Reason: Object refers to text

What decision will you make on this image?
Label: adobe photoshop

Annotate or Reject?

Reject

Reason: Phrase is not in the image

What decision will you make on this image?
Label: face mask

Annotate or Reject?

Annotate

Mask each instance of the face mask.

This job came with an “Accept” mask, what should be done?

Label: man

Modify the mask

Is the mask drawn correctly?
Label: 5 men

Yes or No?

Note: The different colors of masks show different masks

No.

There should be only one mask and it should contain 5 men.

What decision will you make on this image?
Label: fair-skinned man

Annotate or Reject?

Annotate

We do not flag complexion, so this object is to be masked.