Multi-Annotator Collaboration

Multi-Annotator Collaboration is a feature that enables efficient collaboration among multiple annotators within a project.

Create a Project

As an admin, you have the ability to create a project for your team. For example, you can create a project called project1:

After the project is created, you can select project1 from the dropdown to see team members and datasets associated with the project.

To start, there are no other team members and datasets in new projects.

Add Team Members to Project

As an admin, you have the ability to invite other individuals to collaborate on a project. By extending invitations, you can create a team of annotators who can collectively work on annotating datasets within the project. This collaborative approach promotes efficient and diverse perspectives while ensuring accuracy and consistency in annotations.

Here we add john@anote.ai as an annotator, and roger@anote.ai as an admin.

Roles

Within a project, there are two main roles assigned to the participants:

Admin: The admin role is typically assigned to the individual who initially creates the project. Admins have the highest level of access and control over the project settings. They can invite other annotators, manage dataset access, and perform administrative tasks such as project configuration and user management.

Annotator: Annotators are the individuals invited to collaborate on a project. They have limited administrative privileges but possess the necessary permissions to access and annotate datasets assigned to them. Annotators can contribute to the project by providing annotations, which the admins can review.

Adding Datasets to Projects

As an admin, you have the ability to add datasets to projects which require collaboration. For example, in the project called rajat-natan, we can add the Amazon Reviews to the datasets in project by clicking the checkmark to the left of the dataset we want to add's name, and by pressing Add Datasets:

Below you can see that now the Amazon Reviews dataset is in the rajat-natan project.

All members of the rajat-natan project now have access to the Amazon Reviews dataset.

Accessing Datasets

In your My Datasets tab, admins and annotators can access datasets that they are collaborators on. For instance, if an admin creates the Amazon Reviews dataset, the annotators part of the rajat-natan project should be able to load the Amazon Reviews dataset here to begin annotating, even if they have not created that dataset.

In My Datasets, admins and annotators can also see datasets that as a user, they have created, but are not part of particular projects they are in. Once selected, users can click load dataset to obtain access to the dataset.

Ownership of Datasets

In a project, the ownership of datasets is tied to the project itself. Annotators can annotate datasets within the projects they are a part of, even if they didn't create the project. However, the permissions and level of control an annotator has over the project are limited compared to admins. Annotators can only work on the datasets assigned to them and contribute annotations within their assigned datasets.

Annotators - Annotating Datasets

Once assigned to a project, annotators can access and annotate the data within datasets in that project. Annotating involves adding labels, tags, marking features important or any other relevant information to the data based on the project's annotation guidelines or requirements.

Admins - Collaboration and Communication

Admins have all permissions that annotators have, but in addition can review each annotators annotations, discuss uncertainties, clarify guidelines, and resolve conflicts. Here is a more detailed breakdown of the annotator and admin roles.