Skip to content

Commit a2fb669

Browse files
authored
update multiannotator links
1 parent 1235108 commit a2fb669

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

multiannotator_cifar10/multiannotator_cifar10.ipynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
"id": "44a0555b",
1414
"metadata": {},
1515
"source": [
16-
"This example shows how to repeatedly improve consensus labels established from data labeled by multiple annotators by iterating the following steps: (1) train a model on the current consensus labels, (2) leverage the model's predictions to obtain superior consensus labels that can be used to subsequently train a better model in the next round. In each round, consensus labels are established using the [CROWDLAB algorithm](TODO:linktopaper), for which a quickstart tutorial is available in the [cleanlab documentation](https://docs.cleanlab.ai/). \n",
16+
"This example shows how to repeatedly improve consensus labels established from data labeled by multiple annotators by iterating the following steps: (1) train a model on the current consensus labels, (2) leverage the model's predictions to obtain superior consensus labels that can be used to subsequently train a better model in the next round. In each round, consensus labels are established using the [CROWDLAB algorithm](https://cleanlab.github.io/multiannotator-benchmarks/paper.pdf), for which a quickstart tutorial is available in the [cleanlab documentation](https://docs.cleanlab.ai/stable/tutorials/multiannotator.html). \n",
1717
"\n",
1818
"Here we demonstrate this functionality using a subset of the [CIFAR-10H](https://github.com/jcpeterson/cifar-10h) dataset from Peterson et al. (2019), in which multiple human annotators were asked to suggest labels for images from the famous [CIFAR-10](https://www.cs.toronto.edu/~kriz/cifar.html) image classification dataset.\n",
1919
"Because this notebook utilizes AutoML for model training and cleanlab is compatible with any model/dataset, you should be able to run the below code with any image classification dataset where each image has been labeled by multiple annotators."

0 commit comments

Comments
 (0)