link to paper

Author: jwmueller

multiannotator_cifar10/multiannotator_cifar10.ipynb

@@ -155,7 +155,7 @@
    "id": "e31453ef",
    "metadata": {},
    "source": [
-    "Next we train our model on these consensus labels and then obtain out-of-sample predicted probabilities for each example in the dataset. We will then use these `pred_probs` to generate improved `consensus_labels` using cleanlab's [CROWDLAB algorithm](TODO:link-to-paper). In turn, these improved `consensus_labels` can then be used to train a better model that generates more accurate `pred_probs`. This process is iterated in the loop below to repeatedly improve both the consensus labels and our classification model.\n",
+    "Next we train our model on these consensus labels and then obtain out-of-sample predicted probabilities for each example in the dataset. We will then use these `pred_probs` to generate improved `consensus_labels` using cleanlab's [CROWDLAB algorithm](https://cleanlab.github.io/multiannotator-benchmarks/paper.pdf). In turn, these improved `consensus_labels` can then be used to train a better model that generates more accurate `pred_probs`. This process is iterated in the loop below to repeatedly improve both the consensus labels and our classification model.\n",
     "\n",
     "Within each round, the `train_model()` method trains a `resnet18` image classifier using cross validation to get out-of-sample predicted class probabilities for each image in the dataset. This function can be replaced with custom code for training your own model. Here we use the [AutoGluon](https://auto.gluon.ai/) AutoML library to train powerful image classifiers (like ResNet or Swin Transformer), but this requires a GPU to run in a reasonable time."
    ]