Active Learning
From a 10-label seed, uncertainty sampling reached 0.5714 and random sampling 0.6418 on the same budget. It only won once the seed reached 400 in this experiment.
-
-
Full Annotation
Leaving 80% of training positives unlabeled reduced recall from 0.6341 to 0.0031 in this experiment. Evaluation-label omissions have different effects.
-
Weak Supervision
Oracle source weights achieved 0.8088 accuracy at 93.7% coverage. Coverage also depends on how ties and abstentions are resolved.
-
Label Noise
Random label noise in training barely moved accuracy, 0.7732 to 0.7681. The same noise in the test set took it to 0.5595, and AUROC to 0.5743.