Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning

GitHub Link

The GitHub link is https://github.com/LiheYoung/ShrinkMatch

Introduce

The GitHub repository "LiheYoung/ShrinkMatch" presents the official PyTorch implementation of the ICCV 2023 paper titled "Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning." The paper addresses the issue of uncertainty in pseudo labels by introducing the ShrinkMatch framework. Unlike fixed confidence thresholds, ShrinkMatch identifies and removes confusion classes for the top-1 class, converting uncertain samples into certain ones. This is achieved by adapting a shrunk class space and applying consistency regularization. The method shows promising results on various datasets, including CIFAR-10, CIFAR-100, STL-10, SVHN, and ImageNet-1K, outperforming previous techniques like SimMatch and FixMatch. Detailed results and training logs are provided for reference. The codebase is available for use, and the project appreciates contributions from other related codebases. If used, the paper should be cited as specified in the citation section. To mitigate potentially incorrect pseudo labels, recent frameworks mostly set a fixed confidence threshold to discard uncertain samples.

Content

This codebase provides the official PyTorch implementation of our ICCV 2023 paper: In semi-supervised learning, to mitigate potentially incorrect pseudo labels, recent frameworks mostly set a fixed confidence threshold to discard uncertain samples. This practice ensures high-quality pseudo labels, but incurs a relatively low utilization of the whole unlabeled set. Our key insight is that these uncertain samples can be turned into certain ones, as long as the confusion classes for the top-1 class are detected and removed. For each uncertain sample, our proposed ShrinkMatch adaptively seeks a shrunk class space, which merely contains the original top-1 class as well as remaining less likely classes, to satisfy the pre-defined threshold, e.g., 0.95. We then impose a consistency regularization in this shrunk space. Furthermore, considering the varied reliability among uncertain samples and the gradually improved model during training, we correspondingly design two reweighting principles for our uncertain loss. We provide all training logs. You can refer to them when reproducing. Please enter the corresponding directory according to your dataset. Many thanks to SimMatch, TorchSSL, and FixMatch-PyTorch for sharing their codebases and training logs. If you find this project useful, please consider citing:

---