Out-of-Distribution Detection with Semantic Mismatch under Masking

Deep neural networks (DNNs) are trained under a “close-world” assumption, where all the samples fed to the model are assumed to follow a narrow semantic distribution. However, when deployed in the wild, the model is exposed to an “open-world” with all kinds of inputs not necessarily following this distribution. Such out-of-distribution (OOD) samples with significantly different semantics may mislead DNN models and generate wrong prediction results with extremely high confidence, thereby hindering DNN’s deployment safety.

To tackle this problem, we propose a novel distance-based OOD detection framework, named Masked OOD Catcher (MoodCat), wherein we consider the semantic mismatch under masking as the distance metric. Specifically, for image classifiers, we first randomly mask a portion of the input image, use a generative model to synthesize the masked image to a new image conditioned on the classification result, and then calculate the semantic difference between the original image and the synthesized one for OOD detection.

Our insight is that, the classification result carries discriminative semantic information and it imposes strong constraints onto the synthesis procedure, especially when trying to recover the masked portions. With MoodCat, for correctly classified In-D data, the generative model can use the unmasked region to make up the masked part with sufficient training. In contrast, for OOD samples that are semantically different, the synthesized image based on the classification result tends to be dramatically different, especially for the masked region.

1. Pipeline of MoodCat.

2. Comparison of OOD Detection Methods.

3. Training pipeline of the proposed Conditional Binary Classifier.

4. OOD Detection Performance on Cifar-10 as In-D without using external OOD data for training.

5. OOD Detection Performance on Cifar-100 as In-D with Tiny-ImageNet as external data for training.

@inproceedings{yang2022out,
  title={Out-of-distribution detection with semantic mismatch under masking},
  author={Yang, Yijun and Gao, Ruiyuan and Xu, Qiang},
  booktitle={Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXIV},
  pages={373--390},
  year={2022},
  organization={Springer}
}