弱监督语义分割导读
一般认为，图像级的标注是弱标注（例如图像分类的类别标注），像素级的标注是强标注（例如分割标注的mask标注），对于普通的分割任务来说：
- 数据是图像，标注是mask，这属于完全监督问题Supervised；
- 如果标注是annotations或者图像级标注，这属于弱监督问题Weakly-supervised；
- 如果标注只有少部分是mask，剩余是annotations或者图像级标注，这属于半监督问题Semi-supervised。
常见的弱标注：bounding box（检测框），scribbles（涂鸦），points（点），image-level labels（图像级别）。利用弱标注可以显著的减少标注时间，如果可以利用弱标签就可以获得与mask标签不相上下的精度和准确率，那我们利用弱标签和弱监督将会促进产品和数据的迭代。我们按照不同的弱标签–>语义分割的标签mask做一下整理 ⬇️。
ps：我们聚焦于语义分割，实例分割和全景分割也有很多成果，并未在本篇博客中列出。
弱监督语义分割论文整理
基于Bounding box的弱监督语义分割
利用深度学习方法和bounding box作为弱标签的语义分割研究，之前，基于Bounding box的分割以Grabcut这类算法一枝独秀，但是效果虽然有突破但不甚理想，所以2016年的Deepcut对于Grabcut做了进一步的优化得到了较好的效果。2015年ICCV中的BoxSup算作是开山之作（He,K为二作的又一次大佬的实力展现），但其中需要以无监督获取到的Candidate mask来迭代更新标签。在这一想法上，发表于2020年CVPR的Self-correcting Networks可以说是将这一想法又往前推进了一步（其实这个工作2018年就已经初见雏形，但2020年才正式发表），将金字塔模型、辅助分割模型和自矫正模块结合在一起，不过这个网络里面有少部分的全监督数据。
| Year/Meeting | Author/Paper |
|---|---|
| 2015 ICCV | Dai, J., He, K., & Sun, J. BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation |
| 2015 ICCV | Papandreou, G., Chen, L.-C., Murphy, K., & Yuille, A. L. Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation |
| 2016 Axiv | Rajchl, Martin, et al. DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks |
| 2017 CVPR | Khoreva, A., Benenson, R., Hosang, J., Hein, M., & Schiele, B. Simple Does It: Weakly Supervised Instance and Semantic Segmentation |
| 2020 CVPR | Mostafa S. Ibrahim, Arash Vahdat, & William G. Macready. Weakly Supervised Semantic Image Segmentation with Self-correcting Networks |
详解Paper的博客连接：
【2020 CVPR】Semi-Supervised Semantic Image Segmentation with Self-correcting Networks
基于Image-level labels的弱监督语义分割
基于Image-level labels的弱标签主要是图像分类标签（同理可以扩展到视频分类中），目前，大多数的基于图像分类标签来进行的弱监督语义分割研究的思路：将对图像分类标签响应最强烈的区域作为最初始的种子，通过不同的扩张手段得到更多的区域，从而得到分割的mask。在此过程中，和传统的机器学习算法结合比较多，因为需要得到底层特征的相关性。
基于Image-level labels弱标签的弱监督语义分割列表：
| Year/Meeting | Author/Paper |
|---|---|
| 2014 arXiv | Pathak, D., Shelhamer, E., Long, J., & Darrell, T. Fully Convolutional Multi-Class Multiple Instance Learning. |
| 2015 CVPR | Pinheiro, P. O., Collobert, R., & Epfl, D. L. From Image-level to Pixel-level Labeling with Convolutional Networks |
| 2015 ICCV | Papandreou, G., Chen, L.-C., Murphy, K., & Yuille, A. L. Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation. |
| 2016 Pattern Recognition | Wei, Y., Liang, X., Chen, Y., Jie, Z., Xiao, Y., Zhao, Y., & Yan, S. Learning to segment with image-level annotations. |
| 2016 ECCV | Shimoda, W., & B, K. Y. Distinct Class-Specific Saliency Maps for Weakly Supervised Semantic Segmentation. |
| 2016 ECCV | Saleh, F., Akbarian, M. S. A., Salzmann, M., Petersson, L., Gould, S., & Alvarez, J. M. Built-in Foreground/Background Prior for Weakly-Supervised Semantic Segmentation. |
| 2016 ECCV | Kolesnikov, A., & Lampert, C. H. Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation. |
| 2016 ECCV | Qi, X., Liu, Z., Shi, J., Zhao, H., & Jia, J. Augmented feedback in semantic segmentation under image level supervision. |
| 2017 PAMI | Wei, Y., Liang, X., Chen, Y., Shen, X., Cheng, M.-M., Zhao, Y., & Yan, S. STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation. |
| 2017 CVPR | Roy, A., & Todorovic, S. Combining Bottom-Up, Top-Down, and Smoothness Cues for Weakly Supervised Image Segmentation. |
| 2017 CVPR | Durand, T., Mordan, T., Thome, N., & Cord, M. WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation. |
| 2017 CVPR | Wei, Y., Feng, J., Liang, X., Cheng, M.-M., Zhao, Y., & Yan, S. Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach. |
| 2017 AAAI | Hong, S., Yeo, D., Kwak, S., Lee, H., & Han, B. Weakly Supervised Semantic Segmentation using Web-Crawled Videos. |
| 2018 CVPR | Wang, X., You, S., Li, X., & Ma, H. Weakly-Supervised Semantic Segmentation by Iteratively Mining Common Object Features. |
| 2018 CVPR | Huang, Z., Wang, X., Wang, J., Liu, W., & Wang, J. Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing. |
| 2018 CVPR | Ahn, J., & Kwak, S. Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation. |
| 2018 CVPR | Wei, Y., Xiao, H., Shi, H., Jie, Z., Feng, J., & Huang, T. S. Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation |
| 2019 CVPR | Shen Y, Ji R, Wang Y, et al. Cyclic Guidance for Weakly Supervised Joint Detection and Segmentation |
| 2019 ICCV | Shimoda W, Yanai K. Self-Supervised Difference Detection for Weakly-Supervised Semantic Segmentation |
| 2019 ICCV | Zeng Y, Zhuge Y, Lu H, et al. Joint learning of saliency detection and weakly supervised semantic segmentation |
| 2020 Neurocomputing | Wang X, Ma H, You S. Deep clustering for weakly-supervised semantic segmentation in autonomous driving scenes. |
| 2020 IJCV | Wang X, Liu S, Ma H, Yang M-H. Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning. |
基于Scribbles的弱监督语义分割
| Year/Meeting | Author/Paper |
|---|---|
| 2016 CVPR | Lin, D., Dai, J., Jia, J., He, K., & Sun, J. ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation. |
| 2016 MICCAI | Çiçek, Özgün, et al. “3d u-net: learning dense volumetric segmentation from sparse annotation.” |
基于Points的弱监督语义分割
| Year/Meeting | Author/Paper |
|---|---|
| 2016 ECCV | Bearman, A., Russakovsky, O., Ferrari, V., & Fei-Fei, L. What’s the point: Semantic segmentation with point supervision. |
本篇博客会不断的更新！！！其中有些很好的论文会另外开博客详细和大家一起共同学习一下，如果伙伴们有什么好的学习弱监督分割的方法请和我们一起分享，大家一起进步！博客中如果有不周到的地方还请大家多多指正！
