AbstractConventional RGB‐T salient object detection treats RGB and thermal modalities equally to locate the common salient regions. However, the authors observed that the rich colour and texture information of the RGB modality makes the objects more prominent compared to the background; and the thermal modality records the temperature difference of the scene, so the objects usually contain clear and continuous edge information. In this work, a novel mirror‐complementary Transformer network (MCNet) is proposed for RGB‐T SOD, which supervise the two modalities separately with a complementary set of saliency labels under a symmetrical structure. Moreover, the attention‐based feature interaction and serial multiscale dilated convolution (SDC)‐based feature fusion modules are introduced to make the two modalities complement and adjust each other flexibly. When one modality fails, the proposed model can still accurately segment the salient regions. To demonstrate the robustness of the proposed model under challenging scenes in real world, the authors build a novel RGB‐T SOD dataset VT723 based on a large public semantic segmentation RGB‐T dataset used in the autonomous driving domain. Extensive experiments on benchmark and VT723 datasets show that the proposed method outperforms state‐of‐the‐art approaches, including CNN‐based and Transformer‐based methods. The code and dataset can be found at https://github.com/jxr326/SwinMCNet.

More from our Archive

• DOI: 10.1049/ipr2.12854 2023

  Image segmentation technology based on transformer in medical decision‐making system

  Keke He, Fangfang Gou, Jia Wu

• DOI: 10.3390/jimaging9070140 2023

  Conv-ViT: A Convolution and Vision Transformer-Based Hybrid Feature Extraction Method for Retinal Disease Detection

  Pramit Dutta, Khaleda Akther Sathi, Md. Azad Hossain, M. Ali Akber Dewan

• DOI: 10.1049/cit2.12259 2023

  Thermoelectric energy harvesting for internet of things devices using machine learning: A review

  Tereza Kucova, Michal Prauzek, Jaromir Konecny, Darius Andriukaitis, Mindaugas Zilys, Radek Martinek

• DOI: 10.1049/ipr2.12841 2023

  Multimodal predictive classification of Alzheimer's disease based on attention‐combined fusion network: Integrated neuroimaging modalities and medical examination data

  Hui Chen, Huiru Guo, Longqiang Xing, Da Chen, Ting Yuan, Yunpeng Zhang, Xuedian Zhang

• DOI: 10.3390/jimaging9070134 2023

  Diagnostic Applications of Intraoral Scanners: A Systematic Review

  Francesca Angelone, Alfonso Maria Ponsiglione, Carlo Ricciardi, Giuseppe Cesarelli, Mario Sansone, Francesco Amato

• DOI: 10.1049/cvi2.12221 2023

  Mirror complementary transformer network for RGB‐thermal salient object detection

  Xiurong Jiang, Yifan Hou, Hui Tian, Lin Zhu

• DOI: 10.1142/s0219467824500402 2023

  Computer-Aided Classification of Cell Lung Cancer Via PET/CT Images Using Convolutional Neural Network

  Dhekra El Hamdi, Ines Elouedi, Ihsen Slim

• DOI: 10.3390/jimaging9070144 2023

  Varroa Destructor Classification Using Legendre–Fourier Moments with Different Color Spaces

  Alicia Noriega-Escamilla, César J. Camacho-Bello, Rosa M. Ortega-Mendoza, José H. Arroyo-Núñez, Lucia Gutiérrez-Lazcano

• DOI: 10.1049/ipr2.12871 2023

  LiteDEKR: End‐to‐end lite 2D human pose estimation network

  Xueqiang Lv, Wei Hao, Lianghai Tian, Jing Han, Yuzhong Chen, Zangtai Cai

• DOI: 10.1049/ipr2.12870 2023

  FCUnet: Refined remote sensing image segmentation method based on a fuzzy deep learning conditional random field network

  Xiangyue Ma, Jindong Xu, Qiangpeng Chong, Shifeng Ou, Haihua Xing, Mengying Ni