CFMW: Cross-modality Fusion Mamba for Robust Object Detection under Adverse Weather

Haoyuan Li, Qi Hu, Binjia Zhou, You Yao, Jiacheng Lin, Kailun Yang^*, Peng Chen^*

^*Corresponding Authors

🔍 Overview

Model

Dataset

Data Preparation

1. Download the four compressed files here. They require about 20GB of storage space.
2. Run the following command to merge the two files into one and uncompress it. This will produce a folder named SWVI containing 60K visible images, infrared images and visible images with weather-influence.

cat SWVI.* > SWVI.zip
unzip SWVI.zip

Demo

🏠 About

In this work, we propose the Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment stability and cost-effectiveness under adverse weather conditions. Leveraging the proposed Perturbation-Adaptive Diffusion Model (PADM) and Cross-modality Fusion Mamba (CFM) modules, CFMW is able to reconstruct visual features affected by adverse weather, enriching the representation of image details. To bridge the gap in relevant datasets, we construct a new Severe Weather Visible-Infrared (SWVI) dataset, encompassing diverse adverse weather scenarios such as rain, haze, and snow.

🔗 Citation

If you find our work and this codebase helpful, please consider starring this repo 🌟 and cite:

@ARTICLE{11077409,
  author={Li, Haoyuan and Hu, Qi and Zhou, Binjia and Yao, You and Lin, Jiacheng and Yang, Kailun and Chen1, Peng},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={CFMW: Cross-modality Fusion Mamba for Robust Object Detection under Adverse Weather}, 
  year={2025},
  doi={10.1109/TCSVT.2025.3587918}}

👏 Acknowledgements

• YOLOv5
• CFT
• Vison Mamba
• WeatherDiffusion