Smoke or Cloud: Real-Time Satellite Image Segmentation in a Wildfire Data Integration Application

Advanced satellite data is increasingly used for wildfire detection and monitoring, yet the most accurate satellites have low temporal resolution with data uploads only 1-2 times per day. While real-time satellite data products, from sources such as the GOES-R series, provide sufficient temporal resolution with data uploads every five minutes, GOES-R fire products have substantially worse performance at hot spot detection than their less frequent counterparts. Since aerosol contamination impacts the confidence of the GOES-R hot spot detection algorithm, regardless of contamination from fire-indicating smoke or false positive-indicating clouds, differentiating smoke from cloud has the potential to improve the accuracy of real-time hot spot detection. The primary contribution of this paper is a multi-class smoke and cloud segmentation model that classifies smoke, cloud, and neither pixels from GOES-R true color images in a real-time application. We use a pre-trained and fine-tuned U-Net model, which exhibits comparable performance the larger and slower ResUnet model. Our model improves upon existing satellite-based smoke segmentation, with 85\% accuracy and 68\% mean intersection-over-union on the test set. The model is deployed in an Open Data Integration for wildfire management (ODIN) application, allowing for real-time smoke and cloud detection to improve situational awareness regarding smoke location. From real-time image import to smoke-cloud segmentation display in the browser, the total run time is approximately 74 seconds, with 52 seconds total from the segmentation model pipeline.