Article Contents

Monitoring Sea Fog over the Yellow Sea and Bohai Bay Based on Deep Convolutional Neural Network

Funding:

National Key R&D Program of China 2021YFC3000905

Open Research Program of the State Key Laboratory of Severe Weather 2022LASW-B09

National Natural Science Foundation of China 42375010


doi: 10.3724/j.1006-8775.2024.020

  • In this paper, we utilized the deep convolutional neural network D-LinkNet, a model for semantic segmentation, to analyze the Himawari-8 satellite data captured from 16 channels at a spatial resolution of 0.5 km, with a focus on the area over the Yellow Sea and the Bohai Sea (32°–42°N, 117°–127°E). The objective was to develop an algorithm for fusing and segmenting multi-channel images from geostationary meteorological satellites, specifically for monitoring sea fog in this region. Firstly, the extreme gradient boosting algorithm was adopted to evaluate the data from the 16 channels of the Himawari-8 satellite for sea fog detection, and we found that the top three channels in order of importance were channels 3, 4, and 14, which were fused into false color daytime images, while channels 7, 13, and 15 were fused into false color nighttime images. Secondly, the simple linear iterative super-pixel clustering algorithm was used for the pixel-level segmentation of false color images, and based on super-pixel blocks, manual sea-fog annotation was performed to obtain fine-grained annotation labels. The deep convolutional neural network D-LinkNet was built on the ResNet backbone and the dilated convolutional layers with direct connections were added in the central part to form a string-and-combine structure with five branches having different depths and receptive fields. Results show that the accuracy rate of fog area (proportion of detected real fog to detected fog) was 66.5%, the recognition rate of fog zone (proportion of detected real fog to real fog or cloud cover) was 51.9%, and the detection accuracy rate (proportion of samples detected correctly to total samples) was 93.2%.
  • 加载中
  • Figure 1.  Schematic diagram of the model.

    Figure 2.  Schematic diagram of the deep neural network.

    Figure 3.  AI monitoring and identification of sea fog on April 6, 2022.

    Figure 4.  AI monitoring and identification of sea fog on April 7, 2022.

    Table 1.  Details of the Himawari-8 dataset.

    Channel serial number Central wavelength Physical properties
    1 0.46 Vegetation, aerosols
    2 0.51 Vegetation, aerosols
    3 0.64 Low clouds, fog
    4 0.86 Vegetation, aerosols
    5 1.6 Cloud phase
    6 2.3 Particle size
    7 3.9 Low clouds, fog, forest fires
    8 6.2 Upper middle layer humidity
    9 7.0 Middle layer humidity
    10 7.3 Upper middle layer humidity
    11 8.6 Cloud phase, SO2
    12 9.6 Ozone content
    13 10.4 Cloud map, genting information
    14 11.2 Cloud map, sea surface temperature
    15 12.3 Cloud map, sea surface temperature
    16 13.3 Genting height
    DownLoad: CSV

    Table 2.  Comparison of the satellite test data results with those from sea observations.

    Data Sea observation with fog Sea observation without fog Fog area accuracy Fog area recognition rate Detection correct rate
    Satellite with fog 393 198 / / /
    Satellite without fog 522 9446 66.5% 51.9% 93.2%
    Cloud coverage 158 / / / /
    DownLoad: CSV
  • [1] GULTEPE I, MILBRANDT J A. Microphysical observations and mesoscale model simulation of a warm fog case during FRAM project [M]// GULTEPE I (ed), Fog and Boundary Layer Clouds: Fog Visibility and Forecasting. Basel: Birkhäuser, 2007: 1161–1178.
    [2] ZHANG S, REN Z, LIU J, et al. Variations in the lower level of the PBL associated with the Yellow Sea fog-new observations by L-band radar [J]. Journal of Ocean University of China, 2008, 7: 353–361, https://doi.org/10.1007/s11802-008-0353-1
    [3] LEE H Y, CHANG E C. Impact of land-sea thermal contrast on the inland penetration of sea fog over the coastal area around the Korean Peninsula [J]. Journal of Geophysical Research: Atmospheres, 2018, 123(12): 6487–6504, https://doi.org/10.1029/2017JD027633
    [4] HE J X, REN X Y, WANG H, et al. Analysis of the microphysical structure and evolution characteristics of a typical sea fog weather event in the Eastern Sea of China [J]. Remote Sensing, 2022, 14(21): 5604, https://doi.org/10.3390/rs14215604
    [5] XIAN J, HAN Y, HUANG S, et al. Novel lidar algorithm for horizontal visibility measurement and sea fog monitoring [J]. Optics Express, 2018, 26(26): 34,853–34,863, https://doi.org/10.1364/OE.26.034853
    [6] GAO Y, JIANG G. Research on influencing factors and countermeasures of fog navigation in Weihai Harbour [C]// Proceedings of the 5th International Conference on Education, Management, Arts, Economics and Social Science. Zhengzhou: Atlantis Press, 2018.
    [7] DORMAN C E, MEJIA J, KORAČIN D, et al. World marine fog analysis based on 58-years of ship observations [J]. International Journal of Climatology, 2019, 40(1): 145–168, https://doi.org/10.1002/joc.6200
    [8] FU G, GUO J, PENDERGRASS A, et al. An analysis and modeling study of a sea fog event over the Yellow and Bohai Seas [J]. Journal of Ocean University of China, 2008, 7: 27–34, https://doi.org/10.1007/s11802-008-0027-z
    [9] DU P, ZENG Z, ZHANG J W, et al. Fog season risk assessment for maritime transportation systems exploiting Himawari-8 data: A case study in Bohai Sea, China [J]. Remote Sensing, 2021, 13(17): 3530, https://doi.org/10.3390/rs13173530
    [10] HEO K Y, PARK S, HA K J, et al. Algorithm for sea fog monitoring with the use of information technologies [J]. Meteorological Applications, 2014, 21(2): 350–359.
    [11] WANG S, LI H, ZHANG M, et al. Assessing gridded precipitation and air temperature products in the Ayakkum Lake, Central Asia [J]. Sustainability 2022, 14(17): 10654, https://doi.org/10.3390/su141710654
    [12] MIAO K C, HAN T T, YAO Y Q, et al. Application of LSTM for short term fog forecasting based on meteorological elements [J]. Neurocomputing 2020, 408: 285–291, https://doi.org/10.1016/j.neucom.2019.12.129
    [13] GUIJO-RUBIO D, GUTIÉRREZ P, CASANOVA-MATEO C, et al. Prediction of low-visibility events due to fog using ordinal classification [J]. Atmospheric Research, 2018, 214: 64–73, https://doi.org/10.1016/j.atmosres.2018.07.017
    [14] PARK J, LEE Y J, JO Y D, et al. Spatio-temporal network for sea fog forecasting [J]. Sustainability, 2022, 14(23): 16163, https://doi.org/10.3390/su142316163
    [15] LEIPPER, DALE F. Fog on the US west coast: A review [J]. Bulletin of the American Meteorological Society, 1994, 75(2): 229–240, https://doi.org/10.1175/1520-0477(1994)075<0229:FOTUWC>2.0.CO;2 doi:
    [16] ERNST J A. Fog and stratus "invisible" in meteorological satellite infrared (IR) imagery [J]. Monthly Weather Review, 1975, 103(11): 1024–26, https://doi.org/10.1175/1520-0493(1975)103<1024:FASIMS>2.0.CO;2 doi:
    [17] AHN M H, SOHN E H, HWANG B J. A new algorithm for sea fog/stratus detection using GMS-5 IR data [J]. Advances in Atmospheric Sciences, 2003, 20: 899–913, https://doi.org/10.1007/BF02915513
    [18] EYRE J R, BROWNSCOMBE J L, ALLAM R J. Detection of fog at night using Advanced Very High Resolution Radiometer (AVHRR) imagery [J]. Meteorological Magazine, 1984, 113(1346): 266–271.
    [19] TURNER J, ALLAM R J, MAINE D R. A case study of the detection of fog at night using channels 3 and 4 on the Advanced Very High Resolution Radiometer (AVHRR) [J]. Meteorological Magazine, 1986, 115(1370): 285–290.
    [20] DERRIEN M, FARKI B, HARANG L, et al. Automatic cloud detection applied to NOAA-11/AVHRR imagery [J]. Remote Sensing of Environment, 1993, 46(3): 246–267, https://doi.org/10.1016/0034-4257(93)90046-Z
    [21] MILLÁN L F, LIVESEY N J, SANTEE M L, et al. Characterizing sampling and quality screening biases in infrared and microwave limb sounding [J]. Atmospheric Chemistry and Physics, 2018, 18(6): 4187–4199, https://doi.org/10.5194/acp-18-4187-2018
    [22] WANG T, ZENG J Y, CHEN K S, et al. Comparison of different intercalibration methods of brightness temperatures from FY-3D and AMSR2[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1–17, https://doi.org/10.1109/TGRS.2022.3176748
    [23] XIE X, WU S, XU H, et al. Ascending-descending bias correction of microwave radiation imager on board FengYun-3C [J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(6): 3126–3134, https://doi.org/10.1109/TGRS.2018.2881094
    [24] YANG H, ZOU X, LI X, et al. Environmental data records from FengYun-3B microwave radiation imager [J]. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(12): 4986–4993, https://doi.org/10.1109/TGRS.2012.2197003
    [25] WU D, LU B, ZHANG T, et al. A method of detecting sea fogs using CALIOP data and its application to improve MODIS-based sea fog detection [J]. Journal of Quantitative Spectroscopy and Radiative Transfer, 2015, 153: 88–94, https://doi.org/10.1016/j.jqsrt.2014.09.021
    [26] YI L, THIES B, ZHANG S, et al. Optical thickness and effective radius retrievals of low stratus and fog from MTSAT daytime data as a prerequisite for Yellow Sea fog detection [J]. Remote Sensing, 2016, 8: 8, https://doi.org/10.3390/rs8010008
    [27] LI J, HAN Z G, CHEN H B, et al. Fog detection over China's adjacent sea area by using the MTSAT geostationary satellite data [J]. Atmospheric and Oceanic Science Letters, 2015, 5(2): 128–133, https://doi.org/10.1080/16742834.2012.11446978
    [28] HARUN-AL-RASHID A, YANG C S. A simple sea fog prediction approach using GOCI observations and sea surface winds [J]. Remote Sensing Letters, 2017, 9(1): 21–30, https://doi.org/10.1080/2150704X.2017.1375609
    [29] YANG Z H, WU M, XU M Q, et al. MoANet: A motion attention network for sea fog detection in time series meteorological satellite imagery [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2024, 17: 1976–1987, https://doi.org/10.1109/JSTARS.2023.3340909
    [30] BAI C, ZHANG M, ZHANG J, et al. LSCIDMR: large-scale satellite cloud image database for meteorological research [J]. IEEE Transactions on Cybernetics, 2022, 52(11): 12,538–12,550, https://doi.org/10.1109/TCYB.2021.3080121
    [31] YI L, LI M Y, LIU S X, et al. Detection of dawn sea fog/low stratus using geostationary satellite imagery [J]. Remote Sensing of Environment, 2023, 294: 113622, https://doi.org/10.1016/j.rse.2023.113622
    [32] SIM S, IM J. Improved ocean-fog monitoring using Himawari-8 geostationary satellite data based on machine learning with SHAP-based model interpretation [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16: 7819–7837, https://doi.org/10.1109/JSTARS.2023.3308041
    [33] XU M Q, WU M, GUO J, et al. Sea fog detection based on unsupervised domain adaptation [J]. Chinese Journal of Aeronautics, 2022, 35(4): 415–425, https://doi.org/10.1016/j.cja.2021.06.019
    [34] ZHU C Y, WANG J H, LIU S W, et al. Sea fog detection using u-net deep learning model based on Modis data [C]// 201910th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS). Amsterdam: IEEE, 2019: 1–5.
    [35] LI Z, SHEN H, CHENG Q, et al. Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 150: 197–212, https://doi.org/10.1016/j.isprsjprs.2019.02.017
    [36] JEON H K, KIM S, EDWIN J, et al. Sea fog identification from GOCI images using CNN transfer learning models [J]. Electronics, 2020, 9(2): 311, https://doi.org/10.3390/electronics9020311
    [37] HUANG Y, WU M, GUO J, et al. A correlation context-driven method for sea fog detection in meteorological satellite imagery [J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1–5, https://doi.org/10.1109/LGRS.2021.3095731
    [38] ZHU X, XU M, WU M, et al. Annotating only at definite pixels: A novel weakly supervised semantic segmentation method for sea fog recognition [C]// 2022 IEEE International Conference on Visual Communications and Image Processing (VCIP). Suzhou: IEEE, 2022: 1–5.

Get Citation+

HUANG Bin, GAO Shi-bo, YU Run-ling, et al. Monitoring Sea Fog over the Yellow Sea and Bohai Bay Based on Deep Convolutional Neural Network [J]. Journal of Tropical Meteorology, 2024, 30(3): 223-229, https://doi.org/10.3724/j.1006-8775.2024.020
HUANG Bin, GAO Shi-bo, YU Run-ling, et al. Monitoring Sea Fog over the Yellow Sea and Bohai Bay Based on Deep Convolutional Neural Network [J]. Journal of Tropical Meteorology, 2024, 30(3): 223-229, https://doi.org/10.3724/j.1006-8775.2024.020
Export:  

Share Article

Manuscript History

Manuscript received: 18 October 2023
Manuscript revised: 15 May 2024
Manuscript accepted: 15 August 2024
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Monitoring Sea Fog over the Yellow Sea and Bohai Bay Based on Deep Convolutional Neural Network

doi: 10.3724/j.1006-8775.2024.020
Funding:

National Key R&D Program of China 2021YFC3000905

Open Research Program of the State Key Laboratory of Severe Weather 2022LASW-B09

National Natural Science Foundation of China 42375010

Abstract: In this paper, we utilized the deep convolutional neural network D-LinkNet, a model for semantic segmentation, to analyze the Himawari-8 satellite data captured from 16 channels at a spatial resolution of 0.5 km, with a focus on the area over the Yellow Sea and the Bohai Sea (32°–42°N, 117°–127°E). The objective was to develop an algorithm for fusing and segmenting multi-channel images from geostationary meteorological satellites, specifically for monitoring sea fog in this region. Firstly, the extreme gradient boosting algorithm was adopted to evaluate the data from the 16 channels of the Himawari-8 satellite for sea fog detection, and we found that the top three channels in order of importance were channels 3, 4, and 14, which were fused into false color daytime images, while channels 7, 13, and 15 were fused into false color nighttime images. Secondly, the simple linear iterative super-pixel clustering algorithm was used for the pixel-level segmentation of false color images, and based on super-pixel blocks, manual sea-fog annotation was performed to obtain fine-grained annotation labels. The deep convolutional neural network D-LinkNet was built on the ResNet backbone and the dilated convolutional layers with direct connections were added in the central part to form a string-and-combine structure with five branches having different depths and receptive fields. Results show that the accuracy rate of fog area (proportion of detected real fog to detected fog) was 66.5%, the recognition rate of fog zone (proportion of detected real fog to real fog or cloud cover) was 51.9%, and the detection accuracy rate (proportion of samples detected correctly to total samples) was 93.2%.

HUANG Bin, GAO Shi-bo, YU Run-ling, et al. Monitoring Sea Fog over the Yellow Sea and Bohai Bay Based on Deep Convolutional Neural Network [J]. Journal of Tropical Meteorology, 2024, 30(3): 223-229, https://doi.org/10.3724/j.1006-8775.2024.020
Citation: HUANG Bin, GAO Shi-bo, YU Run-ling, et al. Monitoring Sea Fog over the Yellow Sea and Bohai Bay Based on Deep Convolutional Neural Network [J]. Journal of Tropical Meteorology, 2024, 30(3): 223-229, https://doi.org/10.3724/j.1006-8775.2024.020
  • Sea fog is a type of condensation phenomenon that occurs in the lower atmosphere over coastal areas, islands, or sea surface, where a large number of water droplets and ice crystals suspended in the atmospheric boundary layer make the atmospheric horizontal visibility less than 1 km (Gultepe and Milbrandt [1]; Zhang et al. [2]; Lee et al. [3]; He et al. [4]). Notably, sea fog poses a significant threat, often leading to numerous accidents at sea and in coastal regions (Xian et al. [5]; Gao and Jiang [6]; Dorman et al. [7]; Fu and Guo [8]; Du et al. [9]). The observations from shore-based stations often fail to reflect the condition of sea fog at long distances, making the detection of this phenomenon a crucial yet challenging aspect in enhancing the overall capabilities of sea fog forecasting (Heo et al. [10]; Wang et al. [11]; Miao et al. [12]; Guijo-Rubio et al. [13]; Park et al. [14]).

    The experimental land and sea observation network, established by the United States Naval Air Systems Command during 1972–1982, incorporated radiosondes, ships, aircraft, balloons, and kites for detection (Leipper [15]). However, ground-based observations do not cover the entire sea area, and more resource-intensive space-based observations are needed. Ernst [16] pointed out that observations from the geostationary satellite GMS-1 can be used for cloud and fog observation. Based on improved spatial and temporal coherence methods, Ahn et al. [17] used the GMS-5 satellite observations with a high temporal resolution to develop a synthetic map of clear-sky radiation. They proposed an algorithm for monitoring sea fog using the infrared channel data from the GMS-5 satellite by comparing them with hourly infrared radiation observations. Eyre et al. [18] and Turner et al. [19] explored the dual channel difference method based on the difference in brightness temperature at wavelengths of 10.8 and 3.7 μm to delineate fog or stratus in terms of optical properties, and this method was later widely used (Derrien et al. [20]; Luis et al. [21]; Wang et al. [22]; Xie et al. [23]; Yang et al. [24]). In contrast to most passive-microwave observations, Wu et al. [25] used active cloud-aerosol lidar with orthogonal polarization observations to identify sea fog and evaluated the results by comparing them with the data from a moderate-resolution imaging spectroradiometer. Moreover, Li et al. [26] successfully extracted sea fog information from the multi-functional transport satellite-1R (MTSAT-1R) infrared channels using principal component analysis, texture analysis, and threshold detection (Jun et al. [27]). Sea fog detection using the optical properties of individual pixels cannot consider neighboring pixels and is prone to misidentify clouds as sea fog. Harun-Al-Rashid and Yang [28] estimated and predicted sea fog motions using the sea fog region of interest from the images of the geostationary ocean color imager and simulated wind data based on weather studies and forecasts. Overall, with the advantages of high temporal resolution and wide coverage, geostationary satellite observations can achieve continuous dynamic observations and become an effective means to monitor sea fog (Yang et al. [29]; Bai et al. [30]; Yi et al. [31]; Sim et al. [32]; Xu et al. [33]; Zhu et al. [34]; Li et al. [35]; Jeon et al. [36]).

    Recently, deep learning methods, especially semantic segmentation methods, have been widely applied to remote sensing inversion and have achieved remarkable performance. For example, Huang et al. [37] proposed a two-stage superpixel-based fully convolutional network (SFCNet) to achieve sea fog detection by learning the dependencies between pixels. Furthermore, Zhu et al. [38] attempted to propose a weakly supervised semantic segmentation method to deal with sea fog recognition using both satellite images and observational data.

    The remainder of this paper is organized as follows. Section 2 introduces the data and methods used in this research. Section 3 presents the AI-based sea fog monitoring model. Section 4 verifies model performance. Finally, the discussion and main conclusions are shown in Section 5.

  • In this research, we focused on sea fog over the Bohai Sea and the Yellow Sea, regions prone to frequent occurrences of such weather events. The data used in this study were primarily the Himawari-8 half-hourly standard data. For satellite observation, there were 16 channels, covering the region of 32°–42°N, 117°–127°E, as shown in Table 1.

    Channel serial number Central wavelength Physical properties
    1 0.46 Vegetation, aerosols
    2 0.51 Vegetation, aerosols
    3 0.64 Low clouds, fog
    4 0.86 Vegetation, aerosols
    5 1.6 Cloud phase
    6 2.3 Particle size
    7 3.9 Low clouds, fog, forest fires
    8 6.2 Upper middle layer humidity
    9 7.0 Middle layer humidity
    10 7.3 Upper middle layer humidity
    11 8.6 Cloud phase, SO2
    12 9.6 Ozone content
    13 10.4 Cloud map, genting information
    14 11.2 Cloud map, sea surface temperature
    15 12.3 Cloud map, sea surface temperature
    16 13.3 Genting height

    Table 1.  Details of the Himawari-8 dataset.

    This dataset contained two parts, i.e., images and labels. Each image measured 1024×1024 pixels and incorporated information from 16 distinct channels. To ensure consistency, multi-channel data were processed using linear interpolation, achieving a uniform resolution of 0.5 km across all channels with varying spatial resolutions. Color images can be generated by combining three channels. Similar to Xu et al. [33] and Huang et al. [37], we utilized the XGBoost algorithm to rank the correlation features of 16 channels related to sea fog, and selected the top three channels, namely channels 3, 4, and 14, to compose a false color image of daytime sea fog detection. Similarly, for nighttime sea fog, the algorithm selected channels 7, 13, and 15 to form a corresponding false color image.

    Regarding the labels, each label shares the same dimensions as the corresponding images, measuring 1024×1024 pixels. The sea fog areas were distinctly labeled as white, while all other regions were designated as black. Two metrics were used to evaluate the results, i.e., the mean intersection over union (mIOU) and the observation test. When using the metric mIOU, we randomly divided 201 daytime images with pixel-level manual labels from 2017 and 2018 into 151 labeled training images and 50 labeled test images.

    When using the other evaluation metric, we employed a satellite training dataset comprising 201 manually labeled pixel-level daytime satellite images for training, spanning the years 2017 and 2018. Additionally, we incorporated test data consisting of 22 daytime images per day, captured from March 11 to July 20, 2019, across a 132-day period. All images were acquired during the time frame of 07:00 to 17:30 Beijing Time (BJT), with only minor gaps in data, summing up to a total of 2756 images.

    We selected the observations from 37 stations in the Yellow Sea and Bohai Sea regions, including island and buoy data. The data from sea stations were selected to be able to intersect with the satellite test data in time and space, with a time interval of three hours and four sets per day (08:00, 11:00, 14:00, and 17:00 BJT). The observations from three stations in Qingdao were complete, and their time interval was one hour. In the following paragraphs, the 40 stations in the Yellow Sea, the Bohai Sea, and Qingdao are collectively referred to as sea stations, and in total, there were 10, 559 sets of observational data, which were not used for modeling but only to verify the satellite image results.

  • Figure 1 is the schematic diagram of the model. Specifically, the first step was to acquire continuous geostationary satellite images (GSI) as source data. The second step was to arrange the above source data in chronological order to obtain temporal image data, and in this step, cloud and fog regions or cloud-fog mixture regions of each image in the source data were extracted by using color-based image segmentation method to obtain an initial mask corresponding to each image. In the third step, the Farneback optical flow method was adopted to generate an optical flow map of the temporal image data. In the fourth step, the average optical flow size of each pixel of the cloud area and fog area was calculated based on the optical flow map and the initial mask, and the cloud-fog classification threshold was determined based on the average optical flow size of the cloud and fog areas and the GSI data. The fifth step was to perform cloud-fog classification at each pixel point of the GSI based on the threshold to obtain automatic identification results of clouds and fog.

    Figure 1.  Schematic diagram of the model.

  • Figure 2 is the schematic diagram of the deep neural network. In this neural network, the 16-channel data from the Himawari-8 satellite was first evaluated by using the extreme gradient boosting algorithm for sea fog detection. The results indicate that the top three most influential channels were, in descending order of significance, channels 3, 4, and 14. The images in these three channels were fused into a false color image. Then, the superpixel segmentation of the false color image was conducted based on the simple linear iterative clustering superpixel algorithm. According to superpixel blocks, manual sea-fog annotation was performed to obtain fine-grained annotation labels. The deep convolutional neural network D-LinkNet was built on the ResNet backbone, and in its central part, dilated convolutional layers with direct connections were added to form a string-and-combine structure with five branches having different depths and receptive fields.

    Figure 2.  Schematic diagram of the deep neural network.

    Accordingly, we also proposed an automatic cloud-fog recognition system based on GSI sequences. This system included an acquisition module for acquiring continuous-time source data of GSI. Also, the automatic cloud-fog recognition system included a data pre-processing module, which was used to arrange the GSI source data in chronological order to obtain temporal image data, and for color-based image segmentation to extract the cloud and fog areas from each image of geostationary satellite source data, in order to obtain the initial mask corresponding to each image. Moreover, this system included an optical flow algorithm module for generating an optical flow map of the temporal image data based on the Farneback optical flow method, as well as a threshold calculation module. The threshold calculation module was used to calculate the average optical flow of each pixel point of the cloud and fog regions based on the optical flow map and the initial mask to determine the cloud-fog classification threshold. Additionally, a classification module was also included in this system, which was used to classify clouds and fog according to the above cloud-fog classification threshold in each connected domain of the initial mask of GSI source data. In this way, the automatic cloud recognition results can be obtained.

  • The mIOU and the observation test were selected as the evaluation metrics of the semantic segmentation results.

    The mIOU is a standard metric for measuring semantic segmentation, which can be described as the ratio of the intersection of two sets to the union. In the semantic segmentation, these two sets are the actual values and the predicted values. The mIOU can be expressed as Eq. 1:

    $$ {\rm{mIOU}} = \frac{1}{{k + 1}}\sum\limits_k^{i = 0} \frac{{{\rm{TP}}}}{{{\rm{TP}} + {\rm{FN}} + {\rm{FP}}}} $$ (1)

    where TP indicates the true and positive values, FN the false and negative values, and FP the false and positive values. The mIOU is the result of summing and then averaging the intersection over union (IOUs) for each set, where the IOU indicates the ratio of the agreement between the predicted values and the actual values. The mIOU can be rewritten as Eq. 2:

    $$ {\rm{mIOU}} = \frac{1}{{k + 1}} \sum\limits_k^{i = 0} \frac{{{p_{ii}}}}{{ \sum\nolimits_{j = 0}^k{p_{ij}} + \sum\nolimits_{j = 0}^k{p_{ji}} - {p_{ii}}}} $$ (2)

    where i represents the true values, j the predicted values, and pij the prediction of i into j.

    The observation test, which compares the obtained satellite test data with the observations of sea stations, uses the accuracy rate of fog area, the recognition rate of fog area and the detection accuracy rate as evaluation metrics, as expressed in Eqs. 3–5:

    $$ {P_a} = \frac{{{m_f} {\cap }{s_f}}}{{\left( {{m_f} {\cap } {s_f}} \right) + \left( {{m_n} {\cap } {s_f}} \right)}} $$ (3)
    $$ {P_r} = \frac{{{m_f}{\cap} {s_f}}}{{\left( {{m_f}{\cap} {s_f}} \right) + \left( {{m_f}{\cap} {s_n}} \right) - {c_c}}} $$ (4)
    $$ {D_a} = \frac{{\left( {{m_f}{\cap} {s_f}} \right) + \left( {{m_n}{\cap} {s_n}} \right)}}{{{n_{{\rm{total}}}}}} $$ (5)

    where mf indicates fog data observed by the offshore observatory, sf indicates fog data detected by our method, mn indicates no-fog data observed by the offshore observatory, sn indicates no-fog data detected by our method, cc indicates dense impermeable medium-high cloud cover, ntotal is the number of all observed samples, Pa indicates the accuracy rate of fog area, Pr indicates the recognition rate of fog area and Da indicates the detection accuracy rate. The recognition rate of fog area does not have statistical cloud coverage because it is based on images and is difficult to recognize.

  • A sea fog recognition model based on the D-LinkNet deep convolutional neural network was used to test the satellite data test set. The training and test sets were 151 and 50 daytime sea fog images from 2017 to 2018, respectively, and the mIOU was used as an evaluation metric for the semantic segmentation results. The mIOU on the test set was 0.9436. The test data results of the trained model were compared with the results of the sea observations, which were tested from March 11, 2019 to July 20, 2019, with a total of 10, 559 observations from 40 observation stations.

    The satellite test data results and the offshore observational data results were counted. According to the formulae for the accuracy rate of fog area, the recognition rate of fog area and the detection accuracy rate in Section 3.1 of this paper, the accuracy rate of fog area (proportion of detected real fog to detected fog) was calculated to be 66.5%, the recognition rate of fog area (proportion of detected real fog to real fog or cloud cover) was 51.9%, and the detection accuracy rate (proportion of samples detected correctly to total samples) was 93.2%. The final data are recorded in Table 2.

    Data Sea observation with fog Sea observation without fog Fog area accuracy Fog area recognition rate Detection correct rate
    Satellite with fog 393 198 / / /
    Satellite without fog 522 9446 66.5% 51.9% 93.2%
    Cloud coverage 158 / / / /

    Table 2.  Comparison of the satellite test data results with those from sea observations.

  • We selected two cases of the artificial intelligence (AI) sea fog inversion recognition at 14:00 BJT on April 6, 2022 (daytime) and 04:00 BJT on April 7, 2022 (nighttime) for case test, as shown in Fig. 3 and Fig. 4. The false color images on the left were generated by fusing data from channels 3, 4, and 14.

    Figure 3.  AI monitoring and identification of sea fog on April 6, 2022.

    Figure 4.  AI monitoring and identification of sea fog on April 7, 2022.

    As shown in Fig. 3, the left column is the Himawari-8 satellite cloud image, and the right column is the AI sea fog inversion recognition at the same time. As can be seen, the deep-learning-based image fusion segmentation algorithm can better and more accurately inverse the pattern of sea fog over the southern Yellow Sea. Fig. 4 shows that the algorithm also has a good inversion recognition effect for night fog, and also has a good recognition effect for the sea fog area in the central–eastern part of the Yellow Sea and the southern Yellow Sea on April 7, 2022.

    In summary, whether for the overall test from March to July 2019 or for individual cases, the geostationary satellite multi-channel image fusion segmentation algorithm based on deep learning had better results for the recognition of sea fog in the Yellow and Bohai Sea by using the D-LinkNet deep convolutional neural network semantic segmentation algorithm model. This paper proposed an AI-based sea fog recognition system, which automatically recognized cloud and fog through acquisition module, data pre-processing module, optical flow algorithm module, threshold calculation module, and classification module. It was based on the optical flow algorithm, and the difference in timing information between fog and other meteorological phenomena can significantly reduce the interference of similar meteorological phenomena on cloud and fog differentiation, which effectively improves the accuracy of cloud and fog recognition. Overall, the algorithm does not need too much modification when recognizing clouds and fog in different time and location. The difference in motion of clouds and fog is universal, so it has high universality.

  • This paper proposed a deep learning-based multi-channel image fusion segmentation algorithm for geostationary meteorological satellites, using the D-LinkNet deep convolutional neural network semantic segmentation algorithm model for 16 channels of Himawari-8 satellite data with a spatial resolution of 0.5 km in the Yellow and Bohai Sea area (32°–42°N, 117°–127°E). The training and test sets were derived from 151 and 50 daytime sea fog images from 2017 to 2018, and the mIOU was used as an evaluation metric for the semantic segmentation results, with a mIOU of 0.9436 on the test set. The results of the trained model were compared with that of sea observations. The testing period was from March 11, 2019 to July 20, 2019, with a total of 10, 559 observations from 40 observation stations. The accuracy rate of fog area (proportion of detected real fog to detected fog) was calculated to be 66.5%, the recognition rate of fog area (proportion of detected real fog to real fog or cloud cover) was 51.9%, and the detection accuracy rate (proportion of samples detected correctly to total samples) was 93.2%.

  • The main conclusions of this paper are as follows. The system used an optical flow algorithm to recognize clouds and fog based on time-continuous GSI source data. It is found that the use of the difference between fog and other meteorological phenomena timing information can obviously reduce the interference of similar meteorological phenomena (especially stratus) on cloud and fog distinction.

    However, the theory of interpretability of deep learning methods has not yet been well developed, and the recognition of sea fog did not correspond to the evolutionary mechanism of sea fog. If the interpretability of deep learning in sea fog monitoring can be enhanced and corresponded with the physical mechanism of sea fog, fostering a deeper integration between meteorology and artificial intelligence, we are confident that AI technology could yield superior recognition outcomes in sea fog monitoring and contribute to a deeper understanding of the mechanisms behind the formation and dissipation of sea fog.

Reference (38)

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return