Assessing the Applicability of Multi-Source Precipitation Products over the Chinese Mainland and Its Seven Regions

Precipitation is integral to the global cycles of matter and energy, and plays an important role in hydrology and meteorology (Gat and Airey ^[1]). Despite its significance, precipitation remains a complex phenomenon characterized by considerable spatial and temporal variability (Immerzeel et al. ^[2]; Marzano et al. ^[3]). Achieving reliable and precise measurements of precipitation is both essential and challenging (Pipunic et al. ^[4]; Terink et al. ^[5]). Currently, there are three main methods of precipitation observation: ground station measurements, weather radar, and satellite sensors (Tapiador et al. ^[6]). Ground station measurements are the traditional observation method and are considered the most reliable precipitation data (Niu et al. ^[7]). However, due to the uneven distribution of rain gauges in China, the spatial interpolation method produces large errors in the spatially-continuous data, especially in areas with low station density and complex topography (Villarini et al. ^[8]). Meteorological radar offer high-resolution spatially-continuous precipitation observations, but the limited spatial coverage of radar, signal masking by complex terrain, signal attenuation, and inaccuracy of the reflectance-precipitation rate (Z-R) relationship lead to uncertainty in precipitation estimation (Crochet et al. ^[9]). In recent decades, centralized global multi-satellite-based precipitation products have emerged, providing new methods for rapidly acquiring continuous precipitation information (Sun et al. ^[10]). With its wide coverage, high spatial and temporal resolution, and independent topographic factors, satellite data have become an important data source for precipitation studies, providing reliable support for global and regional precipitation studies (Liu et al. ^[11]; Shen et al. ^[12]). Moreover, with the development of numerical models, reanalysis has also become an important way to obtain precipitation information. Reanalysis systems continuously simulate meteorological observations with many physical and dynamic models. Such products have near-global coverage, are often more spatially accurate than ground station data, and have longer span than satellite-based precipitation products (Copernicus et al. ^[13]; Amjad et al. ^[14]).

Given the different data sources and production principles of precipitation products, their adaptability under different spatial and temporal conditions can vary greatly. A thorough evaluation of their performance in terms of spatial and temporal distribution, accuracy, and frequency distribution is crucial for selecting the appropriate precipitation product for different studies and applications, and the development of multi-source precipitation product integration.

Many researchers have evaluated the applicability of different gridded precipitation products in different regions. Alijianian et al. ^[15] compared the performance of five daily precipitation products in the Iranian region and showed that MSWEP was in good agreement with the station data, followed by TRMM and PERSIANN-CDR. Mantas et al. ^[16] validated the TMPA product in the Peruvian Andes (3B42V7, 3B42RT) in terms of geographic factors, topographic conditions and climate. They showed that the dataset had a good agreement with surface precipitation. Amjadet et al. ^[17] evaluated the accuracy of four precipitation products in Turkey and found that IMERG, TMPA, and ERA-Interim underestimated observed precipitation in relatively wet areas and overestimated precipitation in relatively dry areas, while ERA5 consistently overestimated precipitation in relatively wet and topographically complex areas. Beck et al. ^[18] conducted a comprehensive performance assessment of 22 global gridded precipitation products in daily precipitation and showed that the MSWEP products had the best temporal correlation with the observed data. In the Chinese mainland, Li et al. ^[19] evaluated four satellite precipitation products, GSMaP_Gauge, GSMaP_NRT, IMERG, and MSWEP, and showed that MSWEP had the best spatial correlation for total annual precipitation, annual precipitation days, consecutive wet days, consecutive dry days and very wet days. In contrast, IMERG showed an advantage in maximum daily precipitation. Zhou et al. ^[20] evaluated the performance of the GPM and GSMaP precipitation products over the Chinese mainland and showed that all satellite precipitation products had similar spatial performances: underestimated precipitation in southern China and overestimated precipitation in northern China. Fu et al. ^[21] evaluated the error characteristics of four precipitation products (PERSIANN-CDR, MSWEP, GSMAP-gauge, and GPM) for 2016–2019 over the Chinese mainland, in which GSMAP-gauge and GPM performed best in reproducing the spatial distribution of precipitation, MSWEP had the best ability to record precipitation events, and PERSIANN-CDR performed the worst. Yuan et al. ^[22] evaluated the precipitation detection accuracy, and the spatial and temporal characteristics of ERA5, GLDAS, HAR V2 and TRMM on the Tibetan Plateau. The study shows that TRMM and GLDAS had better detection of daily precipitation events frequency but could not accurately detect daily precipitation events. ERA5 had better detection of daily precipitation events, but overestimated event frequency. Jiang et al. ^[23] evaluated the performance of CPC, IMERG, TMPA and ERA5 over the Chinese mainland in terms of multiple time scales and extreme precipitation estimation. The results show that the CPC and IMERG are better than TMPA and ERA5 over Chinese mainland. Guo et al. ^[24] evaluated eight satellite precipitation products on a seasonal and annual scale in the Xiangjiang River Basin of China, and the results show that GSMAP and MSWEP performed the best, while CHIRPS and SM2RAIN performed the worst. Li et al. ^[25] evaluated IMERG, MSWEP, and CMFD using observational data from 36 rainfall stations in Beijing, China from 2001 to 2016. The results show that MSWEP had the highest correlation coefficient between daily precipitation and station observations, with the smallest absolute deviation. CMFD had the strongest ability to accurately detect daily precipitation events. Hisam et al. ^[26] evaluated various monthly precipitation products such as PERSIANN and IMERG in Türkiye based on data from 193 ground meteorological stations. The results show that most products underestimated heavy precipitation, and all products performed better in detecting low and moderate precipitation. After evaluating the performance of GSMAP, IMERG, and CHIRPS on Bali, Liu et al. ^[27] found that GSMAP, IMERG, and CHIPRS all underestimated the frequency of light and heavy rainfall, but overestimated the frequency of moderate rainfall events. Wei et al. ^[28] systematically compared two versions of IMERG products with TRMM and CMORPH, and the results show that both IMERG products had better precipitation detection capabilities than TRMM and CMORPH.

While numerous studies have assessed high-resolution global precipitation data products, but most of them only focused on a certain type of precipitation data products and rarely made systematic comparisons based on ground stations, satellites, and reanalysis of three different types of precipitation data. Furthermore, most of the research areas selected by the research institutes were mostly global or a certain region in China, and the research on the seven geographical regions of China (North West, North China, South West, North East, Central China, East China, South China) was very insufficient. In terms of research methods, most of the research indicators were single evaluation error, correlation, and other indicators, and there were few studies on a comprehensive evaluation from multiple perspectives through different indicators.

Based on observational data from ground stations during 2000–2018, this paper evaluated the applicability of four daily precipitation products (MSWEP, ERA5, PERSIANN, and TRMM) in terms of spatial distribution and precipitation classes over the Chinese mainland and its seven regions. It is hoped that the results can provide a reference for applying precipitation products in China.

This paper chose the Chinese mainland as the study area for assessing multi-source precipitation. China is located on the west coast of the Pacific Ocean and the eastern part of the Asian and European continents. China has a vast area, high topography in the west, low topography in the east and a three-stage gradient distribution. China is the first to receive abundant precipitation from the southeast Asian summer monsoon that brings abundant water vapor to China. In contrast, the northwest region of China is deeply inland and more influenced by the topography. The overall precipitation trend in China is decreasing from southeast to northwest (Wei et al. ^[29]). To better evaluate the performance of precipitation products over China, this paper divides China into seven regions (Northeast China, North China, East China, Central China, South China, Northwest China, and Southwest China) according to topography, climate characteristics, and administrative divisions (Fig. 1) (Chen et al. ^[30]; Zhao et al. ^[31]).

Figure 1.  Map of the study area and distribution of ground observation stations within China.

This study evaluated four gridded precipitation products, including MSWEP, TRMM, PRESIANN, and ERA5. The main features of each gridded precipitation product are listed in Table 1.

MSWEP uses multi-source weighted fusion to produce a new multi-source enhanced integrated precipitation dataset by combining the advantages of station, satellite and reanalysis data (Beck et al. ^[32]). Station observations include CPC and GPCC, satellite data include CMOPRH, GSMaP-MVK and TMPA 3B42RT, and reanalysis data include ERA-Interim and JRA55. For each grid unit, the weight assigned to the estimated values based on the site is calculated based on the network density of the site, while the weight assigned to the estimated values based on satellites and reanalysis is calculated by comparing their performance at nearby sites. The concept behind the design is to optimize and merge high-quality precipitation datasets with time and position as functions, fully utilizing the complementarity of satellite, ground station, and reanalysis data. The data have a spatial and temporal resolution of 0.1° and 3 h, respectively. The updated version 2.8 has improved weighting values, longer records, a multi-source weather-compatible near-peak dataset, and near real-time estimates compatible with multi-source weather (Beck et al. ^[33]). The main steps in acquiring MSWEP include: (1) quality control of site data, (2) use of infrared precipitation estimates from the GridSat B1 archive to complement the TRMM satellite pre-use reanalysis and site data, (3) determination of weights for each input source, (4) determination of long-term average precipitation using WorldClim data, and (5) use of interpolated weights by weighting averaging the fused satellite and reanalysis precipitation datasets.

TRMM is a meteorological satellite jointly developed by NASA and JAXA to observe tropical and subtropical precipitation and energy exchange. The satellite carries the world's first precipitation radar and a variety of sensors such as scanners, imagers, and radiant energy systems for monitoring physical information about clouds (Liu et al. ^[34]). Although TRMM was forced to close in 2015, input data retrieval from other satellites within the constellation can still be used to continue updating the TRMM dataset. TRMM's three-level precipitation products TRMM 3B42 and TRMM 3B43 have been widely used in the study of precipitation spatio-temporal distribution. This article selected TRMM 3B42 V7 product, which has a spatial coverage range of 50°N–50°S and a spatial resolution of 0.25° × 0.25°, with a time resolution of 3 h. Compared with previous versions, 3B42 V7 has significant improvements, including an updated infrared brightness temperature dataset and additional satellite input, which can maximize data quality (Huffman et al. ^[35]).

PERSIANN is a satellite precipitation inversion product developed by the Center for Hydrometeorology and Remote Sensing. The product generates precipitation estimates using artificial neural networks by inversion of GridSat-B1 satellite infrared data of raster images and low-frequency instantaneous precipitation data. It uses the PERSIANN algorithm to process GridSat-B1 data and trains the artificial neural network using hourly precipitation data from the National Center for Environmental Prediction (NCEP) in the fourth stage. The dataset is available at different time scales ranging from 1 h to annual for the period 2000 to the present (Ashouri et al. ^[36]), with a spatial resolution of 0.25°×0.25°. The 6-h data were used in this study.

ERA5 is the fifth-generation reanalysis product launched by the European Centre for Medium-Range Weather Forecasts (ECMWF) after ERA-Interim. It benefits from the development of model physics, core dynamics, and data assimilation over the past decade, with higher spatial and temporal resolution than the ERA-Interim data. ERA5 achieves 0.25° × 0.25° grid covering the Earth and the dataset contains over 200 parameters, providing a large number of hourly atmospheric, terrestrial, and oceanic climate variables (Albergel et al. ^[37]). This product is based on the prediction system Cy41r2, using four-dimensional data assimilation technology and analyzing 137 atmospheres from the ground to an altitude of 80 kilometers. It has the advantages of high spatio-temporal resolution, fast updates, and multiple parameters (Hersbach et al. ^[38]). This article used the total precipitation data provided by it, which is the sum of large-scale precipitation and convective precipitation. Large-scale precipitation comes from cloud systems in integrated forecasting systems. The cloud system determines the situation of clouds and large-scale precipitation on a grid or larger scale based on changes in atmospheric quality.

Precipitation data from ground-based meteorological stations are usually considered "true values" and are used to evaluate multi-source precipitation products. In this paper, meteorological station data were obtained from the China Meteorological Data Service Center (http://data.cma.cn/en). The data are subject to strict quality control, such as polar value checking, spatial and temporal consistency checking, and manual verification and correction. Six hourly cumulative observed precipitation was collected from 2462 meteorological stations within the study area based on data continuity and temporal consistency from January 2000 to December 2018.

Since the four precipitation product data sets are based on Universal time (UTC) and the observational data of ground stations are based on Beijing time (UTC+8), in order to eliminate differences in time zone and time scale, time series matching of precipitation product data and ground station observation precipitation data was carried out as follows: when the precipitation product dataset was converted into Beijing time by adding 8 hours, the corresponding time of precipitation products and ground station observation precipitation data were selected respectively, and the daily precipitation data were accumulated.

Since the spatial resolution of the four precipitation products was inconsistent, the bilinear interpolation method was used to downscale TRMM, PERSIANN and ERA5 to 0.1°×0.1° to unify the spatial resolution of the precipitation products. This paper adopted the inverse distance weighting method to interpolate the precipitation product into the station data. This method weighted the grid points according to the distance between the station and the precipitation product. The farther the distance, the smaller the weight of the point, which is crucial for deriving the estimated value of the corresponding station, and matching the precipitation product with the station data.

This study used four continuous error statistical indicators to quantify the error in the precipitation products and four categorical statistical indicators to characterize the ability of the precipitation products to capture precipitation events (Sun et al. ^[39]) (Table 2). The statistical indicators included root mean square error (RMSE), mean absolute error (MAE), relative bias (RB), and coefficient of determination (R²). RMSE reflects the overall error level and fluctuation of the precipitation series, MAE reflects the true error between the precipitation products and observational data, and RB describes the degree of deviation between precipitation products and observed values, including positive (overestimation) and negative (underestimation) deviation. R² indicates the closeness of the correlation between precipitation products and station measurements. Classification statistics quantify the ability of precipitation products to detect real precipitation events and to identify precipitation events of different levels of intensity. The classification metrics included probability of detection (POD), false alarm rate (FAR), missed alarm rate (MAR), and critical success index (CSI). This paper used a POD of 0.01 mm d^-1 to indicate the probability of the precipitation products correctly capturing precipitation events. FAR indicates the ratio of false alarms of precipitation products to precipitation events and MAR indicates the ratio of underreporting of precipitation products to precipitation events. CSI reflects the comprehensive detection capability of precipitation products.

In Table 2, n represents the number of stations; S_i and G_i denote grid points from precipitation products and ground observation, respectively, and S̅ and G̅ denote the average value of all grid points from precipitation products and ground station, respectively. H_δ denotes that precipitation events are detected for precipitation products and observational data from ground stations under the δ threshold. M_δ indicates that precipitation events are detected for ground station data but not for precipitation products, and F_δ indicates that precipitation events are detected for precipitation products but not for ground station.

Figure 2 shows the spatial distribution of RMSE, CSI, POD, and FAR. The results show that the RMSE was consistent with the spatial distribution of precipitation for all four precipitation products, with a strong regional characteristic and an increasing trend from northwest to southeast. This may be due to the uneven distribution of observation stations within China. RMSE was more influenced by anomalous values. Lower RMSE was noted in the northwestern region, where the precipitation was smaller and less variable. PERSIANN had the worst overall performance, with a generally higher RMSE than the other three precipitation products, especially in Northeast, Central and South China. According to CSI results, the four precipitation products gradually showed a decreasing trend from South to Northwest China, which indicates that the precipitation products were better at detecting precipitation in the eastern region than the northwest region. MSWEP and ERA5 had the best CSI performance in East and Central China. The POD results show that MSWEP, ERA5, and TRMM had high hit rates in Northeast, North, Central, East, and South China, among which ERA5 had the highest hit rate for Central China. The FAR of the four precipitation products is 0.7–0.8 in the northwest and southwest regions, and they all overestimated light rain. In contrast, MSWEP had the lowest FAR in China, with the rate of only about 0.2 in East China.

Figure 2.  Spatial distribution of daily scale indicators of four precipitation products over the Chinese mainland from 2000 to 2018.

In this study, the results of RMSE, MAE, CSI and FAR evaluations of the four sets of products at the observation sites were statistically presented in terms of the overall distribution, as shown in Fig. 3, which is a box plot of the error statistics of the four sets of grid-point precipitation products, demonstrating the error distribution characteristics of the four performance metrics at the site scale. The results show that in terms of RMSE, the upper edge, the median, and the lower edge of MSWEP outperformed the other gridded precipitation products. For MAE, the upper and lower bounds of MSWEP were better than the other products, and the anomalies were concentrated in the 5 mm range. In terms of CSI, there was no significant difference between the median of ERA5 and MSWEP, but for the lower bound, MSWEP was slightly higher than ERA5. Moreover, both products performed better than PERSIANN and TRMM. In terms of FAR, PERSIANN had the highest FAR, and TRMM had a lower median than ERA5 and MSWEP. However, TRMM's upper and lower bounds were worse than these two products, indicating that TRMM fluctuated more in FAR and some precipitation events cannot be captured accurately. The box plots show that all four precipitation products had certain errors compared with ground observations. However, MSWEP and ERA5 performed better than PERSIANN and TRMM. Furthermore, there were relatively large outliers in MSWEP, but the product's performance was more stable and had a certain superiority over the upper and lower bounds and median.

Figure 3.  Box plots of error statistics for four precipitation products at the daily scale from 2000 to 2018.

The Taylor plot can comprehensively and intuitively compare the consistency between precipitation products and station observations. It is a polar coordinate plot that integrates three accuracy indicators: correlation coefficient (CC), standard deviation (STD), and root mean square error (RMSE), which can comprehensively evaluate the characteristics of precipitation products. The calculation formulas for CC and STD are as follows, and the RMSE calculation formula is shown in Table 1.

In Eqs. 1–2, S_i and G_i represent grid precipitation data and ground observation precipitation data, respectively; S And G represent the average precipitation data of all grid points and the average precipitation data of ground stations, respectively; x represents precipitation products or ground observational data.

To objectively and comprehensively compare the accuracy of four precipitation products in different regions of the country, the Taylor chart of four precipitation products in China and the Chinese mainland was drawn to analyze the spatial estimation ability of precipitation products. The scatter points in the Taylor plot represent precipitation products, the solid semi-circular arc with the origin of the Taylor plot as the center represents the standard deviation, the points falling on the horizontal axis represent station-based observational data, the blue dashed arc with the observational data as the center represents the RMSE, and the black radiation line emitted from the origin represents the correlation coefficient. The closer to the station, the higher the correlation coefficient with the benchmark data, the smaller the RMSE of centralization, and the closer the standard deviation, indicating a higher accuracy of precipitation, as shown in Fig. 4. In the northwest region, both MSWEP and TRMM were closest to the measured values. Still, the standard deviation of MSWEP was closer to the standard deviation of the measured values, making it the ideal dataset for the region. MSWEP was undoubtedly the best choice in North China, as it was closest to the measured values under all three indicators. In the Southwest region, TRMM performed the best among satellite-based precipitation products, and all three satellite precipitation products were superior to ERA5 products. In the Northeast region, the performance of MSWEP and TRMM was similar, both better than PERSIANN and ERA5. In Central China, the performance ranking of each product was consistent with that of the Northeast region, but the accuracy of each product was lower than that of the Northeast region. In East China, South China and the whole Chinese mainland, TRMM was slightly better than MSWEP, and its ERA5 was the worst among the four products. In summary, the performance of the three satellite precipitation products was better than that of ERA5, with MSWEP and TRMM performing well in the North and Southwest China regions respectively, and similar performance in other regions, but both were better than PERSIANN.

Figure 4.  The Taylor plot of four daily scale precipitation products in the seven regions and the Chinese mainland from 2000 to 2018.

Figure 5 shows the density scatter plots of MSWEP, TRMM, PRESIANN, and ERA5 against station data at the daily scale. MSWEP shows RMSE (4.01–9.71 mm), MAE (1.03–3.92 mm), R² (0.26–0.52), and linear fitting (0.49–0.57). Their optimal values occurred in the Northwest, North, East, and Northeast regions. Their worst values were found in the South, South, Northwest and Northwest region, respectively. MSWEP did not perform well in the Southwest region and performed better in the North and East China regions. PERSIANN showed RMSE, MAE, R² and linear fit of 5.49–12.24 mm, 2.01–4.51 mm, –0.38–0.18, and 0.20–0.31. Their optimal values were found in the Northwest, North, South and South China regions, respectively. Their worst values were found in the South, South, Northwest and Northwest China regions, respectively. PERSIANN did not perform well in the East China region and performed better in the North China region. The RMSE, MAE, R², and linear fit for ERA5 ranged between 3.36–10.65 mm, 1.09–4.36 mm, 0.29–0.48, and 0.41–0.64. Their optimal values were found in the Northwest region, while their worst values were found in the South, South, Southwest, and South China regions, respectively. It can be seen that ERA5 did not perform as well in the South China region and performed better in the Northwest China. TRMM presented RMSE, MAE, R² and linear fit of 4.02–10.42 mm, 1.29–4.14 mm, 0.22–0.39, and 0.55–0.64. Their optimal values occurred in the Northwest, Northwest, South, and Northeast regions and their worst values were found in the South, South, Southwest and South China region, respectively. TRMM did not perform well in North China and performed better in East China. After comparing the performance of the four precipitation products in China and its seven regions, we found that MSWEP and TRMM had a better overall performance than ERA5 and PERSIANN, and both performed better in the East China region. It is found that the overall performance of MSWEP and ERA5 was better than that of TRMM and PERSIANN, while PERSIANN performed the worst, especially for RMSE and R². Due to the small number of ground observation sites and low precipitation in the Northwest region, the performance of MSWEP in this region was not as good as that in other regions. In Northeast China, TRMM lacked part of the inverse precipitation results due to the limitation of its products, which overfitted precipitation in this region. The regression lines of all four precipitation products in each region were below the 1:1 line, indicating that the products underestimated daily precipitation.

Figure 5.  Scatter density map of four daily precipitation products and station observations over the Chinese mainland and seven regions from 2000 to 2018.

To complement the error assessment, the temporal variability of the four precipitation products and ground observation in 2018 were plotted for China and its seven regions (Fig. 6). The results show that PERSIANN significantly overestimated light precipitation in January and early February in Northeast, North and Northwest China, and deviated from the ground observations in China. MSWEP, ERA5, and TRMM can capture precipitation events well in East, South, Central and Northeast China, presenting an overall good agreement with ground observations. However, these three precipitation products often underestimated or overestimated the rainy season in China from June to August. Moreover, TRMM more obviously overestimated the dry months when the precipitation values were low. In the Northwest and Southwest regions, with few ground stations, only ERA5 had no significant deviation from observations, while MSWEP overestimated small precipitation in winter. In summary, the four precipitation products overestimated or underestimated the daily precipitation in different regions, with poor performance in the Northwest and Southwest regions. PERSIANN performed worse than the other three precipitation products in all regions and MSWEP outperformed the other three precipitation products.

Figure 6.  Changes in daily average precipitation over time for four sets of precipitation products and ground observations in 2018.

To explore the estimation difference of precipitation of each month by four sets of precipitation products, we studied and calculated the measured precipitation of ground stations and the average precipitation values of the corresponding four sets of precipitation products in January to December in the study interval of 2000 to 2018 in seven regions of China and the Chinese mainland. To facilitate the comparison of the differences between their values, they were plotted as a column chart. As shown in the Fig. 7, in the Northwest region, ERA5 and PERSIANN were significantly and slightly higher than the measured precipitation from January to December, respectively. TRMM showed significant underestimation in February and March, overestimated the measured precipitation in June, July, August, September, and November, and was very close to the measured precipitation in other months; MSWEP was the closest product to the measured value among the four sets of products, but there was a slight underestimation in most months. In the North China region, PERSIANN and ERA5 overestimated measured precipitation to varying degrees in all months, especially in June, July, and August. MSWEP and TRMM were the two products closest to measured precipitation, but both showed slight overestimation and underestimation in the summer months. In the Southwestern region, ERA5, PERSIANN, and MSWEP all significantly overestimated the measured precipitation, while TRMM also showed slight overestimation in other months except winter. In the Northeast, Central China, and East China regions, the estimated monthly precipitation values for all products were closest to the measured values compared with other regions. In the South China region, PERSIANN, TRMM, and ERA5 all showed varying degrees of overestimation from May to August, while MSWEP was consistent with the measured precipitation. From the perspective of the Chinese mainland as a whole, ERA5 had a large deviation from the measured value relative to other products in each month, and PERASIANN's performance was also inferior to the other two satellite products. In summary, over different zones, the performance of different products varied in different months. In the Northwest and Southwest regions, the differences between each product and the measured values in each month were greater than that in other regions. Moreover, in summer months, most products were prone to overestimating the measured precipitation.

Figure 7.  Comparison of the average monthly precipitation of four precipitation products and station-based observations in the Chinese mainland and seven regions from 2000 to 2018.

By analyzing the daily precipitation products, we found that apart from differences in precipitation in different regions, varying degrees of fluctuations in different months also existed. This suggests that the accuracy of the four precipitation products varied from month to month. Radar plots of the six accuracy evaluation metrics were plotted, where the precipitation threshold score was 0.1 mm (Fig. 8). The monthly radar plots show that the RMSE values of each precipitation product were significantly higher in June, July, and August than in other months. MSWEP had a higher RMSE than the other products in all months except for January and December. The RB results show that PERSIANN performed poorly, with significant overestimation and underestimation in different months, while MSWEP and TRMM performed more consistently and were close to zero in all months except for December, January, and February. PERSIANN had lower CSI values than the other three products in all months. TRMM performed poorly in the months with less precipitation (January, February, November, and December), but outperformed the other three precipitation products in July and August when precipitation was high. The POD showed lower values for TRMM and PERSIANN than ERA5 and MSWEP in all months. ERA5 outperformed MSWEP in January, February, November, and December, and there was no significant difference in the performance of the two products in other months. The FAR results show that PERSIANN had a higher value than the other three products in most months. The FAR of TRMM was better than that of the other three products in summer, but its performance was lower than that of MSWEP and ERA5 in winter. The MAR plot shows that ERA5 and MSWEP had lower rates than TRMM and PERSIANN in all months (< 0.1). Moreover, the MAR of TRMM was higher in winter, while the MAR of PERSIANN was higher in summer.

Figure 8.  Accuracy of four precipitation products over the Chinese mainland on a daily scale from 2000 to 2018.

This study followed the precipitation classification proposed by Jiang et al. ^[40] to group the data into four categories based on the precipitation intensity of ground observations: light rain (0.1–10 mm d^–1), moderate rain (10–25 mm d^–1), heavy rain (25–50 mm d^–1), and rainstorm (≥50 mm d^–1). The POD, SR, and CSI were calculated and compared for different precipitation intensities from the four precipitation products and ground observations during 2000–2018 as shown in Fig. 9.

Figure 9.  Comparison of four daily precipitation products under different precipitation intensities from 2000 to 2018. l: light rain, m: moderate rain, h: heavy rain, and rs: rainstorm.

The results for light rain show that the difference between MSWEP and ERA5 is negligible. TRMM had the highest success rate (SR) but low hit rate (POD), indicating that the product failed to report whether there was rain. Overall, all four products can detect light precipitation well. This is because most regional light precipitation events were usually generated by weather-scale systems with high detectability, indicating that the four precipitation products can better detect weather-scale systems. For moderate-intensity precipitation, the higher the intensity, the higher the error between the precipitation products and ground observations. In heavy precipitation events (20–50 mm d^–1), MSWEP and ERA5 had the highest success rate (MSWEP was higher than ERA5). However, TRMM had a better hit rate and its deviation (BIAS) was closer to the BIAS=1 line, indicating that TRMM had better detection capability than these two sets of products for heavy precipitation events and had a lower FAR. For rainstorms (≥50 mm d^–1), the deviation between TRMM and ground observation was around 1.0, and the deviation between ERA5, MSWEP and PERSIANN and ground observation was around 0.5. TRMM was closer to the BIAS=1 line, and the FAR was still lower than that of ERA5, MSWEP, and PERSIANN. In general, it can be seen that the test results of the four sets of precipitation products on light rain were distributed in the upper left or upper half of the performance diagram, with high false positive rate but low success rate. The test results of heavy rain and rainstorm were mainly distributed in the lower right part of the performance chart, and the phenomenon of missing information was obvious.

To better understand the assessment metrics at different precipitation intensities and geographical locations, we plotted the CSI skill scores to investigate the spatial distribution characteristics of different precipitation intensities for the four precipitation products in China (Fig. 10). It can be seen from the figure that the skill scores of the four precipitation products decreased as the precipitation intensity increased. The spatial distribution of CSI shows that four precipitation products had the strongest detection ability for light precipitation in the range of 0.5–0.7 in East China, South China, and Central China, followed by Northeast and North China. There were local areas in these two regions with scores of 0.4–0.5. The worst detection capability (0.4–0.5) was found in the northwest and southwest regions. Among the four precipitation products, MSWEP and PERSIANN had the best and worst detection capability for light precipitation, respectively. For heavy precipitation (25–50 mm d^–1) and rainstorm (≥50 mm d^–1), the four precipitation products still had a high score over Northeast, North, East, Central, and South China. For heavy precipitation, ERA5, TRMM, and MSWEP presented scores of 0.3–0.5, while TRMM performed better for heavy precipitation. Moreover, TRMM outperformed the other three precipitation products in some areas of Northwest and Southwest China.

Figure 10.  Spatial distribution of CSI of four precipitation products over the Chinese mainland on a daily scale under different precipitation intensities from 2000 to 2018.

This paper comprehensively analyzed the spatial and temporal accuracy and applicability of MSWEP, TRMM, PERSIANN, and ERA5 over the Chinese mainland and seven regions from the perspectives of spatial distribution characteristics, time series changes, and other perspectives using observational data from stations in the Chinese mainland from 2000 to 2018. The research results of this article will hopefully provide important references for appropriate precipitation products selection in meteorological applications. The main conclusions of this article are as follows:

According to the precipitation characteristics and precision performance of different types of precipitation products in China, it can be found that the performance of different types of precipitation datasets over the Chinese mainland and its different regions was different. According to the legend with conventional evaluation indicators, the product with strong comprehensive applicability in the Chinese mainland was MSWEP, which performed best, with RMSE (6.377) and MAE (2.058) being the lowest, R² (0.478) being the highest, TRMM and ERA5 taking second place, and PERSIANN performing poorly, whose RMSE and MAE were the largest in each partition. From different geographical regions, MSWEP performed better than the other three products in most regions, especially in North China and East China. The overall performance of all precipitation products in the Northwest and Southwest regions was the worst among all regions.

The accuracy of different types of precipitation products varied significantly with the month. The accuracy of the four precipitation products was relatively poor in summer months, with a large root mean square error (all greater than 9), and there was a significant overestimation phenomenon in the northwest region. The overall performance of MSWEP and TRMM in each region was better than the other two products, with estimated values similar to measured precipitation in each month, lower RB and higher CSI. PERSIANN and ERA5 performed poorly, showing varying degrees of overestimation and underestimation in different months, especially in the Northwest, Southwest, Northeast, and East China regions.

According to the analysis of accuracy under different rainfall intensities, the four products had good detection performance for light rain (0.1–10 mm d^–1) in the Chinese mainland, with POD of more than 50%, and high CSI in East, Central and South China. However as the precipitation intensity increased, the detection ability of the four precipitation products relatively weakened, and there was a significant difference in performance. MSWEP and ERA5 had demonstrated the ability to capture moderate rain (10–25 mm d^–1), especially in the Southeast and Southwest regions, with high hit rates and low false alarm rates, demonstrating certain advantages. The ability of TRMM to detect heavy rain (25–50 mm d^–1) and rainstorm (≥ 50 mm d^–1) was better than the other three products. The detection ability of PERSIANN for heavy rain and rainstorm was poor, showing very low POD and CSI, both lower than 20%, and there was a large system deviation.