Wind Speed and Altitude Dependent AMDAR Observational Error and Its Impacts on Data Assimilation and Forecasting

Aircraft Meteorological Data Relay (AMDAR) is a kind of weather report automatically obtained on commercial aircraft and transmitted to the ground receiving department by radio waves. Aircraft reports were ingested automatically from the Aircraft Communications Addressing and Reporting System (ACARS) originally ^[1]. Later ACARS and other communication systems formed the complete system named AMDAR ^[2-3]. Worldwide collaboration on AMDAR was established in 1998 and AMDAR weather report was later included in the World Meteorological Organization(WMO) Global Telecommunications System ^[4]. In 2002, China established the real-time collection and operating procedures of AMDAR reports ^[5-6].

Data assimilation is a useful technique to combine observations and model to provide better initial condition for numerical weather prediction (NWP) ^[7-8]. The ADMAR reports have high spatial and temporal resolution and play an important role in increasing the skill of analysis and subsequent forecasts at both regional and global scales for short-and medium-range forecasts ^[9-11].

Observational errors and background errors provide a weighting of observed values and background fields in the data assimilation system. Reasonable AMDAR observational error is a key part of data assimilation. It is a crucial step to correctly estimate the observational error of AMDAR before data assimilation. The observational error of aircraft reports is influenced by aircraft flight altitude and atmospheric conditions ^{[1], [3], [12]}. Benjamin estimated the wind and temperature observational errors for ACARS observations that collected over an area in the western and central United States for a 13-month period at different heights and indicated that observational errors decreased as the altitude increased^[13]. Ding compared temperature and wind observations between Chinese AMDAR reports and rawinsonde data, and the results demonstrated that the root-mean-square error (RMSE) between these two data of wind speed (direction) increasing (decreasing) with wind speed is a characteristic of AMDAR wind observation ^[3]. In the present Weather Research and Forecasting Model (WRF) data assimilation (WRFDA) system, the temperature and the observational error in the wind speeds of AMADR are constant values of 3.6 m s^-1 and 1 K, respectively. Therefore, the main purpose of this study is to estimate the wind speed and altitude dependent AMDAR observational error so as to improve the AMDAR data assimilation as well as the NWP.

The rest of the paper is arranged as follows. In section 2, the error statistics methodology and the data sources are introduced and wind speed and altitude dependent AMDAR observational error is also estimated in this section. A brief description of system setup and experimental design is presented in section 3. In section 4, single AMDAR observation tests that compare the new observational error with the default observational error is discussed. The results of the sensitivity experiments are discussed in section 5. Finally, conclusions and plans for future are provided in section 6.

The AMDAR reports used in this study are provided by China Meteorological Administration (CMA). There are quality control (QC) codes for the AMDAR reports. Quality control is not the focus of this article, and we just apply it there, so we don't know the specific quality control process. In addition, the WRFDA will remove data in which OMB exceeds the observational error by 5 times. The number of these AMDAR reports in the Chinese region is far more than that in the Global Telecommunication System (GTS). The spatial and temporal distributions of the AMDAR observations on 22 August 2017 are displayed in Fig. 1. It is found that there were more AMDAR observations in central and eastern China, especially near large cities (Fig. 1a). Fig. 1b shows the temporal distributions of the AMDAR observations on 22 August 2017. It can be seen that the AMDAR observations mainly concentrated between 0300 UTC and 1500 UTC.

Figure 1.  The distribution of AMDAR observations on 22 August 2017. (a) Spatial distribution (colors represent altitude), and (b) temporal distribution.

The frequency of temperature and wind speed differences between AMDAR and ERA-Interim from 0000 UTC 25 August 2017 to 1800 UTC 5 November 2017 are shown in Fig. 2. It is found that the frequencies follow the Gaussian distribution, with average values of -0.118 ℃ and 0.628 m s^-1, respectively.

Figure 2.  The frequency distribution of the (a) temperature, (and) b wind speed differences (AMDAR-ERA-Interim) during 0000 UTC 25 August 2017-1800 UTC 5 November 2017. Red solid lines represent the Gaussian distribution according to the mean values and standard deviations of the differences.

The procedure used to estimate observational error is to calculate the root mean square (RMS) differences between AMDAR and ERA Interim. The RMS differences at small spatial and temporal separation include contributions from observational error of both aircraft (σ_AMDAR1, σ_AMDAR2) and from mesoscale variability within that small separation^[13]:

where σ_total (h) is the RMS difference between two reports (AMDAR and ERA-Interim) at altitude (h) and σ_meso is mesoscale variability.

Researchers point out that the AMDAR observational error is also influenced by wind speed ^{[1, 3]}. Therefore, in the study, the observational error is expanded as a function of altitude and wind speed:

where σ_total(h, sp) is the RMS difference between AMDAR and ERA-Interim at altitude (h) and wind speed (sp), σ_meso is mesoscale variability and it can be ignored when the spatial and temporal separation between the two data is small. If the mesoscale variability is zero, Eq. (2) can be expressed as

Further assume that there is no correlated error between two AMDAR reports from two different aircrafts and the expected error from each aircraft is equal (σ_AMDAR1(h, sp)=σ_AMDAR2(h, sp)). The observational error for an individual aircraft may be estimated as

Employing Eq. (4), the observational error are calculated, using the AMDAR reports and ERA-Interim from 25 August 2017 to 5 November 2017. The change with height of the temperature and wind speed error for AMDAR are shown in Fig. 3. The WRFDA system provides default observations for various observations, and it is found that the default observational errors (black lines) of wind speed and temperature in the WRFDA for AMDAR are fixed as 3.6 m s^-1 and 1 K respectively. Fig. 3 indicates the magnitude of observational error of NOAA(red line). The observational error of NOAA was estimated by the same method as the one used in this article, but it was obtained by counting American aircraft reports ^[13]. The present study (Blue line) is similar to NOAA's study in that their values are both smaller than the default value in the WRFDA; the observational errors of this study and NOAA are both altitude dependent. There are only about 200 samples at the highest level. It may lead to unstable statistics, so there is a jump at high altitudes in the red line. And the area and amount of sample data will affect the statistical results of observational errors. The observational error of the red line is compared with sounding data, and the observational error of the red line is compared with ERA-interim. On the other hand, the sample of the observational error of the red line is from North America, which is somewhat different from China. Thus, there is a slight difference between the observational error of the red line and the observational error of the blue line.

Figure 3.  AMDAR observational error change with height. Black solid lines represent default value of WRFDA, red solid lines rep-resent statistical value of NOAA, and blue solid lines represent statistical value of this study. (a) Observational error in wind speed; (b) temperature observational error; (c) total number of AMDAR observations used for statistics at different altitude.

Figure 4 shows the variation of temperature and AMDAR wind speed observational errors with wind speed. It can be seen that the magnitude of the observational error in the wind speed of Ding (red line, the observational error of Ding was estimated by comparing sounding with AMDAR reports) and this study (blue line) are similar; the observational error in the wind speeds of this study and Ding are both smaller than the default value in the WRFDA (black line); the observational error in the wind speeds of Ding and this study both obviously increase with wind speed while the default observational error in the WRFDA is constant. However, the temperature observational error of this study has different trends compared with that of Ding.

Figure 4.  AMDAR observational error change with wind height. Black solid lines represent default value of WRFDA, red solid lines represent statistical value of Ding, and blue solid lines represent statistical value of this study. (a) Observational error in wind speed; (b) temperature observational error.

As discussed earlier, the AMDAR observational error may depend on wind speed and altitude. This study aims to design a set of wind speed and altitude dependent AMDAR observational error (Table 1), and replace the default AMDAR observational error in the WRFDA with the new observational error of Table 1. The new observational error scheme (Table 1) is speed and altitude dependent, so it has certain differences with Figs. 3 and 4. Almost at all heights, observational error in wind speed decreases with wind speed and slightly decreases with altitude. Observational error in temperature does not change significantly with the wind speed below 6km and decreases with the wind speed above 6km; observational error in temperature obviously decreases with altitude. In general, the observational error in wind speed changes more obviously with the wind speed, and the observational error in temperature changes more obviously with height.

The version 3.8 of the Advanced Research WRF Model (ARW-WRF, hereinafter WRF) ^[14] is used as the forecasting model in this study. All experiments are based on the WRF Data Assimilation (WRFDA) three-dimensional variational data (3DVAR) assimilation system. Doubly-nested domains with horizontal resolutions of 9 and 3 km, and with grid points of 649× 500 and 550 × 424 for outer domain and inner domain respectively are employed (Fig. 5). The domain is configured with 50 vertical levels and a 50hPa top. The main physics parameterization schemes include the WRF Single Moment 6-class (WSM6) microphysics scheme ^[15], the Yonsei University (YSU) boundary layer scheme ^[16], the Rapid Radiative Transfer Model (RRTM) longwave radiation scheme ^[17], the Dudhia shortwave radiation scheme ^[18] and the Noah land-surface model ^[19]. The Kain-Fritsch cumulus scheme ^[20] is only used in the outer domain.

Figure 5.  The domain for model forecasts and the distributions of the main observations in this domain (without AMDAR) at 1200 UTC 17 September 2017.

The background error covariance used in this study is calculated by the NMC method ^[21]. Two month 12h and 24 h forecasts from 1 September 2018 to 31 October 2018 are used to calculate background error covariance. The CV_UV is selected as control variable option. The control variables are U, V, Ps, T, and RHs. CV_UV did not consider the multivariate correlation. Researches show the control variables of CV__ψχ with that of CV_UV in the 3DVAR system and conclude that CV_UV performs better than the CV__ψχ in limited area convection-scale data assimilation^[22-23].

To investigate the influence of new AMDAR observational error on NWP forecasts, this study presents two groups of two-month 3-hourly cycling data assimilation and forecast experiments during 0000 UTC 1 September 2017-2100 UTC 31 October 2017 (Table 2). The CTL (Control) assimilates the AMDAR observations using the default observational error in WRFDA (σ - Default), and the NEW assimilates the AMDAR observations using new observational error (σ-NEW) (Table 1). The first forecast cycle uses the NCEP global forecast system (GFS) analysis interpolated onto the 9 km domain (Fig. 5) at 2100 UTC each day to create the initial conditions, and then spin-up to 0000 UTC the next day. Subsequent data assimilation cycles are run every 3h from 0000 UTC to 2100 UTC with the 24-h forecasts initialized at each cycle (Fig. 6). The 3-h forecasts of WRF issued from the previous analysis are used as the background.

Figure 6.  Flow chart of the cycling DA experiments.

To analyze the influence of new observational error (σ - NEW) on analysis, pseudo single AMDAR observations tests using σ - Default and σ - NEW are performed. For each experiment, two pseudo observation pairs are introduced at the model grid point. The observation-background (O-B) of four pseudo observations are set to be equal; thus, analysis increments can partly reflect the structure of observational error. To discuss the influence of altitude dependent observational error on data assimilation, two pseudo AMDAR observations for A and B with the same wind speed (6.3 m s^-1) are introduced at different model levels (Table 3). Similarly, two pseudo AMDAR observations for C and D at the same model level (12^th level) are introduced with different wind speed (Table 3) to reflect the influence of wind speed dependent observational error on data assimilation.

Figure 7 shows the vertical profiles of analysis increment of v-wind by assimilating pseudo AMDAR observations at different altitudes with the same wind speed. The analysis increment of point A and point B in the experiment NEW using σ - NEW (Fig. 7b) is larger than that in the experiment CTL using σ - Default (Fig. 7a), because the observational error in the wind speed of σ - NEW is smaller than that of σ - Default and the background error covariance used in CTL and NEW are the same. It is found that, in experiment CTL, the analysis increment of v-wind at point A is smaller than that at point B (Fig. 7a), indicating that the background error of point A is smaller than that of point B, because vertical observational error of the σ-Default is constant. Nevertheless, the analysis increment of v-wind at point B is smaller than that at point A in experiment NEW (Fig. 7b), though the background error of point A is smaller than that of point B. It is because that the observational error in the wind speed of the σ-NEW is decreasing with altitude, the observational error in point A is smaller than that of point B.

Figure 7.  The vertical profiles of analysis increment of v-wind by assimilating pseudo AMDAR observations at different altitudes with the same wind speed using σ-Default and σ-NEW respectively. (a) CTL, and (b) NEW. The O-B of v-wind are 1 m s^-1 of A and B.

Figure 8 shows the vertical analysis increment of temperature by assimilating pseudo AMDAR observations at different altitudes with the same wind speed. Similarly, it shows the difference in the size of temperature analysis increments since temperature observational error of σ-NEW is smaller than that of σ-Default, and the difference in analysis increments with altitude indicates that temperature observational error of σ-NEW is altitude dependent.

Figure 8.  The same as Fig. 7 but for temperature increment. (a) CTL, and (b) NEW. The O-B of temperature are 1 K of A and B.

Figure 9 shows the analysis increment of v-wind at the 12th level by assimilating pseudo AMDAR observations with different wind speeds at the same level. It can be seen that the analysis increment of point A and point B of v-wind in the experiment NEW using σ-NEW (Fig. 9b) is larger than that in the experiment CTL using σ - Default(Fig. 9a). This is because the observational error in the wind speed of σ - NEW is smaller than σ - Default since the domain averaged background error covariance is used in the study. It is found that the analysis increment of v-wind at point C is equal to that at point D (Fig. 9a) using σ - Default, because σ - Default is not wind speed dependent. However, it is found that the analysis increment of v-wind at point C is larger than that at point D using σ - NEW (Fig. 9b), indicating that the observational error of point C is smaller tthan that of point D. This is because the observational error in the wind speed of σ - NEW increases with the wind speed at this altitude (the σ - Default is constant).

Figure 9.  Analysis increment of v-wind at 12th level by assimilating pseudo AMDAR observations with different wind speeds at the same level using σ-Default and σ-NEW respectively. (a) CTL, and (b) NEW. The O-B of v-wind are 1 m s^-1 of C and D. Vector represents the wind field of the background.

Figure 10 displays the analysis increment of temperature at the 12th level by assimilating pseudo AMDAR observations with different wind speeds at the same level. Similarly, the difference in the size of temperature analysis increments are found since the temperature observational error of σ - NEW is smaller than that of σ - Default, and the difference in analysis increments with wind speed indicates that temperature observational error of σ-NEW is wind speed dependent. In short, the results of Fig. 7, Fig. 8, Fig. 9, and Fig. 10 indicate that altitude and wind speed dependent increments can be achieved due to the use of the altitude and wind speed dependent AMDAR observational error.

Figure 10.  The same as Fig. 9 but for temperature increment. (a) CTL, and (b) NEW. The O-B of temperature are 1 K of C and D.

In this section, two-month 3-hour cycle assimilation experiments (Table 2) are performed during 0000 UTC 1 September 2017-2100 UTC 31 October 2017. The root mean square error (RMSE) of analysis, 12 h forecast and 24 h forecast against ECMWF reanalysis and radiosonde observations are calculated, and the Fractions Skill Score (FSS) is also calculated to evaluate the precipitation forecast skill quantitatively.

The CV_UV is chosen as the control variable of background error covariance; therefore, there is no correlation between u, v and other variables in the background error covariance. Fig. 11 shows the 500hPa height field of the analysis field at 0000 UTC 16 September 2017. It can be seen that during the first assimilation process, although the wind speed observation error is different, there is almost no difference in the height field because there is no correlation between u, v and other variables in the background error. However, because the experiments use cyclic assimilation, the model will be adjusted during the forecast process, which will affect other variables. Fig. 12 shows the 500hPa height field of the analysis field at 1200 UTC 16 September 2017. Although the height field is not a direct influence variable, the 500hPa height field of the two experiments are significantly different after 5 cycles of assimilation and mode adjustment. Compared with the ERA-interim data, the height field of the CTL is significantly higher in south China. Although the NEW also has a high situation, it has a significant improvement compared with the CTL.

Figure 11.  Analyzed geopotential height (GHT) (shaded color, Units: m² s^-2), and wind speed (vector, Units: m s^-1) of 500hPa at 00: 00 on September 16 2017 from (a) ERA-interim, and experiments (b) CTL and (c) NEW.

Figure 12.  Analyzed geopotential height (GHT) (shaded color, Units: m² s^-2), and wind speed (vector, Units: m s^-1) of 500hPa at 12: 00 on September 16 2017 from (a) ERA-interim, and experiments (b) CTL and (c) NEW.

In order to evaluate the impact of the new observational error (σ - NEW) on assimilation and forecast, the root mean square error (RMSE) of analysis, 12-h forecast and 24-h forecast against ECMWF reanalysis are calculated. Fig. 10 shows the average vertical profiles of the RMSE scores for the two experiments including NEW and CTL which measure the differences of u-wind(U), v-wind(V), w-wind(WV), temperature(T), geo-potential height (Z) and water vapor (Q) between the outputs (analysis, 12-h forecasts and 24-h forecasts) of model and the ECMWF reanalysis respectively. The RMSE is the average value from 1 September to 31 October, 2017.

Figure 13a, b, c indicates that the RMSE for U, V and WV are smaller in NEW than that in CTL, with the most significant reductions at the altitudes of the 200hPa-700hPa. Compared with CTL, NEW improves wind forecasts. Most of the improvements from the NEW scheme in analyses still retain in the 24 h WRF forecast. Fig. 13d shows that the RMSE scores of NEW and CTL for temperature are comparable. This may be contributed to the fact that the wind and temperature analysis fields show different responses to the observational errors. The response is considerably much stronger relating to the increased observational errors in the wind analysis field ^[24]. Similarly, the RMSE scores of NEW and CTL for water vapor are comparable (Fig. 13f). This is probably because there is no humidity variable in the AMDAR observations. In Fig. 13e, the results for geo-potential height of NEW are clearly better than those of CTL. In general, NEW shows better results for wind fields and geo-potential height than CTL, but displays comparable results for water vapor and temperature to CTL.

Figure 13.  Averaged vertical RMSE profile of analysis(red), 12-(blue) and 24-h(black) WRF forecasts for NEW (solid line) and CTL (dashed line) against ECMWF analysis: (a) u-wind, (b) v-wind, (c) w-wind, (d) temperature, (e) geo-potential height, and (f) water vapor.

Model Evaluation Tools (MET) is a set of verification tools developed by the Developmental Testbed Center (DTC). It is to help users from the numerical weather forecasting community, especially users of the Weather Research and Forecasting(WRF) model, evaluate the performance of numerical weather predictions. In order to further evaluate the impact of the σ - NEW, the root mean square error (RMSE) scores of analysis, we also calculated 12 h forecast and 24 h forecast against radiosonde observations using MET. Fig. 14 presents the vertical profiles over the period of 2-month of the RMSE for the two experiments including NEW and CTL which measure the differences of u-wind (U), v-wind(V), geo-potential height(HGT) and temperature(T) between the outputs (analysis, 12 h forecasts and 24 h forecasts) of model and the radiosonde observations. The RMSE is the average value from 1 September 2017 to 31 October 2017.

Figure 14.  Averaged vertical RMSE profile of analysis (red), 12-(blue) and 24-h(black) WRF forecasts for NEW (solid line) and CTL (dashed line) against radiosonde observations: (a) u wind, (b) v wind, (c) geo-potential height and (d) temperature. The right hand side of the vertical axis presents the total number of observations used for generating the verification scores.

Similarly, the vertical profiles of the RMSE scores over the period of 2-month for the u-wind and v-wind also indicate that the NEW shows better results for wind than CTL including analysis, 12 h forecast and 24 h forecasts, especially between 700hPa and 200hPa. This may be the result of the large amount of AMDAR data between 700hPa and 200hPa. As the forecasting time increases, the difference of the two experiments become smaller. For geo-potential height variable, the NEW also shows smaller RMSE values than CTL does. The RMSE scores of 12 h forecasts and 24 h forecast for temperature in NEW are slightly smaller than that in CTL. However, the RMSE scores of analysis for temperature in NEW and CTL are compared. It suggests that the RMSE scores for the temperature variable in NEW grows slower than those in CTL.

Fractions Skill Score (FSS) is one of the neighborhood verification methods calculated to evaluate the precipitation forecast skill ^[25]. FSS has a range from 0 to 1. A score of 1 represents complete overlap between forecast and observed events, and a score of 0 means no overlap between forecast and observed events. In this study, 6-h accumulated precipitation is verified against the China Hourly Merged Precipitation Analysis (CHMPA) ^[26] from China Meteorological Administration, and the resolution of CHMPA data is 10km. The neighbor size used in calculating FSS is 6km and the domain for the FSS is d02.

Figure 15 shows averaged FSS over the period of 2-month for the forecasts of 6-h accumulated precipitation over different lead times with precipitation thresholds of (a) 0.1mm, (b) 4mm, (c) 13mm and (d) 25mm for CTL (blue bars) and NEW (red bars). Fig. 13a shows clearly the FSS of NEW is better than that of CTL with precipitation thresholds of 0.1mm. However, for 6-h accumulated precipitation greater than 4mm, FSS is larger in CTL than in NEW during the 12-h and 24-h of forecasts (Fig. 13b). Fig. 13c and d shows the FSS of NEW is higher than that of CTL with precipitation thresholds of 13mm and 25mm, and it suggests that the result of NEW is better than that of CTL for heavy rainfall forecasts. It can be concluded that new AMDAR observational error (σ-NEW) can improve the forecasts of rainfall.

Figure 15.  Average fractions skill score for the forecasts of 6-h accumulated precipitation over different lead times with precipitation thresholds of (a) 0.1mm, (b) 4mm, (c) 13mm and (d) 25mm for CTL (blue bars) and NEW (red bars).

Figure 16 shows averaged BIAS over the period of 2-month for the forecasts of 6-h accumulated precipitation over different lead times with precipitation thresholds of (a) 0.1mm, (b) 4mm, (c) 13mm and (d) 25mm for CTL (blue bars) and NEW (red bars). BIAS scores show that false alarm ratio of precipitation is high for two experiments. Compared with the CTL, NEW has BIAS scores closer to 1, which indicates that NEW reduces the false alarm ratio.

Figure 16.  Average bias score for the forecasts of 6-h accumulated precipitation over different lead times with precipitation thresh‐ olds of (a) 0.1mm, (b) 4mm, (c) 13mm and (d) 25mm for CTL (blue bars) and NEW (red bars).

The AMDAR sensor network provides temperature and wind observations with high spatial and temporal resolution. In this study, the altitude and wind speed dependent AMDAR observational error is estimated. Pseudo single AMDAR observation tests and two-month 3-hourly cycling data assimilation and forecast experiments are performed based on the Weather Research and Forecasting Model (WRF) and its Data Assimilation system (WRFDA).

The statistical results show that the AMDAR observational error is altitude and wind speed dependent. The observational error in the temperature and wind speed of AMDAR slightly decreases with altitude increase. A typical characteristic of AMDAR observation is that the observational error in wind speed of AMDAR increases with the magnitude of wind speed. Therefore, a new observational error that is altitude and wind speed dependent is identified.

The results of Pseudo AMDAR observation assimilation tests show altitudes and wind speeds dependent increments can be achieved with the use of the altitude and wind speed dependent AMDAR observational error. New observational error is able to provide more reasonable analysis increments than the default observational error in WRFDA does.

The root mean square error (RMSE) of two-month 3-hourly cycling data assimilation experiments indicates that the experiment NEW using altitude and wind speed dependent AMDAR observational error has superiority over the experiment CTL for wind field and temperature field. There is less impact of the new observational error scheme on the humidity field. FSSs of the two-month cycling for rainfall forecasts also indicate that the use of new observational error has a positive impact on the precipitation forecasting skill. The enhancement for heavy rainfall is more noticeable.

These conclusions highlight the value of the altitude and wind speed dependent AMDAR observational error in data assimilation for prediction of atmospheric variables and precipitation. However, the wind direction error of AMDAR used in this study is the default value in WRFDA. Wind direction error will be analyzed in our next step. Additionally, the characteristic of AMDAR observational error in different seasons will also need to be analyzed.