Atmos. Meas. Tech., 7, 2147-2158, 2014 www.atmos-meas-tech.net/7/2147/2014/ doi:10.5194/amt-7-2147-2014 © Author(s) 2014. CC Attribution 3.0 License.

Atmospheric f Measurement I Techniques

Validation of GOMOS ozone precision estimates in the stratosphere

V. F. Sofieva1, J. Tamminen1, E. Kyrola1, A. Laeng2, T. von Clarmann2, F. Dalaudier3, A. Hauchecorne3, J.-L. Bertaux3, G. Barrot4, L. Blanot4, D. Fussen5, and F. Vanhellemont5

1 Finnish Meteorological Institute, Helsinki, Finland

2Karlsruhe Institute of Technology, Institute for Meteorology and Climate Research, Germany

3Universite Versailles St-Quentin, UPMC University Paris 06, CNRS/INSU, LATMOS-IPSL, 78280 Guyancourt, France 4ACRI-ST, Sophia-Antipolis, France

5Belgian Institute for Space Aeronomy (IASB-BIRA), Brussels, Belgium Correspondence to: V F. Sofieva (viktoria.sofieva@fmi.fi)

Received: 19 February 2014 - Published in Atmos. Meas. Tech. Discuss.: 12 March 2014 Revised: 28 May 2014 - Accepted: 3 June 2014 - Published: 16 July 2014

Abstract. Accurate information about uncertainties is required in nearly all data analyses, e.g., inter-comparisons, data assimilation, combined use. Validation of precision estimates (viz., the random component of estimated uncertainty) is important for remote sensing measurements, which provide the information about atmospheric parameters by solving an inverse problem. For the Global Ozone Monitoring by Occultation of Stars (GOMOS) instrument, this is a real challenge, due to the dependence of the signal-to-noise ratio (and thus precision estimates) on stellar properties, small number of self-collocated measurements, and growing noise as a function of time due to instrument aging. The estimated ozone uncertainties are small in the stratosphere for bright star occultations, which complicates validation of precision values, given the natural ozone variability.

In this paper, we discuss different methods for geophysical validation of precision estimates and their applicability to GOMOS data. We propose a simple method for validation of GOMOS precision estimates for ozone in the stratosphere. This method is based on comparisons of differences in sample variance with differences in uncertainty estimates for measurements from different stars selected in a region of small natural variability.

For GOMOS, the difference in sample variances for different stars at tangent altitudes 25-45 km is well explained by the difference in squared precisions, if the stars are not dim. Since this is observed for several stars, and since normalized X 2 is close to 1 for these occultations in the stratosphere, we conclude that the GOMOS precision estimates are realistic in occultations of sufficiently bright stars. For dim stars, errors

are overestimated due to improper accounting of the dark charge correction uncertainty in the error budget. The proposed method can also be applied to stratospheric ozone data from other instruments, including multi-instrument analyses.

1 Introduction

Nearly all data analyses, e.g., data comparisons, data assimilation, and combined use, require information about data uncertainty. Validation of precision estimates (viz., the random component of the estimated uncertainty) is needed when the measurement uncertainty cannot be fully characterized or is based on assumptions. This is especially important for remote-sensing measurements, which use retrievals of atmo-

spheric parameters by solving inverse problems. Precision of the remote sensing measurements is usually estimated via propagation of instrumental noise through the inversion algorithm. These precision estimates can be imperfect due to incomplete forward models or retrieval approximations.

This paper is dedicated to validation of stratospheric ozone precision estimates for the Global Ozone Monitoring by Occultation of Stars (GOMOS) instrument. GOMOS is a stellar occultation instrument onboard the Envisat satellite (Bertaux et al., 2010; Kyrola et al., 2010), which operated in 20022012. Vertical profiles of ozone, NO2, NO3 and aerosol extinction are retrieved from ultraviolet and visible (UV-VIS)

stellar spectra measured as a star sets behind the Earth's limb, with a sampling frequency of 2 Hz. The spectra observed through the atmosphere are normalized by the reference

stellar spectrum observed above the atmosphere thus giving self-calibrated transmission spectra (transmittances), the basis for retrievals of trace gases from GOMOS measurements (Bertaux et al., 2010; Kyrola et al., 2010). The GOMOS inversion of chemical composition is performed in two steps (Kyrola et al., 2010). First, atmospheric transmission data from every tangent height are inverted to horizontal column densities (along the line of sight) for gases and optical thickness for aerosols (spectral inversion). Then, for every constituent, the collection of the horizontal column densities at successive tangent heights is inverted to vertical density profiles (vertical inversion). Although the measurements are performed during night and day, only nighttime measurements have been used in scientific studies so far, as the scattered solar light significantly degrades the quality and altitude coverage of daytime occultations. In this paper, only nighttime measurements (with solar zenith angles larger than 107° at the tangent points) are discussed. GOMOS has performed about 150-200 nighttime occultations per day with global coverage. Typical examples of GOMOS data coverage and distribution over the globe are shown, for example, in Bertaux et al. (2010), Kyrola et al. (2010), Sofieva et al. (2013), and Tamminen et al. (2010). The vertical resolution (including the smoothing properties of the inversion) of GOMOS ozone profiles is 2 km below 30 km and 3 km above 40 km; it is the same for all occultations.

In this paper, we discuss challenges in the validation of GOMOS ozone precision estimates and propose a simple method that allows for such validation in the stratosphere. The paper is organized as follows. Section 2 outlines the principles of the GOMOS precision derivation. In Sect. 3, we review the existing methods for validation of precision estimates and discuss their applicability to GOMOS measurements. In Sect. 4, we describe the proposed differential method for validation of precision estimates, present its application to GOMOS ozone profiles in the stratosphere and discuss its extension for other instruments. The summary concludes the paper.

2 Outlines of GOMOS precision derivation and characterization of retrieval quality

For GOMOS, the random component dominates in the total error budget. For nighttime occultations, instrumental noise consists of three components: photon noise, the dark charge of the charge-coupled devices (CCD), and readout noise (Bertaux et al., 2010; Kyrola et al., 2010). Statistics of pho-tocounts obey a Poisson distribution, which can be approximated to good accuracy by a normal distribution due to large values of photocounts. In the GOMOS processing, the mean dark charge is estimated and subtracted from the recorded signal as an offset signal, but its variance is taken into account in the noise term. The dark charge increases with time due to instrument aging (Tamminen et al., 2010). Its relative

contribution to the noise budget is larger for dim stars and at lower altitudes due to the attenuation of stellar flux caused by the atmosphere. Note that, at the stellar spectra level, dark charge and readout noise are additive - they do not depend on the mean stellar signal. However, at the transmission spectra level (after dividing by the reference spectrum), all noise components become non-additive: they depend strongly on the stellar signal (the noise is small for bright stars and larger for dimmer stars).

In the stratosphere, the main source of the modeling error is the incomplete scintillation correction, which is of a random nature. The residual scintillation error (i.e., the uncertainty due to incomplete scintillation correction) has been characterized in depth (Sofieva et al., 2009) and has been included in the inversion algorithm (Sofieva et al., 2010). The residual scintillation error depends on the obliquity of occultation; it vanishes for vertical (in-orbital-plane) occultations. Both the instrumental noise and the residual scintillation error are then propagated through the inversion (for details, see Kyrola et al., 2010; Sofieva et al., 2010; Tamminen et al., 2010), and the uncertainties of the retrieved profiles are given by the corresponding covariance matrices. The square roots of their diagonal elements are often referred to (also in this paper) as precision estimates.

The normalized x2 statistics, x^im, is commonly used for validating the adequacy of the theoretical description of measurements (forward model) and as an indication of the correctness of error estimates. For GOMOS, x12orm after the spectral inversion is

(T - T mod)T C 1 (T - T mod),

where T is the vector of observed transmittances, Tmod is the vector of modeled (theoretical) transmittances, C is the covariance matrix of measurement errors, N is the number of measurements and p is the number of retrieved parameters. If the theoretical model describes the experimental data well and the measurement errors are properly defined, xn2orm ^ 1. For GOMOS v.6 data, x,2orm is very close to unity in the stratosphere (Fig. 1) thus indicating realistic uncertainty characterization in this altitude range. Some increase of xn2orm at lower altitudes is quite expected and is related mainly to uncertainties in aerosol modeling. For dim and medium-brightness stars, xn2orm can be even smaller than 1 in the mesosphere. This indicates overestimated instrumental noise and can result in overestimated uncertainty of the retrieved profiles. Overestimation of ozone uncertainties for dim stars is confirmed also by our validation analysis of precision estimates, as presented below in the paper.

3 Methods for validation of precision estimates

In this section, we review existing validation methods and discuss their applicability to GOMOS measurements.

Figure 1. Xnorm for stars of different magnitudes. Dots: individual values, solid lines: median profiles, dashed lines: 16th and 84th percentiles. Number of occultations is specified in the figure. The 2008 data in the Northern Hemisphere are used.

3.1 Approaches to validation of precision estimates for atmospheric measurements

In the laboratory, the experimental precision estimates can be obtained using repeated measurements under the same conditions: the sample variance s2 = var(x) approaches the variance of random error distribution a2 (i.e., squared precision) when the size of sample N tends to infinity. The sample variance has a x2 distribution with N - 1 degrees of freedom. It can be approximated for large N by a Gaussian distribution with variance:

var(s2) « a4 —,

giving the uncertainty of the experimentally estimated precision.

Contrary to laboratory experiments, geophysical observation conditions cannot be kept exactly constant for atmospheric measurements. Therefore, the sample variance contains a contribution due to the natural variability o^t: s2 = a 2 + an2at. For validation of uncertainty estimates, a^ should be minimized by selecting collocated measurements or it should be known from independent sources.

Approaches to validation of error estimates usually rely on the variance of the difference s^2 in a set of collocated measurements x1 and x2:

(2 \ 2 2 2 (X1 - X2) j - (X1 - X2> = ao,nat + a1 +

In Eq. (3),

o ,nat

stands for the natural variability within a

space-time collocation window (note that agnat is different from a^), and the angular brackets denote the mean. Three methods for precision validation have been developed and applied.

Method 1

For perfectly collocated measurements (oq nat ^ 0) from the same instrument with the same precisions o1 = a2 = a, Eq. (3) is reduced to s22 ^ 2a2, thus allowing validation of the uncertainty estimate a. This method was realized, for example, for closely collocated MIPAS and OSIRIS measurements (Bourassa et al., 2012; Piccolo and Dudhia, 2007). The uncertainty of this experimental precision estimate a2 =

/2 is defined by the uncertainty of sample variance s

Method 2

Fioletov et al. (2006) have proposed estimating simultaneously the measurement precision and natural variability from sample variances of two perfectly collocated data sets and the variance of their difference (see Fioletov et al., 2006, and Appendix A for details). Since the precision estimates by the Fioletov method are linear combinations of three sample variances, they can have large uncertainty if one of the sample variances is large and/or the number of collocated measurements is limited (Appendix A; Fioletov et al., 2006). It is nearly impossible to select measurements exactly at the same location and time (perfectly collocated). In practice, satellite measurements separated by a few hundreds of kilometers and a few hours are considered collocated. The natural variability within the space-time collocation window is small but not zero. This results in additional difficulties in the application of this method, as observed by Bourassa et al. (2012). The small-scale natural variability also disturbs application of the Method 1.

Method 3

Provided many collocated measurements from the same instrument are available (self-collocations), the precision of

the data set can be estimated also by computing a two-dimensional structure function, or the rms difference as a function of increasing separation in time and in space. Then the limit at zero spatiotemporal mismatch will define the measurement precision. This method has been applied to validation of radio-occultation measurements by Constellation Observing System for Meteorology, Ionosphere, and Climate (COSMIC) (Staten and Reichler, 2009), which consists of identical instruments on board of six microsatellites.

For a single instrument, self-collocated data can be successive measurements from the same orbit, or measurements from different (successive) orbits. An analogous method (evaluation of the 1-D structure function) has been applied for validation of precision estimates of the MIPAS (Michel-son Interferometer for Passive Atmospheric Sounding) instrument onboard the Envisat satellite (Laeng et al., 2014).

3.2 Specifics of GOMOS measurements and challenges for precision validation

The stellar flux recorded by GOMOS - and thus signal-to-noise ratio and precision of retrieved profiles - depends on the magnitude and spectral class of the observed star, which means that the measurement precision varies substantially over the data set. Precision of retrieved profiles depends slightly on obliquity of occultation due to the influence of scintillations (Sofieva et al., 2009). The GOMOS error estimates also slightly depend on ozone concentration, but this dependence is much weaker than the dependence on stellar magnitude and the spectral class. In addition, GOMOS uncertainty estimates grow with instrument age (Tamminen et al., 2010). These features present some challenges for the validation of the GOMOS precision.

The number of self-collocated GOMOS measurements which are separated by less than 300 km in space and by less than 3 h in time is very limited. A sufficient amount (~ 200 per year) of collocated occultations of the same star, which would potentially allow a statistically significant validation of precision estimates by methods 1 or 3, are available only for one star, namely star number 30 in the GOMOS catalogue (S30, we will use similar notations for stars in the GOMOS catalogue hereafter). However, S30 collocations occur during winter close to the North Pole where the natural variability of ozone is large. In addition, the measurements in limb-viewing geometry are not local but horizontally integrated with effective horizontal extent ~ 300 km; therefore, GOMOS measurements with the same tangent point but at different azimuth angles are not expected to be exactly the same. We applied a variant of Method 3 to these GOMOS ozone profiles (the details of the analysis are presented in Appendix B) and, as expected, found that a significant excess of natural variability over the GOMOS precision estimates for S30 combined with the absence of very close collocations do not allow us to make definitive conclusions about the quality of GOMOS precision estimates (Appendix B).

Other GOMOS self-collocations (less than 30 per year) are for different stars in successive orbits (Guirlet et al., 2006) in years 2002-2004.

The estimated retrieval uncertainty for GOMOS ozone is small in the stratosphere; for very bright stars it is ~ 0.5-2 % (Sofieva et al., 2009; Tamminen et al., 2010). For an experimental estimation of precision in such occultations by the Fioletov method, a very large number of collocations with measurements from another instrument is needed, even in regions of low natural variability (Appendix B). For example, in equatorial regions with natural ozone variability ~ 5 %, in order to estimate/validate the GOMOS precision ~ 1 % with uncertainty of 50 %, more than 2500 collocated and accurate measurements would be needed. Such a number of collocations for each GOMOS star (or similar stars) is not available, even for satellite instruments with dense sampling like the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS) on board Envisat or the Microwave Limb Sounder (MLS) onboard EOS-Aura. For example, at latitudes 20° S-20° N, there are only ~ 500 collocations per year for MIPAS measurements using the 10 brightest GOMOS stars. Consequently, the method of Fioletov et al. (2006) cannot be applied to validation of GOMOS precision estimates either.

A simple method that allows for the validation of GOMOS ozone precision estimates is presented below.

4 A differential method for validation of GOMOS ozone precision estimates

4.1 Description of the method

If the measurements are selected in a region of small and slowly changing natural variability, then the sample variances corresponding to different data sets i (for GOMOS, corresponding to different stars) can be written as

S2 = <at + of. (4)

If the precision estimates are correct, then the difference in sample variance will be equal to the difference in precision estimates, s2 — s2 = of — of. The term o^ cancels out because it is assumed to be the same for both samples.

At the same time, si2 provides the upper limit for experimental estimates of measurement precision, as s2 > o2. If

°i ^ 0,then s2 ^ °n2at.

Important conditions for successful application of this method (which we call "differential method" hereafter) include the following:

a. Natural variability should be the same for the samples i and j .

b. Natural variability should not be large compared to the precision estimates, otherwise the sample variance estimates will have large uncertainty. This condition of

small natural variability is satisfied for ozone in the tropical stratosphere and in the summer stratosphere at other latitudes.

c. Measurements in each sample should be of the same precision.

4.2 Data and results

For an initial analysis, we selected tropical occultations (20° S-20° N) in the year 2008. The restriction to 1 year was made in order to avoid multi-year variability due to quasi-biennial oscillation. Only stars for which more than 200 occultations were available are considered. The latitudes of these occultations are shown in Fig. 2, and the information about these occultations is presented in Table 1. A particular star is available only for 2-3 months; its successive occultations are at approximately the same latitude and the same local time. The GOMOS measurement scheme provides naturally homogeneous longitudinal (zonal) coverage. For other years, the sampling pattern is very similar. The outliers and suspicious data were removed by the screening procedure presented in Appendix C.

The comparison of sample variances and precision estimates is presented in Fig. 3 for very bright stars S4 and S10 (left), and star S4 and the dim star S134 (right). As seen in Fig. 3 (left), the sample variances for the two bright stars S4 and S10 are very close to each other and show specific variations with height. The sample variance in S4 occultations is slightly larger than for S10 occultations, and this difference is fully accounted for by the difference in squared precisions (although S4 is brighter than S10, uncertainty estimates are larger for S4 because these stars have different effective temperatures). The uncertainty (rms) of sample variance shown by the error bars in Fig. 3 has been estimated according to Eq. (2). Very small error bars on squared precision curves (dashed lines) show the standard deviation of the precision distribution, which is very small for ozone in the tropics. The analogous comparison for the bright star S4 and the dim star S134 shows very similar features: the sample variance for S134 occultations is larger than that for S4 by an amount approximately equal to the difference in the corresponding squared precision estimates. Figure 4 compares the difference in the sample variance - s? with the difference in squared precisions ct| - a12, for different pairs of stars. In this comparison, time intervals are restricted in such a way that the spatiotemporal coverage by the selected stars is as close as possible. In particular, the time interval from 10 September to 20 October is taken for stars 2 and 161, and the interval from 15 August to 28 October for stars 9 and 143, etc. The difference in sample variance is in good agreement with the difference in precision estimates for not dim stars. This can serve as an indicator of correctness of these GOMOS precision estimates, in combination with the upper limit of uncertainty given by the sample variances and

Table 1. Information about the GOMOS occultations used for the analyses (years 2003, 2007, 2008). The obliquity angles are practically the same for different years. The occultations with large obliquity angles (> 60°) were performed in 2003.

Effective Mean

Visual temperature obliquity

Star_id magnitude (K) (deg)

2 -0.7 7000 49

4 0 5800 34

9 0.5 24 000 42

10 0.6 28 000 38

12 0.8 30 000 31

18 1.2 9700 70

29 1.6 10200 7

31 1.7 15 200 55

38 1.8 11 000 68

41 1.9 4100 39

43 1.9 4250 17

45 1.9 26 000 42

63 2.1 2800 55

71 2.2 7000 39

84 2.4 4500 62

113 2.7 5000 53

124 2.8 30 000 25

134 2.8 6600 29

135 2.8 5800 1

141 2.8 4600 40

143 2.9 7200 35

148 2.9 4100 32

157 2.9 9300 62

159 2.9 7200 30

161 2.9 4500 49

proximity of /¿»m to 1. In the comparisons with dim stars (cases S9 & S143 and S4 & S134), the difference in squared precision estimates is somewhat larger than the difference in sample variances, which might indicate an overestimation of error bars for dim stars (see also discussion below).

If the GOMOS ozone error estimates are correct, then the estimates of natural variability <rl2at computed from occultations of each star

-2 2 2 °nat = si -

should be approximately the same (provided the real natural variability in the corresponding samples in the equatorial stratosphere is the same). Figure 5 (top) shows estimates of natural variability in the latitude zone 20o S-20o N at altitudes 25-40 km inferred from the sample variance and precision estimates for different stars, which are plotted as a function of precisions a¡, for the considered year 2008. For this analysis, all measurements shown in Fig. 2 are used. The dashed black line shows <rl2at 0 estimated from the limit s2 as a2 ^ 0 and technically computed as a weighted mean of anat estimates using the seven brightest stars:

Jun Jul Aug

Date in 2008

Figure 2. Locations of occultations selected for the analysis. Each color corresponds to a particular star, whose number in the GOMOS catalogue is written near the corresponding line.

0 10 20 30 40 50 60 70

s2 or o2, (%) 2

s2 or o2, (%) 2

Figure 3. Sample variances s2 and the estimated precision variances a2of GOMOS ozone profiles (left) for stars 4 and 10 in January-March 2008; (right) for stars 4 and 134.

Jnat,0

= E Wi (s2 - -2),

where the weights w; are inversely proportional to the uncertainty of the parameter (s? - a2) estimated as described above (this section and Eq. 2). This estimated natural variability is approximately er?at 0 ^ 32.5 (%)?, or anat,0 ^ 5.7 %. As seen in Fig. 5 (top), many estimates of natural variability from individual stars agree within 2a uncertainty intervals with crj2at 0. This can serve as an indicator of the correctness of GOMOS error estimates for these cases. However, for three stars -S113, S135 and S143 - the estimates of natural variability become negative. This indicates an overestimation of error bars for these occultations. Another interesting feature is variations in natural variability, which can be related

to atmospheric processes or/and to non-uniform sampling by the measurements.

We have performed analogous estimations of natural variability also for 2007 and 2003 (center and bottom panels in Fig. 5). In 2007, the features are very similar to those observed for 2008: very close estimates of natural variability of a few percent for bright stars, and very low, even negative, estimates of natural variability variance for dim stars (stars S113, S134, S141, S143, S159 in 2007; stars S43, S134, S135 in 2003). Note that in 2003 the collection of stars is slightly different from that in 2007-2008, because the azimuth range for GOMOS measurements has been restricted after 2005 due to instrumental problems. Furthermore, uncertainty estimates are smaller for all stars in 2003 than in 2007 and 2008 (this aging effect is observed more clearly for dim stars due to a larger relative contribution of the dark charge uncertainty). It is interesting to note that the estimated

20 -20

1 / /' J-"-""'

i i i ......;......' (\......;...... : l : / : \ J——'—

/ li / / \ / Çs S4 & S10 V y S9 & S143

1 ) 1 / iL iV^ S31 & S63 X/ : : S4 & S134

i 11 ' 1 " !" » » i r /1 ........

s2-s1 and (%)2

Figure 4. Difference in sample variance — sj2 (solid lines) and the difference in precision variances avj — of (dashed lines), for different collocated pairs of stars specified in the legend.

natural variability (we consider here the estimates based on observations of seven brightest stars) is very similar in 2003 and 2007: 4.2 and 4.3 %, respectively; but it is larger in 2008, 5.7%.

4.3 Discussion

The proposed differential method allows for testing of the precision estimates in complicated cases like GOMOS ozone measurements, when collocated measurements are not available and the estimated precision is small. The requirement for application of this method is that natural variability is small and slowly varying.

Small additive errors (i.e., errors, which do not depend on stellar properties) in precisions cannot be detected by this method. Indeed, if precision estimates have an additive component of: o(2 = o.2 -

■ a2, a2 = a2 + a2, then it is canceled

2 ~2 22

out in the difference o.2 — o2-. In this case, it is impossible .j

to say whether of is realistic or not, if of is small enough so that the natural variability estimates s2 — of are positive. All dominating measurement uncertainty components (photon noise, dark charge) strongly depend on the stellar properties. The only exception is the residual scintillation error, which depends on obliquity of occultation (and does not depend on stellar properties); that is, it is additive. However, the residual scintillation error is relatively small, and it has an accurate parameterization in GOMOS retrievals (Sofieva et al., 2010). Therefore, it should not disturb the application of the differential method to GOMOS data (this is consistent with the results presented above).

The analysis of GOMOS occultations in the tropical stratosphere has shown that the difference in sample variances for different stars at altitudes 25-45 km is well explained by the

difference in precisions, if stars are not dim (visual magnitude less than 1.9). Since (i) this is observed for several stars and (ii) x12orm ^ 1 in these occultations in the stratosphere, we can conclude that GOMOS precision estimates are close to reality for such measurements.

For some dim (and cool) stars, the random error has been significantly overestimated, which results in negative estimates of natural variability variance using Eq. (5). This is also confirmed by x^im < 1 down to 30 km in occultations of dim stars. Careful inspection of the inversion algorithm has shown that, most likely, the reason for such behavior is imperfect estimation (a moderate overestimation) of dark charge variance. This overestimation of dark charge uncertainty is present for all occultations, but for bright stars this is insignificant, as the dark charge uncertainty has a small contribution to the total error budget.

The condition of small natural variability is not satisfied in the mesosphere and lower thermosphere (MLT). In addition, GOMOS error estimates for ozone are significantly smaller than the natural variability in the mesosphere; thus, estimating their difference from sample variances would become uncertain. However, there are reasons to trust the GOMOS precision estimates also at upper altitudes. First, the error estimates are based on the same method as in the stratosphere, and we see no reason why they should fail at upper altitudes. Second, xn2orm « 1 holds for the whole altitude range; thus, one would expect correctness of these error estimates also at upper altitudes.

The differential method for validation of precision estimates cannot be applied in the upper troposphere and the lower stratosphere either, because of large ozone variability in this region caused by variations in the tropopause height.

Provided conditions a-c (Sect. 4.1) are satisfied, the method described here can be also applied to other constituents. However, these conditions are often not satisfied for other constituents retrieved by GOMOS. Let us consider NO2 as an example. Condition c is violated for NO2. Precision estimates for NO2 exhibit rather large scatter (not shown here); therefore, small differences invariabilities are not detectable by precision estimates: they are within uncertainty intervals. For GOMOS, the differential method for precision validation is applicable mainly for ozone in the stratosphere, which was the primary scientific motivation for this instrument.

4.4 Extension for other instruments

The extension of the differential method for precision validation by including data from another instrument is quite straightforward: it can be considered as one more sample with corresponding precision estimates.

As an example, we compare the natural variability in the tropics, as estimated by GOMOS data for the seven brightest stars and by the MIPAS instrument on board the En-visat satellite. MIPAS is a Fourier transform spectrometer

160 180

(N — <f)

<0 O £=

>> —'

^ 9 10

113 10

12 431 ^29 41

10** £4-" 124 - — — - '135 - — _ _ _ _ _______

13 54^ 143 113 '141

384 ^29 411

1^931+45 *124 ^157 84

134| 135 43

6 8 10 uncertainty estimate o., %

Figure 5. Estimates of natural variability variance tr^at = s? — of at altitudes 25-40 km for different stars plotted as a function of precisions t. Star numbers in GOMOS catalogue are indicated. Colors indicate star magnitude: from red for bright stars (small magnitude, small star number) to blue for dim stars (large magnitude, large star number). The error bars are 2r uncertainty intervals. Dashed lines indicate the mean natural variability estimates obtained using seven brightest stars. Magenta lines highlight zero.

operating in the infrared, which provides vertical profiles of ozone and other trace gases. For our illustration, we have used the data processed with KIT IMK/IAA version V5R_O3_220/221 research processor (von Clarmann et al., 2003, 2009). MIPAS provides ozone profiles in the altitude range from 6 to 70 km with a vertical resolution of 2-5 km. In the tropical stratosphere, estimated precision of MIPAS ozone profiles is 1-3 %. Recent validation of MIPAS precision estimates using self-collocated measurements (methods 1 and 3 explained in Sect. 3.1) has shown that the MIPAS precision estimates are close to reality (Laeng et al., 2014).

Figure 6 shows profiles of natural variability estimates crnat obtained with Eq. (5) for latitudes 20° S-20° N using MIPAS nighttime data and GOMOS measurements of the seven brightest stars in 2008. As observed in Fig. 6, estimates of natural variability from GOMOS and from MIPAS are very

close to each other, and even the variations with altitude are similar. The values of mean natural variability in the altitude range 25-45 km are very close to each other: ~ 5.8 % for MIPAS and ~ 5.7 % for GOMOS. Such a good agreement can be considered as an additional confirmation that both MIPAS error bars and GOMOS error bars for bright stars are realistic.

5 Summary

In this paper, we have discussed validation of precision estimates for ozone profiles retrieved by the stellar occultation instrument GOMOS/Envisat. The validation of GOMOS precision estimates is challenging because of the dependence of instrumental noise (and thus uncertainty of

retrieved profiles) on stellar properties, insufficient number of GOMOS-GOMOS collocated measurements and small ozone retrieval uncertainties in case of bright stars. We have proposed a simple differential method for validation of GOMOS ozone precision estimates, which is based on comparisons of differences in sample variance with the difference in squared precision estimates, for measurements from different stars selected in a region of low natural variability.

The application of the proposed method to GOMOS ozone profiles in the stratosphere has shown that GOMOS precision estimates are realistic for non-dim stars (magnitude less than ~ 1.9). For dim stars, the uncertainties of retrieved ozone profiles are overestimated. The reason for this overestima-tion is now understood: it is related to improper accounting of the uncertainty associated with dark charge correction in the GOMOS error budget. This issue will be corrected in future GOMOS data processing (the correct error budget affects precision estimates mainly, while the changes in ozone values themselves are insignificant).

The application of the differential method to other altitude ranges and to other constituents is not readily possible due to violation of assumptions needed for the method to work. An extension to use of other instruments is quite natural, as illustrated here with GOMOS and MIPAS measurements. Estimates of ozone natural variability in the tropics from MIPAS and GOMOS using the seven brightest stars are very close to each other; thus, this provides additional confirmation of the correctness of the corresponding precision estimates, for both instruments.

-MIPAS -GOMOS, 7 brightest stars ---MIPAS, mean 25-45 km, 5.8% ---GOMOS, mean 25-45 km, 5.7%

fX " \ N. 11 V "i

11 II II II

natural variability, %

Figure 6. Profiles of natural variability at 20° S-20o N in 2008 (Eq. 5) using MIPAS nighttime data and GOMOS data from occultations of seven brightest stars. Vertical dashed lines indicate the mean natural variability over the altitude range 25-45 km.

Appendix A: Uncertainty of precision estimates by the Fioletov method

Appendix B: Validation of GOMOS precision estimates using self-collocated data

Fioletov et al. (2006) have proposed a method that allows simultaneous estimates of measurement precision and natural variability based on (perfectly) collocated data from two instruments. This method relies on sample variances s2 of the collocated data:

S2 = CTn2at + ^ i = 1 2 and the variance of their difference:

S22 = ((xi - *2)2) = al2 + a2.

In Eqs. (A1) and (A2), o^at is natural variability and ct2 are measurement precisions. Solving Eqs. (A1) and (A2) for these parameters, we get their experimental estimates based on sample variance:

= 0.51

222 ^si + s2 - s12J

a2 = 0.5 (s2 - s2 + s^) ¿22 = 0.5 (s22 - s2 + s22.)

The uncertainty of the natural variability and precision estimates given by Eq. (A3) depend on uncertainty of sample variances, which depend, in turn, on sample variances themselves and the number of measurements. The estimates are thus only as accurate as the least accurate of these parameters. In approximation of large samples (when x 2 distribution for the sample variance can be approximated by a normal distribution with the variance given by Eq. 2), the variance of the estimates Eq. (A3) can be expressed in terms of "true" natural variability and precision variances o^, oj2 and o^ as

var(ôf) = var(â|) = var^t)

= 2N ((a"at + Ol2)2 + (an2at + 022)2 + (of + O^

with the following simple estimates for upper and lower limits:

N (°n4at + O14 + °24) < var(<°122,nat) 1 (on2at + Oj2 + O22)2 .

The estimates can be very uncertain in cases where the natural variability significantly exceeds the measurement precision, or in case of poor accuracy of the collocated data. For example, let o2 ^ o1 and onat/o1 = 5. Then

the relative uncertainty of precision estimate std(<O1) «

i ( l + an4t + ^f ), which is - 255 % for N = 100,

N V -i -i ' 85 % for N = 900 and - 5i % for N = 2500.

Relatively many self-collocated GOMOS occultations exist only for one star, S30 (magnitude 1.7, effective temperature 30 000 K), close to the North Pole in winter. In our analysis, we selected the ozone profiles retrieved from S30 occultations with ground separation d0 less than 300 km and time difference Ai less than 3 h. For the majority of collocated pairs, the time difference is ~ 100 min (one orbit). We have used data from years 2007-2008, in which the number of collocated profiles is maximal, ~ 200 profiles per year (a relatively short time period of 2 years has been used in order to avoid the effect of changes in error estimates with time due to instrument aging).

Figure B1 shows profiles of the parameter s12/V2, s12 being the sample standard deviation of the profile difference (Eq. 3) as a function of the distance d. The parameter s12/V2 approximates the experimental error estimates, which should converge to predicted error estimates when d ^ 0 (if the latter ones are correct). It is analogous to the integral of structure function, which is widely used in the theory of random functions (e.g., Yaglom, 1987). The distance d represent the separation of air parcels corresponding to collocated measurements with the advection of air masses taken into account. It is evaluated as d(z) = |d0(z) + v(z) ■ At|, where d0 (z) is the ground distance and v(z) are the profiles of wind speed from ERA Interim data at locations of GOMOS occultations.

As observed in Fig. B1, the experimental precision estimates s12/V2 decrease with decreasing separation of measurements, as expected, but, because very small separations are not covered with GOMOS data, they are larger than the predicted error estimates. A similar analysis of collocated MIPAS ozone data close to the North Pole in winter has shown a very similar behavior of experimental and estimated uncertainties (not shown here). However, when analyzing collocated MIPAS data at the North Pole in summer, the experimental error estimates converge practically down to the precision estimates, as shown in Laeng et al. (2014). This is a clear indication that the ozone natural variability, even at small scales (a few hundreds of kilometers), is not zero in winter polar regions. The magnitude of this small-scale natural variability ^/s22/2 - o2 is very similar in analyses of GOMOS and MIPAS data; it is ~ 3 % at 20-30 km increasing up to 5-6% at 40-50 km.

Absence of collocated GOMOS data in the regions of low natural variability does not allow a full validation of precision estimates according to Method 3, whose application is illustrated here. However, we observe a reasonable behavior of GOMOS precision estimates: they are lower than the sample variances of ozone profile differences.

d<100km d<150km d<200km d<250km d<300km d<400km error estimate

Table C1. Upper limit for allowed uncertainty and mixing ratio. All values larger than these thresholds are removed.

error estimates, %

Figure B1. Color lines: experimental precision estimates —2 for

different separation distances; black line: mean error estimate. GOMOS self-collocations of star S30 occultations close to the North Pole in 2007-2008 are used.

Appendix C: Screening of outliers in GOMOS data

The following three-step procedure has been applied for removing invalid or suspicious GOMOS data.

1. The whole profile is ignored if

i. values of mixing ratio > 15 ppmv or < -0.5 ppmv are reported at altitudes 25-45 km.

ii. number density >5109 cm-3 is reported at ozone minimum (77-80 km).

iii. value of mixing ratio >100 ppmv is reported at any altitude.

Any of i-iii conditions result in removing the whole profile. This is the filtering of clear outlier profiles.

Maximum Maximum mixing

Altitudes uncertainty ratio (modulus)

<18 km 70% 10 ppmv

18-65 km 30% 50 ppmv

>65 km 150% 50 ppmv

2. A value at a certain altitude level is filtered if

i. the value is flagged (http://earth.eo.esa.int/pcs/ envisat/gomos/documentation/04_Vol10_Gomos_ 3K_v7.pdf).

ii. error estimate or absolute value of mixing ratio exceed the threshold given in Table C1.

3. For stars with UV flux < 900 photons, all values above 42 km are removed.

UV flux at 275 nm was pre-computed for each star using the Planck law. This operation removes the upper part of the ozone profiles from occultations of dim and cool stars, which are unreliable at high altitudes due to insufficient signal-to-noise ratio in the ultraviolet (Kyrôlâ et al., 2006).

Finally, profiles having too few valid points (less than 10) or a valid altitude range smaller than 20 km in the stratosphere were ignored.

Note that this screening procedure differs from the more conservative one recommended in the GOMOS readme document http://earth.eo.esa.int/pcs/envisat/gomos/ documentation/RMF_0117_G0M_NL__2P_Disclaimers. pdf.

Acknowledgements. The work has been supported by the ESA Ozone Climate Change Initiative project. The work of the FMI team has been supported by the Academy of Finland (projects MIDAT, ASTREX and INQUIRE).

Edited by: R. Eckman References

Bertaux, J. L., Kyrölä, E., Fussen, D., Hauchecorne, A., Dalaudier, F., Sofieva, V., Tamminen, J., Vanhellemont, F., Fanton d'Andon, O., Barrot, G., Mangin, A., Blanot, L., Lebrun, J. C., Perot, K., Fehr, T., Saavedra, L., Leppelmeier, G. W., and Fraisse, R.: Global ozone monitoring by occultation of stars: an overview of GOMOS measurements on ENVISAT, Atmos. Chem. Phys., 10, 12091-12148, doi:10.5194/acp-10-12091-2010, 2010. Bourassa, A. E., McLinden, C. A., Bathgate, A. F., Elash, B. J., and Degenstein, D. A.: Precision estimate for Odin-OSIRIS limb scatter retrievals, J. Geophys. Res., 117, D04303, doi: 10.1029/2011JD016976, 2012. Fioletov, V. E., Tarasick, D. W., and Petropavlovskikh, I.: Estimating ozone variability and instrument uncertainties from SBUV(/2), ozonesonde, Umkehr, and SAGE II measurements: Short-term variations, J. Geophys. Res., 111, D02305, doi: 10.1029/2005JD006340, 2006. Guirlet, M., Dalaudier, F., Sofieva, V. F., and Hauchecorne, A.: Comparison of GOMOS Level 2 products for measurements in close coincidence, in Proceeding of Third Workshop on the Atmospheric Chemistry Validation of Envisat (ACVE-3), ESA/ESRIN., 2006. Kyrölä, E., Tamminen, J., Leppelmeier, G. W., Sofieva, V. F., Hassinen, S., Seppälä, A., Verronen, P. T., Bertaux, J.-L., Hauchecorne, A., Dalaudier, F., Fussen, D., Vanhellemont, F., D'Andon, O. F., Barrot, G., Mangin, A., Theodore, B., Guirlet, M., Koopman, R., de Miguel, L. S., Snoeij, P., Fehr, T., Mei-jer, Y., and Fraisse, R.: Nighttime ozone profiles in the stratosphere and mesosphere by the Global Ozone Monitoring by Occultation of Stars on Envisat, J. Geophys. Res., 111, D24306, doi: 10.1029/2006JD007193, 2006. Kyrölä, E., Tamminen, J., Sofieva, V., Bertaux, J. L., Hauchecorne, A., Dalaudier, F., Fussen, D., Vanhellemont, F., Fanton d'Andon, O., Barrot, G., Guirlet, M., Mangin, A., Blanot, L., Fehr, T., Saavedra de Miguel, L., and Fraisse, R.: Retrieval of atmospheric parameters from GOMOS data, Atmos. Chem. Phys., 10,1188111903, doi:10.5194/acp-10-11881-2010, 2010. Laeng, A., Hubert, D., Verhoelst, T., von Clarmann, T., Dinelli, B. M., Dudhia, A., Raspollini, P., Stiller, G., Grabowski, U., Kep-pens, A., Kiefer, M., Sofieva, V., Froideveaux, L., Walker, K. A., Lambert, J.-C., and Zehner, C.: The Ozone Climate Change Initiative: Comparison of four Level 2 Processors for the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS), Remote Sens. Environ., submitted, 2014.

Piccolo, C. and Dudhia, A.: Precision validation of MIPAS-Envisat products, Atmos. Chem. Phys., 7, 1915-1923, doi:10.5194/acp-7-1915-2007, 2007.

Sofieva, V. F., Kan, V., Dalaudier, F., Kyrölä, E., Tamminen, J., Bertaux, J.-L., Hauchecorne, A., Fussen, D., and Vanhellemont, F.: Influence of scintillation on quality of ozone monitoring by GOMOS, Atmos. Chem. Phys., 9, 9197-9207, doi:10.5194/acp-9-9197-2009, 2009.

Sofieva, V. F., Vira, J., Kyrölä, E., Tamminen, J., Kan, V., Dalaudier, F., Hauchecorne, A., Bertaux, J.-L., Fussen, D., Vanhelle-mont, F., Barrot, G., and Fanton d'Andon, O.: Retrievals from GOMOS stellar occultation measurements using characterization of modeling errors, Atmos. Meas. Tech., 3, 1019-1027, doi:10.5194/amt-3-1019-2010,2010.

Sofieva, V. F., Rahpoe, N., Tamminen, J., Kyrölä, E., Kalakoski, N., Weber, M., Rozanov, A., von Savigny, C., Laeng, A., von Clarmann, T., Stiller, G., Lossow, S., Degenstein, D., Bourassa,

A., Adams, C., Roth, C., Lloyd, N., Bernath, P., Hargreaves, R. J., Urban, J., Murtagh, D., Hauchecorne, A., Dalaudier, F., van Roozendael, M., Kalb, N., and Zehner, C.: Harmonized dataset of ozone profiles from satellite limb and occultation measurements, Earth Syst. Sci. Data, 5, 349-363, doi:10.5194/essd-5-349-2013, 2013.

Staten, P. W. and Reichler, T.: Apparent precision of GPS radio occultation temperatures, Geophys. Res. Lett., 36, L24806, doi:10.1029/2009GL041046,2009.

Tamminen, J., Kyrölä, E., Sofieva, V. F., Laine, M., Bertaux, J.-L., Hauchecorne, A., Dalaudier, F., Fussen, D., Vanhellemont, F., Fanton-d'Andon, O., Barrot, G., Mangin, A., Guirlet, M., Blanot, L., Fehr, T., Saavedra de Miguel, L., and Fraisse, R.: GOMOS data characterisation and error estimation, Atmos. Chem. Phys., 10, 9505-9519, doi:10.5194/acp-10-9505-2010, 2010.

Von Clarmann, T., Glatthor, N., Grabowski, U., Höpfner, M., Kellmann, S., Kiefer, M., Linden, A., Mengistu Tsidu, G., Milz, M., Steck, T., Stiller, G. P., Wang, D. Y., Fischer, H., Funke,

B., Gil-Löpez, S., and Löpez-Puertas, M.: Retrieval of temperature and tangent altitude pointing from limb emission spectra recorded from space by the Michelson Interferometer for Passive Atmospheric Sounding (MIPAS), J. Geophys. Res., 108, 4736, doi:10.1029/2003JD003602, 2003.

von Clarmann, T., Höpfner, M., Kellmann, S., Linden, A., Chauhan, S., Funke, B., Grabowski, U., Glatthor, N., Kiefer, M., Schieferdecker, T., Stiller, G. P., and Versick, S.: Retrieval of temperature, H2O, O3, HNO3, CH4, N2O, ClONO2 and ClO from MIPAS reduced resolution nominal mode limb emission measurements, Atmos. Meas. Tech., 2, 159-175, doi:10.5194/amt-2-159-2009, 2009.

Yaglom, A. M.: Correlation theory of stationary and related random functions, Springer Verlag., 1987.

Copyright of Atmospheric Measurement Techniques is the property of Copernicus Gesellschaft mbH and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.