NWP winds impact study

by Mary Forsythe — last modified Nov 28, 2012 04:16 AM

Aim to learn more about the impact of satellite-derived wind data through a collaborative winds impact study.

CGMS-A39.30. The co-chairs of IWWG and the rapporteur are requested to discuss the results from NWP impact studies at IWW11, and to synthesize general observations on performance.  Due date: IWW11 in Feb 2012 and report to CGMS-40.

IWW10.1.    NWP centres to coordinate a joint AMV and scatterometer denial study (possibly two 6 week periods), also looking at adjoint sensitivity statistics where available.  Aim to summarise in a report to the WMO GOS impact workshop and IWW11, both due to be held in the first half of 2012.  

Please contact James Cotton (Met Office) or Christophe Payan (Meteo France) for further information. 

A preliminary study involving the Met Office, ECMWF, Meteo-France, DWD and NCEP was carried out in 2008-09.  The plan, agreed at IWW10, is to expand on this work, by selecting 2 longer seasons and agreeing more coordinated verification.  Where possible, results of sensitivity studies will also be included.  A final report has been released.

Further details will be provided below as the next phase of the project progresses.

Test period:

Two 6-week periods during the 2010 Atlantic hurricane season and the 2010-11 NH winter season. Proposed dates are

• (Period 1)  12z 15 August 2010 - 12z 30 September 2010 (captures all major Atlantic hurricanes). For the denial experiment it is recommended that the first assimilation cycle is 18z 14 August to reduce the possible effects of spin-up, with the first forecast run then on 12z 15 August .
• (Period 2)  12z 01 December 2010 - 12z 15 January 2011. For the denial experiment it is recommended that the first assimilation cycle is 18z 30 November to reduce the possible effects of spin-up, with the first forecast run then on 12z 01 December .
  

Centres involved:

Still to be confirmed, but probably: the Met Office, ECMWF, Meteo-France, NRL, JMA, KMA, CMA, BoM, DWD and NCEP

Trial configurations: 

The idea is to run AMV denial experiments during both of the 6 week periods whilst the scatterometer experiment can be restricted to just the Atlantic hurricane period. It is hoped that as many NWP centres as possible will be able to contribute to both seasons but this will be dependent on the resources available. If centres are only able to contribute to one period then this should be the Atlantic hurricane season (period 1).

Preferred option: a straightforward denial experiment. This implies looking for a smaller signal but reflects the actual impact the winds are giving in each operational system.  The control experiment should be as per operations or as close as possible, e.g. it may be necessary to run at a slightly lower resolution to save on computing resource. As per the previous study, experiments should be based on forecasts starting from 12z.
 1) Atlantic hurricane season 2010
   - Operational Control
   - Control + No AMV
   - Control + No Scatterometer (optional)

2)  NH Winter season 2010/11   
   - Operational Control
   - Control + No AMV
   - Control + No Polar AMV (optional)

Verification recommendations:

It is recommended that participating NWP centres store model results from the above experiments in case we want to re-calculate some statistics that we haven't thought of in advance.

For verification of results it would be useful to (where possible) agree on a standard set of plots.  Those used in the previous study are a good place to start and include:

1. Impact (vector difference map) at Z200 or Z250 on mean wind analysis (denial-control). 
2. Profiles of  U and V wind component first guess (o-b) and analysis (o-a) departures (mean and standard deviation) verified against Sondes for the control and trial. Statistics separated by NH, TR and SH.
3. Map of difference (denial-control) in mean/rms T+48 FC error for 500hPa Geopotential Height, verification against its own analysis.
4. Time series plots of mean speed error and rms vector error for the 850 hPa and 200/250 hPa T+24 hr wind forecasts verified against sondes for the control and trial. Statistics separated by NH, TR and SH.

Notes

• NH denotes Northern Hemisphere (area north of 20N), TR denotes the Tropics (between 20S and 20N) and SH refers to the Southern Hemisphere (south of 20S).
  

• Where appropriate the differences should be calculated as denial-control, so that [denial - control] > 0 indicates positive impact from the observations.
• Other individual verification metrics such as the Met Office NWP Index can also be used.

Comparisons can be improved from the previous study by, for example, using a uniform colour scale for map plots.  

Adjoint sensitivity diagnostics:

Adjoint-based forecast-sensitivity to observations tools should also be used (where available). The default task will be to run these for the summer period (period 1) using the full observing system experiment so as to get diagnostics for the AMV and scatterometer data.

To start with the following diagnostics are suggested

1. Bar graphs showing total and mean impact/sensitivity for each satellite and channel combination.
   

1. Global maps of mean impact/sensitivity separated by channels/layers to identify if there are common areas with good or bad impact.

    Preliminary results and conclusions:

Ideally, preliminary results should be made ready by participating NWP centres by 1 October 2011. This will leave enough time before the Winds Workshop in February 2012 for writting a synthesis and calculating other statistics if necessary. 

Final report:

Released November 2012 - available at: http://cimss.ssec.wisc.edu/iwwg/Docs/windsdenial-synthesisV1-1.pdf