european storm forecast experiment verification of dichotomous lightning forecasts at the european...

European Storm Forecast Experiment

Verification of Dichotomous Lightning Forecasts at the

European Storm Forecast Experiment (ESTOFEX)

Pieter GroenemeijerOscar van der Velde

Helge TuschyChristoph Gatzen

Johannes DahlNicholas Verge

Institute for Meteorology and Climate Research, Research Centre / University KarlsruheEuropean Severe Storms Laboratory

Laboratoire d'Aérologie, Université Paul Sabatier, ToulouseInstitute for Atmospheric Physics, German Aerospace Center (DLR)


Overview

• About ESTOFEX

• ESTOFEX forecasts

• Verification of lightning forecasts

• Verification of threat level forecasts


about ESTOFEX

• the name of a web-based platform for the exchange of knowledge about convective storm forecasting

• the name of a the forecast experiment that is carried out by some of its active members

• goal of the experiment is... “to find out how convective storms can best be

forecast in Europe”


about ESTOFEXESTOFEX issues “convective

forecasts” for the next day

These forecasts consist of two parts:

• forecast discussion

• forecast map

Storm ForecastValid: Sat 02 Jun 2007 06:00 to Sun 03 Jun 2007 06:00 UTCIssued: Sat 02 Jun 2007 09:22Forecaster: VAN DER VELDESYNOPSIS

An mid/upper cut-off low dwells slowly southward with its center just west of Italy. The associated surface pressure field weakens during the period. High pressure...


ESTOFEX is not...

• a new weather service in Europe(no such centre without an initiative from NMS’s)

• funded

• the start of a commercial company


thunder area

level 1

level 2

level 3

ESTOFEX forecasts

The forecast maps show lines that represent

level 1, level 2, level 3 are threat levels for convective wind gusts, tornadoes and large hail


ESTOFEX forecastsAfter the forecast hasbeen issued, verificationdata appears on themap.

Hail reportsTornado reportsWind gust reportsHeavy rain reports(source: ESSL)

Lightning strikessource: UK. Met Officearrival-time-difference system


Verification of lightning forecasts

• Predictand

the occurrence of lightning* within 40 km of a point

• Dichotomous forecast event occurred?

yes no

event forecast?

yes hit false alarm

no misscorrect

negative

* = one or more strike detected


Forecast lightning (1 Sep. 2006 – 31 Aug. 2007)

• number of forecast periods

(of 346 forecasts) in which

lightning was forecast to occur

within 40 km of each point


Detected lightning (1 Sep. 2006 – 31 Aug. 2007)

• number of forecast periods

(of 346 forecasts) in which

lightning occurred within

40 km of a point

More lightning is forecast than

occurs in reality.

The forecasts have a positive bias.


Forecast skill indicators

• False Alarm Rate =

false alarms / (hits + false alarms)

• Probability Of Detection =

hits / (hits + misses)

• Heidke Skill Score


Probability Of Detection(1 Sep. 2006 – 31 Aug. 2007)

High POD over...

• the western and

central European continent• the central and eastern

Mediterranean Sea

Low POD over...

• the Atlantic Ocean• Scandinavia• Eastern Europe• Northern Africa

POD =

hits / (hits + misses)


False Alarm Rate(1 Sep. 2006 – 31 Aug. 2007)

Low FAR over...

• central and eastern parts of

the European continent

High FAR over...

• western and northern Europe• parts of the Mediterranean• the Black Sea

FAR =

false / (hits + false)


Heidke Skill Score(1 Sep. 2006 – 31 Aug. 2007)

HSS = 0 means that the forecast

is no better than chance

HSS = 1 means that the forecast

is perfect

HSS assumes that a miss is just as bad

as a false alarm.

HSS indicates the forecasts are best

over the central and eastern

European continent

and the southern Mediterranean

Forecasts are bad over western and

northern Europe


Temporal evolution of forecast skill


Temporal changes in performance


Individual forecaster performance2006 and 2007 summer periods compared

Probability of Detection False Alarm Ratio


Future plans

event occurred ?

Yes No

event forecast?

> 60 %

30-60%

15-30%

5 -15%

< 5%

• Polychotomous lightning forecasts


Future plans

• switch to probabilistic forecasting of lightning

• start rigorous verification of severe weather(hail, tornadoes, severe winds)

• switch to probabilistic forecasting of severe weather

• test the skill of model-derived parameters for both lightning and hail, tornadoes winds – and compare with human forecasters (interesting!)


Final remarks

• a discussion of individual storm events is presented on the poster lead-authored by Helge Tuschy and in Christoph Gatzen’s talk

• all verification data is available through the forecast archive at www.estofex.org

• I am happy to answer your questions on ESTOFEX and the verification of its forecasts

european storm forecast experiment verification of dichotomous lightning forecasts at the european...

Documents