The reliability table is a graphical means of assessing the three attributes reliability, resolution and sharpness of a set of probability forecasts. The diagram is created by first binning all the forecast-observation pairs, according to the forecast probability. For example, all the forecasts between 0 and 10% would be put in one group, all forecasts between 11 and 20% inclusive in another group and so on. The width of the bins is often 10%, but this is determined according to the nature of the verification data and the sample size. The width of the bins need not be the same over the 0 to 100% range. For example, if high probabilities are not often forecast, then the higher range bins could be wider in order that there be a large enough number of cases in each bin. Once the bins are decided and the data partitioned, then for each bin the observed frequency of occurrence of the event is obtained by dividing the number of occurrences by the total number of cases which are in the bin. Then, the observed frequency of occurrence is plotted against the forecast probability represented by each bin.

In addition to ensuring that there are enough cases in each bin to obtain a stable estimate of the observed frequency (typically 10 or more), one should pay attention to the "permitted" probability forecasts when defining the bins. Often, forecasters limit themselves to deciles of probability, 0, 0.1, 0.2 etc, so that only 11 probability values are actually forecast. In that case, it would be preferable to center the bins on the probability forecasts, in which case the extreme bins would be half-width, 0 to 5% and 95 to 100%.

You have available one year of 48 h probability of precipitation forecasts for Tampere, Finland, along with the corresponding observations. The forecasts and corresponding observations have already been binned. This was easy in this case since only the 11 values 0, 0.1, 0.2…..0.9 and 1.0 were allowed to be forecast. The second column of the table below lists the number of times precipitation occurred when each probability value was predicted. Estimate the observed frequency of occurrence for each bin and drag the appropriate value into the corresponding cell in the third column of the table. Finally, drag and drop the sample climatological frequency (base rate) into the correct box in the lower right corner of the table. When you have finished, click the discussion button for an instant plot of the reliability table using the data from the table.

Discussion

correct

incorrect - try again

Please put the labels on one of the existing boxes.

You have completed the exercise