src.verify#

Module Contents#

Functions#

rXY(hcst, obsv[, correlation])

Return the anomaly cross correlation between two timeseries

ri(hcst, obsv, hist)

Return the initialised component of Pearson anomaly cross correlation

msss(hcst, obsv, ref[, ensemble_mean])

Return the mean squared skill score between a forecast and observations

crpss(hcst, obsv, ref)

Return the Continuous rank probability skill score between a forecast and

Fisher_z(ds)

Return the Fisher-z transformation of ds

_calculate_metric_from_timeseries(*timeseries, metric, ...)

Calculate a skill metric from the provided timeseries

calculate_metric(hindcast, observation, metric[, ...])

Calculate a skill metric for a set of hindcasts. This function will attempt

verify(config, save_dir[, save])

Prepare a skill metric according to a provided config file

main(config, config_dir, save_dir)

Prepare skill metrics according to a provided config file

Attributes#

N_BOOTSTRAP_ITERATIONS

PROJECT_DIR

DATA_DIR

log_fmt

src.verify.N_BOOTSTRAP_ITERATIONS = 1000#
src.verify.PROJECT_DIR#
src.verify.DATA_DIR#
src.verify.rXY(hcst, obsv, correlation='pearson_r')#

Return the anomaly cross correlation between two timeseries

Parameters
hcstxarray Dataset

The forecast timeseries

obsvxarray Dataset

The observed timeseries

src.verify.ri(hcst, obsv, hist)#

Return the initialised component of Pearson anomaly cross correlation between a forecast and observations

Parameters
hcstxarray Dataset

The forecast timeseries

obsvxarray Dataset

The observed timeseries

histxarray Dataset

The historical simulation timeseries

src.verify.msss(hcst, obsv, ref, ensemble_mean=True)#

Return the mean squared skill score between a forecast and observations relative to a reference dataset

Parameters
hcstxarray Dataset

The forecast timeseries

obsvxarray Dataset

The observed timeseries

refxarray Dataset

The reference timeseries

ensemble_mean: boolean, optional

If true, take the mean across the “member” dimension of hcst and ref (if it exists) prior to calculating the MSSS

src.verify.crpss(hcst, obsv, ref)#

Return the Continuous rank probability skill score between a forecast and observations relative to a reference dataset

Parameters
hcstxarray Dataset

The forecast timeseries

obsvxarray Dataset

The observed timeseries

refxarray Dataset

The reference timeseries

src.verify.Fisher_z(ds)#

Return the Fisher-z transformation of ds

Parameters
dsxarray Dataset

The data to apply the Fisher-z transformation to

src.verify._calculate_metric_from_timeseries(*timeseries, metric, metric_kwargs, significance=True, bootstrap_kwargs={}, transform=None, alpha=0.1)#

Calculate a skill metric from the provided timeseries

Statistical significance at 1-alpha is identified at all points where the sample skill metric is positive (negative) and the fraction of transformed values in the bootstrapped distribution below (above) no_skill_value–defining the p-values–is less than or equal to alpha.)

src.verify.calculate_metric(hindcast, observation, metric, reference=None, metric_kwargs={}, significance=False, transform=None, alpha=0.05)#

Calculate a skill metric for a set of hindcasts. This function will attempt to validate over a common set of verification times at all leads and will return a warning if this is not possible. If a common set of verification times cannot be found, this function will verify over all available times at each lead

Parameters
hindcastxarray Dataset

The hindcast data to verify. Must have “init” and “lead” dimensions

observationxarray Dataset

The data to verify against. Must have a “time” dimension

metricstr

The name of the metric to apply to apply to the timeseries. Will look for function in src.verify.

referencestr or xarray Dataset, optional

The skill baseline. Can be an xarray Dataset with a “time” dimension (e.g. an historical simulation) that will be passed on to the metric function. Alternatively, users can specify one of the following methods to automatically create the appropriate baseline timeseries to be passed on to the metric function: - “persistence”: hindcasts skill is baselined against the most recent

observation at the time of forecast initialisation

  • “climatology”: hindcasts skill is baselined against the monthly climatological mean of the observations over the verification period

metric_kwargsdict

kwargs to pass to the function metric

significanceboolean, optional

If True, also return a mask indicating points where skill estimates are significant using the non-parametric bootstrapping approach of Goddard et al. (2013).

transformfunction, optional

Transform to apply prior to estimating significant points

alphafloat, optional

The level [0,1] to apply sigificance at. Statistical significance at 1-alpha is identified at all points where the sample skill metric is positive (negative) and the fraction of transformed values in the bootstrapped distribution below (above) zero–defining the p-values –is less than or equal to alpha.)

src.verify.verify(config, save_dir, save=True)#

Prepare a skill metric according to a provided config file

Parameters
configstr

The name of the config file

save_dirstr

The directory to save to

saveboolean, optional

If True (default), save the prepared dataset(s) in zarr format to save_dir. If False, return an xarray Dataset containing the prepared data. The latter is useful for debugging

src.verify.main(config, config_dir, save_dir)#

Prepare skill metrics according to a provided config file

Parameters
configstr

The name of the config file

config_dirstr

The directory containing the config file

save_dirstr

The directory to save to

src.verify.log_fmt = %(asctime)s - %(name)s - %(levelname)s - %(message)s#