pyemu.ev

Module Contents

Classes

ErrVar

FOSM-based error variance analysis

class pyemu.ev.ErrVar(jco, **kwargs)

Bases: pyemu.la.LinearAnalysis

FOSM-based error variance analysis

Parameters:

• jco (varies, optional) – something that can be cast or loaded into a pyemu.Jco. Can be a str for a filename or pyemu.Matrix/pyemu.Jco object.

• pst (varies, optional) – something that can be cast into a pyemu.Pst. Can be an str for a filename or an existing pyemu.Pst. If None, a pst filename is sought with the same base name as the jco argument (if passed)

• parcov (varies, optional) – prior parameter covariance matrix. If str, a filename is assumed and the prior parameter covariance matrix is loaded from a file using the file extension (“.jcb”/”.jco” for binary, “.cov”/”.mat” for PEST-style ASCII matrix, or “.unc” for uncertainty files). If None, the prior parameter covariance matrix is constructed from the parameter bounds in LinearAnalysis.pst. Can also be a pyemu.Cov instance

• obscov (varies, optional) – observation noise covariance matrix. If str, a filename is assumed and the noise covariance matrix is loaded from a file using the file extension (“.jcb”/”.jco” for binary, “.cov”/”.mat” for PEST-style ASCII matrix, or “.unc” for uncertainty files). If None, the noise covariance matrix is constructed from the obsevation weights in LinearAnalysis.pst. Can also be a pyemu.Cov instance

• forecasts (varies, optional) – forecast sensitivity vectors. If str, first an observation name is assumed (a row in LinearAnalysis.jco). If that is not found, a filename is assumed and predictions are loaded from a file using the file extension. If [str], a list of observation names is assumed. Can also be a pyemu.Matrix instance, a numpy.ndarray or a collection. Note if the PEST++ option “++forecasts()” is set in the pest control file (under the pyemu.Pst.pestpp_options dictionary), then there is no need to pass this argument (unless you want to analyze different forecasts) of pyemu.Matrix or numpy.ndarray.

• ref_var (float, optional) – reference variance. Default is 1.0

• verbose (bool) – controls screen output. If str, a filename is assumed and and log file is written.

• sigma_range (float, optional) – defines range of upper bound - lower bound in terms of standard deviation (sigma). For example, if sigma_range = 4, the bounds represent 4 * sigma. Default is 4.0, representing approximately 95% confidence of implied normal distribution. This arg is only used if constructing parcov from parameter bounds.

• scale_offset (bool, optional) – flag to apply parameter scale and offset to parameter bounds when calculating prior parameter covariance matrix from bounds. This arg is onlyused if constructing parcov from parameter bounds.Default is True.

• omitted_parameters ([str]) – list of parameters to treat as “omitted”. Passing this argument activates 3-term error variance analysis.

• omitted_parcov (varies) – an argument that can be cast to a parcov for the omitted parameters. If None, omitted_parcov will be formed by extracting a sub-matrix from the LinearAnalsis.parcov attribute.

• omitted_predictions (varies) – an argument that can be cast to a “predictions” (e.g. “forecasts”) attribute to form prediction sensitivity vectors with respec to the omitted parameters. If None, these vectors will be extracted from the pyemu.LinearAnalysis.predictions attribute

• kl (bool, optional) – flag to perform Karhunen-Loeve scaling on the jacobian before error variance calculations. If True, the pyemu.ErrVar.jco and pyemu.ErrVar.parcov are altered in place. Default is False.

Example:

ev = pyemu.ErrVar(jco="my.jco",omitted_parameters=["wel1","wel2"])
df = ev.get_errvar_dataframe()

property omitted_predictions

omitted prediction sensitivity vectors

Returns:

a matrix of prediction sensitivity vectors (column wise) to omitted parameters

Return type:

pyemu.Matrix

property omitted_jco

the omitted-parameters jacobian matrix

Returns:

the jacobian matrix instance of non-zero-weighted observations and omitted parameters

Return type:

pyemu.Jco

property omitted_parcov

the prior omitted-parameter covariance matrix

Returns:

the prior parameter covariance matrix of the omitted parameters

Return type:

pyemu.Cov

__load_omitted_predictions()

private: set the omitted_predictions attribute

__load_omitted_parcov()

private: set the omitted_parcov attribute

__load_omitted_jco()

private: set the omitted jco attribute

get_errvar_dataframe(singular_values=None)

primary entry point for error variance analysis.

Parameters:

singular_values ([int], optional) – a list singular values to test. If None, defaults to range(0,min(nnz_obs,nadj_par) + 1).

Returns:

a multi-indexed pandas dataframe summarizing each of the error variance terms for each nominated forecast. Rows are the singluar values tested, columns are a multi-index of forecast name and error variance term number (e.g. 1,2 or (optionally) 3).

Return type:

pandas.DataFrame

Example:

ev = pyemu.ErrVar(jco="my.jco",omitted_parameters=["wel1","wel2"])
df = ev.get_errvar_dataframe()

get_identifiability_dataframe(singular_value=None, precondition=False)

primary entry point for identifiability analysis

Parameters:

• singular_value (int) – the singular spectrum truncation point. Defaults to minimum of non-zero-weighted observations and adjustable parameters

• precondition (bool) – flag to use the preconditioned hessian with the prior parameter covariance matrix (xtqt + sigma_theta^-1). This should be used KL scaling. Default is False.

Returns:

A pandas dataframe of the right solution-space singular vectors and identifiability (identifiabiity is in the column labeled “ident”)

Return type:

pandas.DataFrame

Examples:

ev = pyemu.ErrVar(jco="my.jco")
df = ev.get_identifiability_dataframe(singular_value=20)
df.ident.plot(kind="bar")

variance_at(singular_value)

get the error variance of all three error variance terms at a

given singluar value

Parameters:

singular_value (int) – singular value to test

Returns:

dictionary of (err var term,prediction_name), variance pairs

Return type:

dict

R(singular_value)

get resolution Matrix (V_1 * V_1^T) at a given singular value

Args: singular_value (int): singular value to calculate R at

Returns:

resolution matrix at singular_value

Return type:

pyemu.Matrix

I_minus_R(singular_value)

get I - R at a given singular value

Parameters:

singular_value (int) – singular value to calculate I - R at

Returns:

identity matrix minus resolution matrix at singular_value

Return type:

pyemu.Matrix

G(singular_value)

get the parameter solution Matrix at a given singular value

Parameters:

singular_value (int) – singular value to calc G at

Returns:

parameter solution matrix (V_1 * S_1^(_1) * U_1^T) at singular_value

Return type:

pyemu.Matrix

first_forecast(singular_value)

get the null space term (first term) contribution to forecast (e.g. prediction)

error variance at a given singular value.

Parameters:

singular_value (int) – singular value to calc first term at

Note

This method is used to construct the error variance dataframe

Just a wrapper around ErrVar.first_forecast

Returns:

dictionary of (“first”,prediction_names),error variance pairs at singular_value

Return type:

dict

first_prediction(singular_value)

get the null space term (first term) contribution to prediction error variance

at a given singular value.

Parameters:

singular_value (int) – singular value to calc first term at

Note

This method is used to construct the error variance dataframe

Returns:

dictionary of (“first”,prediction_names),error variance pairs at singular_value

Return type:

dict

first_parameter(singular_value)

get the null space term (first term) contribution to parameter error variance

at a given singular value

Parameters:

singular_value (int) – singular value to calc first term at

Returns:

first term contribution to parameter error variance

Return type:

pyemu.Cov

second_forecast(singular_value)

get the solution space contribution to forecast (e.g. “prediction”) error variance at a given singular value

Parameters:

singular_value (int) – singular value to calc second term at

Note

This method is used to construct error variance dataframe

Just a thin wrapper around ErrVar.second_prediction

Returns:

dictionary of (“second”,prediction_names), error variance arising from the solution space contribution (y^t * G * obscov * G^T * y)

Return type:

dict

second_prediction(singular_value)

get the solution space contribution to predictive error variance at a given singular value

Parameters:

singular_value (int) – singular value to calc second term at

Note

This method is used to construct error variance dataframe

Returns: dict: dictionary of (“second”,prediction_names), error variance

arising from the solution space contribution (y^t * G * obscov * G^T * y)

second_parameter(singular_value)

get the solution space contribution to parameter error variance

at a given singular value (G * obscov * G^T)

Parameters:

singular_value (int) – singular value to calc second term at

Returns:

the second term contribution to parameter error variance (G * obscov * G^T)

Return type:

pyemu.Cov

third_forecast(singular_value)

get the omitted parameter contribution to forecast (prediction) error variance

at a given singular value.

Parameters:

singular_value (int) – singular value to calc third term at

Note

used to construct error variance dataframe just a thin wrapper around ErrVar.third_prediction()

Returns:

a dictionary of (“third”,prediction_names),error variance

Return type:

dict

third_prediction(singular_value)

get the omitted parameter contribution to prediction error variance

at a given singular value.

Parameters:

singular_value (int) – singular value to calc third term at

Note

used to construct error variance dataframe

Returns:

a dictionary of (“third”,prediction_names),error variance

Return type:

dict

third_parameter(singular_value)

get the omitted parameter contribution to parameter error variance

at a given singular value

Parameters:

singular_value (int) – singular value to calc third term at

Returns:

the third term contribution to parameter error variance calculated at singular_value (G * omitted_jco * Sigma_(omitted_pars) * omitted_jco^T * G^T). Returns 0.0 if third term calculations are not being used.

Return type:

pyemu.Cov

get_null_proj(maxsing=None, eigthresh=1e-06)

get a null-space projection matrix of XTQX

Parameters:

• maxsing (int, optional) – number of singular components to use (the truncation point). If None, pyemu.Matrx.get_maxsing() is used to determine the truncation point with `eigthresh. Default is None

• eigthresh (float, optional) – the ratio of smallest to largest singular value to keep in the range (solution) space of XtQX. Not used if maxsing is not None. Default is 1.0e-6

Note

used for null-space monte carlo operations.

Returns:

pyemu.Matrix the null-space projection matrix (V2V2^T)