`pyemu.la`

LinearAnalysis object, which is the base class for other pyemu analysis objects (Schur, ErrVar, MonteCarlo, and EnsembleSmoother). This class is usually not used directly.

Module Contents

Classes

LinearAnalysis

The base/parent class for linear analysis.

class pyemu.la.LinearAnalysis(jco=None, pst=None, parcov=None, obscov=None, predictions=None, ref_var=1.0, verbose=False, resfile=False, forecasts=None, sigma_range=4.0, scale_offset=True, **kwargs)

Bases: object

The base/parent class for linear analysis.

Parameters:

jco (varies, optional) – something that can be cast or loaded into a pyemu.Jco. Can be a str for a filename or pyemu.Matrix/pyemu.Jco object.
pst (varies, optional) – something that can be cast into a pyemu.Pst. Can be an str for a filename or an existing pyemu.Pst. If None, a pst filename is sought with the same base name as the jco argument (if passed)
parcov (varies, optional) – prior parameter covariance matrix. If str, a filename is assumed and the prior parameter covariance matrix is loaded from a file using the file extension (“.jcb”/”.jco” for binary, “.cov”/”.mat” for PEST-style ASCII matrix, or “.unc” for uncertainty files). If None, the prior parameter covariance matrix is constructed from the parameter bounds in LinearAnalysis.pst. Can also be a pyemu.Cov instance
obscov (varies, optional) – observation noise covariance matrix. If str, a filename is assumed and the noise covariance matrix is loaded from a file using the file extension (“.jcb”/”.jco” for binary, “.cov”/”.mat” for PEST-style ASCII matrix, or “.unc” for uncertainty files). If None, the noise covariance matrix is constructed from the obsevation weights in LinearAnalysis.pst. Can also be a pyemu.Cov instance
forecasts (varies, optional) – forecast sensitivity vectors. If str, first an observation name is assumed (a row in LinearAnalysis.jco). If that is not found, a filename is assumed and predictions are loaded from a file using the file extension. If [str], a list of observation names is assumed. Can also be a pyemu.Matrix instance, a numpy.ndarray or a collection of pyemu.Matrix or numpy.ndarray.
ref_var (float, optional) – reference variance. Default is 1.0
verbose (bool) – controls screen output. If str, a filename is assumed and and log file is written.
sigma_range (float, optional) – defines range of upper bound - lower bound in terms of standard deviation (sigma). For example, if sigma_range = 4, the bounds represent 4 * sigma. Default is 4.0, representing approximately 95% confidence of implied normal distribution. This arg is only used if constructing parcov from parameter bounds.
scale_offset (bool, optional) – flag to apply parameter scale and offset to parameter bounds when calculating prior parameter covariance matrix from bounds. This arg is onlyused if constructing parcov from parameter bounds.Default is True.

Note

Can be used directly, but for prior uncertainty analyses only.

The derived types (pyemu.Schur, pyemu.ErrVar) are for different forms of FOMS-based posterior uncertainty analyses.

This class tries hard to not load items until they are needed; all arguments are optional.

The class makes heavy use of property decorator to encapsulated private attributes

Example:

#assumes "my.pst" exists
la = pyemu.LinearAnalysis(jco="my.jco",forecasts=["fore1","fore2"])
print(la.prior_forecast)

property forecast_names

get the forecast (aka prediction) names

Returns:: list of forecast names
Return type:: ([str])

property parcov

get the prior parameter covariance matrix attribute

Returns:: a reference to the LinearAnalysis.parcov attribute
Return type:: pyemu.Cov

property obscov

get the observation noise covariance matrix attribute

Returns:: a reference to the LinearAnalysis.obscov attribute
Return type:: pyemu.Cov

property nnz_obs_names

non-zero-weighted observation names

Returns:: list of non-zero-weighted observation names
Return type:: [‘str`]

Note

if LinearAnalysis.pst is None, returns LinearAnalysis.jco.row_names

property adj_par_names

adjustable parameter names

Returns:: list of adjustable parameter names
Return type:: [‘str`]

Note

if LinearAnalysis.pst is None, returns LinearAnalysis.jco.col_names

property jco

the jacobian matrix attribute

Returns:: the jacobian matrix attribute
Return type:: pyemu.Jco

property predictions

the prediction (aka forecast) sentivity vectors attribute

Returns:: a matrix of prediction sensitivity vectors (column wise)
Return type:: pyemu.Matrix

property predictions_iter

prediction sensitivity vectors iterator

Returns:: iterator on prediction sensitivity vectors (matrix)
Return type:: iterator

Note

this is used for processing huge numbers of predictions

property forecasts_iter

forecast (e.g. prediction) sensitivity vectors iterator

Returns:: iterator on forecasts (e.g. predictions) sensitivity vectors (matrix)
Return type:: iterator

Note

This is used for processing huge numbers of predictions

This is a synonym for LinearAnalysis.predictions_iter()

property forecasts

the forecast sentivity vectors attribute

Returns:: a matrix of forecast (prediction) sensitivity vectors (column wise)
Return type:: pyemu.Matrix

property pst

the pst attribute

Returns:: the pst attribute
Return type:: pyemu.Pst

property fehalf

Karhunen-Loeve scaling matrix attribute.

Returns:: the Karhunen-Loeve scaling matrix based on the prior parameter covariance matrix
Return type:: pyemu.Matrix

property qhalf

square root of the cofactor matrix attribute. Create the attribute if it has not yet been created

Returns:: square root of the cofactor matrix
Return type:: pyemu.Matrix

property qhalfx

half normal matrix attribute.

Returns:: half normal matrix attribute
Return type:: pyemu.Matrix

property xtqx

normal matrix attribute.

Returns:: normal matrix attribute
Return type:: pyemu.Matrix

property mle_covariance

maximum likelihood parameter covariance matrix.

Returns:: maximum likelihood parameter covariance matrix
Return type:: pyemu.Matrix

property prior_parameter

prior parameter covariance matrix

Returns:: prior parameter covariance matrix
Return type:: pyemu.Cov

property prior_forecast

prior forecast (e.g. prediction) variances

Returns:: a dictionary of forecast name, prior variance pairs
Return type:: dict

property mle_parameter_estimate

maximum likelihood parameter estimate.

Returns:: the maximum likelihood parameter estimates
Return type:: pandas.Series

property prior_prediction

prior prediction (e.g. forecast) variances

Returns:: a dictionary of prediction name, prior variance pairs
Return type:: dict

__fromfile(filename, astype=None): a private method to deduce and load a filename into a matrix object. Uses extension: ‘jco’ or ‘jcb’: binary, ‘mat’,’vec’ or ‘cov’: ASCII, ‘unc’: pest uncertainty file.

__load_pst(): private method set the pst attribute

__load_jco(): private method to set the jco attribute from a file or a matrix object

__load_parcov(): private method to set the parcov attribute from: a pest control file (parameter bounds) a pst object a matrix object an uncert file an ascii matrix file

__load_obscov(): private method to set the obscov attribute from: a pest control file (observation weights) a pst object a matrix object an uncert file an ascii matrix file

__load_predictions()

private method set the predictions attribute from:: mixed list of row names, matrix files and ndarrays a single row name an ascii file

can be none if only interested in parameters.

apply_karhunen_loeve_scaling(): apply karhuene-loeve scaling to the jacobian matrix.

Note

This scaling is not necessary for analyses using Schur’s complement, but can be very important for error variance analyses. This operation effectively transfers prior knowledge specified in the parcov to the jacobian and reset parcov to the identity matrix.

clean(): drop regularization and prior information observation from the jco

reset_pst(arg)

reset the LinearAnalysis.pst attribute

Parameters:: arg (str or pyemu.Pst) – the value to assign to the pst attribute

reset_parcov(arg=None)

reset the parcov attribute to None

Parameters:: arg (str or pyemu.Matrix) – the value to assign to the parcov attribute. If None, the private __parcov attribute is cleared but not reset

reset_obscov(arg=None)

reset the obscov attribute to None

Parameters:: arg (str or pyemu.Matrix) – the value to assign to the obscov attribute. If None, the private __obscov attribute is cleared but not reset

drop_prior_information(): drop the prior information from the jco and pst attributes

get(par_names=None, obs_names=None, astype=None)

method to get a new LinearAnalysis class using a subset of parameters and/or observations

Parameters:

par_names ([‘str]) – par names for new object
obs_names ([‘str]) – obs names for new object
astype (pyemu.Schur or pyemu.ErrVar) – type to cast the new object. If None, return type is same as self

Returns:

new instance

Return type:

LinearAnalysis

adjust_obscov_resfile(resfile=None)

reset the elements of obscov by scaling the implied weights based on the phi components in res_file so that the total phi is equal to the number of non-zero weights.

Parameters:: resfile (str) – residual file to use. If None, residual file with case name is sought. default is None

Note

calls pyemu.Pst.adjust_weights_resfile()

get_par_css_dataframe()

get a dataframe of composite scaled sensitivities. Includes both PEST-style and Hill-style.

Returns:: a dataframe of parameter names, PEST-style and Hill-style composite scaled sensitivity
Return type:: pandas.DataFrame

get_cso_dataframe()

get a dataframe of composite observation sensitivity, as returned by PEST in the seo file.

Returns:: dataframe of observation names and composite observation sensitivity
Return type:: pandas.DataFrame

Note

That this formulation deviates slightly from the PEST documentation in that the values are divided by (npar-1) rather than by (npar).

The equation is cso_j = ((Q^1/2*J*J^T*Q^1/2)^1/2)_jj/(NPAR-1)

get_obs_competition_dataframe()

get the observation competition stat a la PEST utility

Returns:: a dataframe of observation names by observation names with values equal to the PEST competition statistic
Return type:: pandas.DataFrame

pyemu.la

Module Contents

Classes

`pyemu.la`