mc

pyEMU Monte Carlo module. Supports easy Monte Carlo and GLUE analyses. The MonteCarlo class inherits from pyemu.LinearAnalysis

`MonteCarlo`

Bases: LinearAnalysis

LinearAnalysis derived type for monte carlo analysis

Parameters

**kwargs : dict dictionary of keyword arguments. See pyemu.LinearAnalysis for complete definitions

Attributes

parensemble : pyemu.ParameterEnsemble pyemu object derived from a pandas dataframe, the ensemble of parameters from the PEST control file with associated starting value and bounds. Object also exposes methods relevant to the dataframe and parameters-- see documentation. obsensemble : pyemu.ObservationEnsemble pyemu object derived from a pandas dataframe, the ensemble of observations from the PEST control file with associated starting weights. Object also exposes methods relevant to the dataframe and observations-- see documentation.

Returns

MonteCarlo pyEMU MonteCarlo object

Example

>>>import pyemu

>>>mc = pyemu.MonteCarlo(pst="pest.pst")

`adj_par_names` `property`

adjustable parameter names

Returns:

Type	Description
	['str`]: list of adjustable parameter names

Note

if LinearAnalysis.pst is None, returns LinearAnalysis.jco.col_names

`fehalf` `property`

Karhunen-Loeve scaling matrix attribute.

Returns:

Type	Description
	`pyemu.Matrix`: the Karhunen-Loeve scaling matrix based on the prior
	parameter covariance matrix

`forecast_names` `property`

get the forecast (aka prediction) names

Returns:

Type	Description
[`str`]	list of forecast names

`forecasts` `property`

the forecast sentivity vectors attribute

Returns:

Type	Description
	`pyemu.Matrix`: a matrix of forecast (prediction) sensitivity vectors (column wise)

`forecasts_iter` `property`

forecast (e.g. prediction) sensitivity vectors iterator

Returns:

Type	Description
	`iterator`: iterator on forecasts (e.g. predictions) sensitivity vectors (matrix)

Note

This is used for processing huge numbers of predictions

This is a synonym for LinearAnalysis.predictions_iter()

`jco` `property`

the jacobian matrix attribute

Returns:

Type	Description
	`pyemu.Jco`: the jacobian matrix attribute

`mle_covariance` `property`

maximum likelihood parameter covariance matrix.

Returns:

Type	Description
	`pyemu.Matrix`: maximum likelihood parameter covariance matrix

`mle_parameter_estimate` `property`

maximum likelihood parameter estimate.

Returns:

Type	Description
	`pandas.Series`: the maximum likelihood parameter estimates

`nnz_obs_names` `property`

non-zero-weighted observation names

Returns:

Type	Description
	['str`]: list of non-zero-weighted observation names

Note

if LinearAnalysis.pst is None, returns LinearAnalysis.jco.row_names

`num_reals` `property`

get the number of realizations in the parameter ensemble

Returns

num_real : int

`obscov` `property`

get the observation noise covariance matrix attribute

Returns:

Type	Description
	`pyemu.Cov`: a reference to the `LinearAnalysis.obscov` attribute

`parcov` `property`

get the prior parameter covariance matrix attribute

Returns:

Type	Description
	`pyemu.Cov`: a reference to the `LinearAnalysis.parcov` attribute

`predictions` `property`

the prediction (aka forecast) sentivity vectors attribute

Returns:

Type	Description
	`pyemu.Matrix`: a matrix of prediction sensitivity vectors (column wise)

`predictions_iter` `property`

prediction sensitivity vectors iterator

Returns:

Type	Description
	`iterator`: iterator on prediction sensitivity vectors (matrix)

Note

this is used for processing huge numbers of predictions

`prior_forecast` `property`

prior forecast (e.g. prediction) variances

Returns:

Type	Description
	`dict`: a dictionary of forecast name, prior variance pairs

`prior_parameter` `property`

prior parameter covariance matrix

Returns:

Type	Description
	`pyemu.Cov`: prior parameter covariance matrix

`prior_prediction` `property`

prior prediction (e.g. forecast) variances

Returns:

Type	Description
	`dict`: a dictionary of prediction name, prior variance pairs

`pst` `property`

the pst attribute

Returns:

Type	Description
	`pyemu.Pst`: the pst attribute

`qhalf` `property`

square root of the cofactor matrix attribute. Create the attribute if it has not yet been created

Returns:

Type	Description
	`pyemu.Matrix`: square root of the cofactor matrix

`qhalfx` `property`

half normal matrix attribute.

Returns:

Type	Description
	`pyemu.Matrix`: half normal matrix attribute

`xtqx` `property`

normal matrix attribute.

Returns:

Type	Description
	`pyemu.Matrix`: normal matrix attribute

`__fromfile(filename, astype=None)`

a private method to deduce and load a filename into a matrix object. Uses extension: 'jco' or 'jcb': binary, 'mat','vec' or 'cov': ASCII, 'unc': pest uncertainty file.

`__load_jco()`

private method to set the jco attribute from a file or a matrix object

`__load_obscov()`

private method to set the obscov attribute from: a pest control file (observation weights) a pst object a matrix object an uncert file an ascii matrix file

`__load_parcov()`

private method to set the parcov attribute from: a pest control file (parameter bounds) a pst object a matrix object an uncert file an ascii matrix file

`__load_predictions()`

private method set the predictions attribute from

mixed list of row names, matrix files and ndarrays a single row name an ascii file

can be none if only interested in parameters.

`__load_pst()`

private method set the pst attribute

`adjust_obscov_resfile(resfile=None)`

reset the elements of obscov by scaling the implied weights based on the phi components in res_file so that the total phi is equal to the number of non-zero weights.

Parameters:

Name	Type	Description	Default
`resfile`	`str`	residual file to use. If None, residual file with case name is sought. default is None	`None`

Note

calls pyemu.Pst.adjust_weights_resfile()

`apply_karhunen_loeve_scaling()`

apply karhuene-loeve scaling to the jacobian matrix.

Note:

This scaling is not necessary for analyses using Schur's
complement, but can be very important for error variance
analyses.  This operation effectively transfers prior knowledge
specified in the parcov to the jacobian and reset parcov to the
identity matrix.

`clean()`

drop regularization and prior information observation from the jco

`draw(num_reals=1, par_file=None, obs=False, enforce_bounds=None, cov=None, how='gaussian')`

draw stochastic realizations of parameters and optionally observations, filling MonteCarlo.parensemble and optionally MonteCarlo.obsensemble.

Parameters

num_reals : int number of realization to generate par_file : str parameter file to use as mean values. If None, use MonteCarlo.pst.parameter_data.parval1. Default is None obs : bool add a realization of measurement noise to observation values, forming MonteCarlo.obsensemble.Default is False enforce_bounds : str enforce parameter bounds based on control file information. options are 'reset', 'drop' or None. Default is None how : str type of distribution to draw from. Must be in ["gaussian","uniform"] default is "gaussian".

Example

>>>import pyemu

>>>mc = pyemu.MonteCarlo(pst="pest.pst")

>>>mc.draw(1000)

`drop_prior_information()`

drop the prior information from the jco and pst attributes

`get(par_names=None, obs_names=None, astype=None)`

method to get a new LinearAnalysis class using a subset of parameters and/or observations

Parameters:

Name	Type	Description	Default
`par_names`	[`'str`]	par names for new object	`None`
`obs_names`	[`'str`]	obs names for new object	`None`
`astype`	`pyemu.Schur` or `pyemu.ErrVar`	type to cast the new object. If None, return type is same as self	`None`

Returns:

Type	Description
	`LinearAnalysis`: new instance

`get_cso_dataframe()`

get a dataframe of composite observation sensitivity, as returned by PEST in the seo file.

Returns:

Type	Description
	`pandas.DataFrame`: dataframe of observation names and composite observation
	sensitivity

Note

That this formulation deviates slightly from the PEST documentation in that the values are divided by (npar-1) rather than by (npar).

The equation is cso_j = ((Q^1/2JJ^T*Q^1/2)^1/2)_jj/(NPAR-1)

`get_nsing(epsilon=0.0001)`

get the number of solution space dimensions given a ratio between the largest and smallest singular values

Parameters

epsilon: float singular value ratio

Returns

nsing : float number of singular components above the epsilon ratio threshold

Note

If nsing == nadj_par, then None is returned

`get_null_proj(nsing=None)`

get a null-space projection matrix of XTQX

Parameters

nsing: int optional number of singular components to use If None, then nsing is determined from call to MonteCarlo.get_nsing()

Returns

v2_proj : pyemu.Matrix the null-space projection matrix (V2V2^T)

`get_obs_competition_dataframe()`

get the observation competition stat a la PEST utility

Returns:

Type	Description
	`pandas.DataFrame`: a dataframe of observation names by
	observation names with values equal to the PEST
	competition statistic

`get_par_css_dataframe()`

get a dataframe of composite scaled sensitivities. Includes both PEST-style and Hill-style.

Returns:

Type	Description
	`pandas.DataFrame`: a dataframe of parameter names, PEST-style and
	Hill-style composite scaled sensitivity

`project_parensemble(par_file=None, nsing=None, inplace=True, enforce_bounds='reset')`

perform the null-space projection operations for null-space monte carlo

Parameters

par_file: str an optional file of parameter values to use nsing: int number of singular values to in forming null subspace matrix inplace: bool overwrite the existing parameter ensemble with the projected values enforce_bounds: str how to enforce parameter bounds. can be None, 'reset', or 'drop'. Default is None

Returns

par_en : pyemu.ParameterEnsemble if inplace is False, otherwise None

Note

to use this method, the MonteCarlo instance must have been constructed with the jco argument.

Example

>>>import pyemu

>>>mc = pyemu.MonteCarlo(jco="pest.jcb")

>>>mc.draw(1000)

>>>mc.project_parensemble(par_file="final.par",nsing=100)

`reset_obscov(arg=None)`

reset the obscov attribute to None

Parameters:

Name	Type	Description	Default
`arg`	`str` or `pyemu.Matrix`	the value to assign to the obscov attribute. If None, the private __obscov attribute is cleared but not reset	`None`

`reset_parcov(arg=None)`

reset the parcov attribute to None

Parameters:

Name	Type	Description	Default
`arg`	`str` or `pyemu.Matrix`	the value to assign to the parcov attribute. If None, the private __parcov attribute is cleared but not reset	`None`

`reset_pst(arg)`

reset the LinearAnalysis.pst attribute

Parameters:

Name	Type	Description	Default
`arg`	`str` or `pyemu.Pst`	the value to assign to the pst attribute	required

`write_psts(prefix, existing_jco=None, noptmax=None)`

write parameter and optionally observation realizations to a series of pest control files

Parameters

prefix: str pest control file prefix

str

filename of an existing jacobian matrix to add to the pest++ options in the control file. This is useful for NSMC since this jco can be used to get the first set of parameter upgrades for free! Needs to be the path the jco file as seen from the location where pest++ will be run

int

value of NOPTMAX to set in new pest control files

Example

>>>import pyemu

>>>mc = pyemu.MonteCarlo(jco="pest.jcb")

>>>mc.draw(1000, obs=True)

>>>mc.write_psts("mc_", existing_jco="pest.jcb", noptmax=1)

mc

MonteCarlo

Parameters

Attributes

Returns

Example

adj_par_names property

fehalf property

forecast_names property

forecasts property

forecasts_iter property

jco property

mle_covariance property

mle_parameter_estimate property

nnz_obs_names property

num_reals property

Returns

obscov property

parcov property

predictions property

predictions_iter property

prior_forecast property

prior_parameter property

prior_prediction property

pst property

qhalf property

qhalfx property

xtqx property

__fromfile(filename, astype=None)

__load_jco()

__load_obscov()

__load_parcov()

__load_predictions()

__load_pst()

adjust_obscov_resfile(resfile=None)

apply_karhunen_loeve_scaling()

clean()

draw(num_reals=1, par_file=None, obs=False, enforce_bounds=None, cov=None, how='gaussian')

Parameters

Example

drop_prior_information()

get(par_names=None, obs_names=None, astype=None)

get_cso_dataframe()

get_nsing(epsilon=0.0001)

Parameters

Returns

Note

get_null_proj(nsing=None)

Parameters

Returns

get_obs_competition_dataframe()

get_par_css_dataframe()

project_parensemble(par_file=None, nsing=None, inplace=True, enforce_bounds='reset')

Parameters

Returns

Note

Example

reset_obscov(arg=None)

reset_parcov(arg=None)

reset_pst(arg)

write_psts(prefix, existing_jco=None, noptmax=None)

Parameters

Example

`MonteCarlo`

`adj_par_names` `property`

`fehalf` `property`

`forecast_names` `property`

`forecasts` `property`

`forecasts_iter` `property`

`jco` `property`

`mle_covariance` `property`

`mle_parameter_estimate` `property`

`nnz_obs_names` `property`

`num_reals` `property`

`obscov` `property`

`parcov` `property`

`predictions` `property`

`predictions_iter` `property`

`prior_forecast` `property`

`prior_parameter` `property`

`prior_prediction` `property`

`pst` `property`

`qhalf` `property`

`qhalfx` `property`

`xtqx` `property`

`__fromfile(filename, astype=None)`

`__load_jco()`

`__load_obscov()`

`__load_parcov()`

`__load_predictions()`

`__load_pst()`

`adjust_obscov_resfile(resfile=None)`

`apply_karhunen_loeve_scaling()`

`clean()`

`draw(num_reals=1, par_file=None, obs=False, enforce_bounds=None, cov=None, how='gaussian')`

`drop_prior_information()`

`get(par_names=None, obs_names=None, astype=None)`

`get_cso_dataframe()`

`get_nsing(epsilon=0.0001)`

`get_null_proj(nsing=None)`

`get_obs_competition_dataframe()`

`get_par_css_dataframe()`

`project_parensemble(par_file=None, nsing=None, inplace=True, enforce_bounds='reset')`

`reset_obscov(arg=None)`

`reset_parcov(arg=None)`

`reset_pst(arg)`

`write_psts(prefix, existing_jco=None, noptmax=None)`