pyemu.pst.pst_handler

Module Contents

Classes

Pst

All things PEST(++) control file

Functions

get_constraint_tags([ltgt])

_check_and_reject_nans(df, name)

_replace_str_in_files(filelist, name_dict[, ...])

_multiprocess_obspar_rename(sys_file, map_dict[, rex])

_multiprocess_obspar_rename_v3(sys_file, map_dict[, rex])

pyemu.pst.pst_handler.get_constraint_tags(ltgt='lt')
pyemu.pst.pst_handler._check_and_reject_nans(df, name)
class pyemu.pst.pst_handler.Pst(filename, load=True, resfile=None)

Bases: object

All things PEST(++) control file

Parameters:

  • filename (str) – the name of the control file

  • load (bool, optional) – flag to load the control file. Default is True

  • resfile (str, optional) – corresponding residual file. If None, a residual file with the control file base name is sought. Default is None

Note

This class is the primary mechanism for dealing with PEST control files. Support is provided for constructing new control files as well as manipulating existing control files.

Example:

pst = pyemu.Pst("my.pst")
pst.control_data.noptmax = -1
pst.write("my_new.pst")
property phi

get the weighted total objective function.

Returns:

sum of squared residuals

Return type:

float

Note

Requires Pst.res (the residuals file) to be available

property phi_components

get the individual components of the total objective function

Returns:

dictionary of observation group, contribution to total phi

Return type:

dict

Note

Requires Pst.res (the residuals file) to be available

property phi_components_normalized
get the individual components of the total objective function

normalized to the total PHI being 1.0

Returns:

dictionary of observation group, normalized contribution to total phi

Return type:

dict

Note

Requires Pst.res (the residuals file) to be available

property res

get the residuals dataframe attribute

Returns:

a dataframe containing the residuals information.

Return type:

pandas.DataFrame

Note

if the Pst.__res attribute has not been loaded, this call loads the res dataframe from a file

Example:

# print the observed and simulated values for non-zero weighted obs
print(pst.res.loc[pst.nnz_obs_names,["modelled","measured"]])
property nprior

number of prior information equations

Returns:

the number of prior info equations

Return type:

int

property nnz_obs

get the number of non-zero weighted observations

Returns:

the number of non-zeros weighted observations

Return type:

int

property nobs

get the number of observations

Returns:

the number of observations

Return type:

int

property npar_adj

get the number of adjustable parameters (not fixed or tied)

Returns:

the number of adjustable parameters

Return type:

int

property npar

get number of parameters

Returns:

the number of parameters

Return type:

int

property forecast_names

get the forecast names from the pestpp options (if any). Returns None if no forecasts are named

Returns:

a list of forecast names.

Return type:

[str]

property obs_groups

get the observation groups

Returns:

a list of unique observation groups

Return type:

[str]

property nnz_obs_groups
get the observation groups that contain at least one non-zero weighted

observation

Returns:

a list of observation groups that contain at least one non-zero weighted observation

Return type:

[str]

property adj_par_groups

get the parameter groups with atleast one adjustable parameter

Returns:

a list of parameter groups with at least one adjustable parameter

Return type:

[str]

property par_groups

get the parameter groups

Returns:

a list of parameter groups

Return type:

[str]

property prior_groups

get the prior info groups

Returns:

a list of prior information groups

Return type:

[str]

property prior_names

get the prior information names

Returns:

a list of prior information names

Return type:

[str]

property par_names

get the parameter names

Returns:

a list of parameter names

Return type:

[str]

property adj_par_names

get the adjustable (not fixed or tied) parameter names

Returns:

list of adjustable (not fixed or tied) parameter names

Return type:

[str]

property obs_names

get the observation names

Returns:

a list of observation names

Return type:

[str]

property nnz_obs_names

get the non-zero weight observation names

Returns:

a list of non-zero weighted observation names

Return type:

[str]

property zero_weight_obs_names

get the zero-weighted observation names

Returns:

a list of zero-weighted observation names

Return type:

[str]

property estimation

check if the control_data.pestmode is set to estimation

Returns:

True if control_data.pestmode is estmation, False otherwise

Return type:

bool

property tied

list of tied parameter names

Returns:

a dataframe of tied parameter information. Columns of tied are parnme and partied. Returns None if no tied parameters are found.

Return type:

pandas.DataFrame

property template_files

list of template file names

Returns:

a list of template file names, extracted from

Pst.model_input_data.pest_file. Returns None if this attribute is None

Return type:

[str]

Note

Use Pst.model_input_data to access the template-input file information for writing/modification

property input_files

list of model input file names

Returns:

a list of model input file names, extracted from

Pst.model_input_data.model_file. Returns None if this attribute is None

Return type:

[str]

Note

Use Pst.model_input_data to access the template-input file information for writing/modification

property instruction_files

list of instruction file names :returns:

a list of instruction file names, extracted from
Pst.model_output_data.pest_file. Returns None if this

attribute is None

Return type:

[str]

Note

Use Pst.model_output_data to access the instruction-output file information for writing/modification

property output_files

list of model output file names :returns:

a list of model output file names, extracted from

Pst.model_output_data.model_file. Returns None if this attribute is None

Return type:

[str]

Note

Use Pst.model_output_data to access the instruction-output file information for writing/modification

property less_than_obs_constraints

get the names of the observations that are listed as active (non-zero weight) less than inequality constraints.

Returns:

names of observations that are non-zero weighted less than constraints (obgnme starts with ‘l_’ or “less”)

Return type:

pandas.Series

Note

Zero-weighted obs are skipped

property less_than_pi_constraints

get the names of the prior information eqs that are listed as active (non-zero weight) less than inequality constraints.

Returns:

names of prior information that are non-zero weighted less than constraints (obgnme starts with “l_” or “less”)

Return type:

pandas.Series

Note

Zero-weighted pi are skipped

property greater_than_obs_constraints

get the names of the observations that are listed as active (non-zero weight) greater than inequality constraints.

Returns:

names obseravtions that are non-zero weighted greater than constraints (obgnme startsiwth “g_” or “greater”)

Return type:

pandas.Series

Note

Zero-weighted obs are skipped

property greater_than_pi_constraints

get the names of the prior information eqs that are listed as active (non-zero weight) greater than inequality constraints.

Returns:

pandas.Series names of prior information that are non-zero weighted greater than constraints (obgnme startsiwth “g_” or “greater”)

Note

Zero-weighted pi are skipped

parameter_data

‘* parameter data’ information. Columns are standard PEST variable names

Example:

pst.parameter_data.loc[:,"partrans"] = "log"
pst.parameter_data.loc[:,"parubnd"] = 10.0
Type:

pandas.DataFrame

observation_data

‘* observation data’ information. Columns are standard PEST variable names

Example:

pst.observation_data.loc[:,"weight"] = 1.0
pst.observation_data.loc[:,"obgnme"] = "obs_group"
Type:

pandas.DataFrame

prior_information

‘* prior information’ data. Columns are standard PEST variable names

Type:

pandas.DataFrame

control_data

‘* control data’ information. Access with standard PEST variable names

Example:

pst.control_data.noptmax = 2
pst.control_data.pestmode = "estimation"
Type:

pyemu.pst.pst_controldata.ControlData

svd_data

‘* singular value decomposition’ section information. Access with standard PEST variable names

Example:

pst.svd_data.maxsing = 100
Type:

pyemu.pst.pst_controldata.SvdData

reg_data

‘* regularization’ section information. Access with standard PEST variable names.

Example:

pst.reg_data.phimlim = 1.00 #yeah right!
Type:

pyemu.pst.pst_controldata.RegData

__setattr__(key, value)

Implement setattr(self, name, value).

classmethod from_par_obs_names(par_names=['par1'], obs_names=['obs1'])

construct a shell Pst instance from parameter and observation names

Parameters:

  • par_names ([str]) – list of parameter names. Default is [par1]

  • obs_names ([str]) – list of observation names. Default is [obs1]

Note

While this method works, it does not make template or instruction files. Users are encouraged to use Pst.from_io_files() for more usefulness

Example:

par_names = ["par1","par2"]
obs_names = ["obs1","obs2"]
pst = pyemu.Pst.from_par_obs_names(par_names,obs_names)
static get_phi_components(ogroups, res, obs, pi_ogroups=None, pi_df=None)
set_res(res)

reset the private Pst.res attribute.

Parameters:

res – (pandas.DataFrame or str): something to use as Pst.res attribute. If res is str, a dataframe is read from file res

static _read_df(f, nrows, names, converters, defaults=None)

a private method to read part of an open file into a pandas.DataFrame.

Parameters:

  • f (file) – open file handle

  • nrows (int) – number of rows to read

  • names ([str]) – names to set the columns of the dataframe with

  • converters (dict) – dictionary of lambda functions to convert strings to numerical format

  • defaults (dict) – dictionary of default values to assign columns. Default is None

Returns:

dataframe of control file section info

Return type:

pandas.DataFrame

_read_line_comments(f, forgive)

private method to read comment lines from a control file

_read_section_comments(f, forgive)

private method to read comments from a section of the control file

static _parse_external_line(line, pst_path='.')

private method to parse a file for external file info

static _parse_path_agnostic(filename)

private method to parse a file path for any os sep

static _cast_df_from_lines(section, lines, fieldnames, converters, defaults, alias_map={}, pst_path='.')

private method to cast a pandas dataframe from raw control file lines

_cast_prior_df_from_lines(section, lines, pst_path='.')

private method to cast prior information lines to a dataframe

_load_version2(filename)

private method to load a version 2 control file

load(filename)

entry point load the pest control file.

Parameters:

filename (str) – pst filename

Note

This method is called from the Pst construtor unless the load arg is False.

_reset_file_paths_os()
_try_load_longnames()
_parse_pestpp_line(line)
_update_control_section()

private method to synchronize the control section counters with the various parts of the control file. This is usually called during the Pst.write() method.

rectify_pgroups()

synchronize parameter groups section with the parameter data section

Note

This method is called during Pst.write() to make sure all parameter groups named in * parameter data are included. This is so users don’t have to manually keep this section up. This method can also be called during control file modifications to see what parameter groups are present and prepare for modifying the default values in the * parameter group section

Example:

pst = pyemu.Pst("my.pst")
pst.parameter_data.loc["par1","pargp"] = "new_group"
pst.rectify_groups()
pst.parameter_groups.loc["new_group","derinc"] = 1.0
_parse_pi_par_names()

private method to get the parameter names from prior information equations. Sets a ‘names’ column in Pst.prior_information that is a list of parameter names

add_pi_equation(par_names, pilbl=None, rhs=0.0, weight=1.0, obs_group='pi_obgnme', coef_dict={})

a helper to construct a new prior information equation.

Parameters:

  • par_names ([str]) – parameter names in the equation

  • pilbl (str) – name to assign the prior information equation. If None, a generic equation name is formed. Default is None

  • rhs (float) – the right-hand side of the pi equation

  • weight (float) – the weight of the equation

  • obs_group (str) – the observation group for the equation. Default is ‘pi_obgnme’

  • coef_dict (dict) – a dictionary of parameter name, coefficient pairs to assign leading coefficients for one or more parameters in the equation. If a parameter is not listed, 1.0 is used for its coefficients. Default is {}

Example:

pst = pyemu.Pst("pest.pst")
# add a pi equation for the first adjustable parameter
pst.add_pi_equation(pst.adj_par_names[0],pilbl="pi1",rhs=1.0)
# add a pi equation for 1.5 times the 2nd and 3 times the 3rd adj pars to sum together to 2.0
names = pst.adj_par_names[[1,2]]
pst.add_pi_equation(names,coef_dict={names[0]:1.5,names[1]:3})
rectify_pi()

rectify the prior information equation with the current state of the parameter_data dataframe.

Note

Equations that list fixed, tied or missing parameters are removed completely even if adjustable parameters are also listed in the equation. This method is called during Pst.write()

_write_df(name, f, df, formatters, columns)

private method to write a dataframe to a control file

sanity_checks(forgive=False)

some basic check for strangeness

Parameters:

forgive (bool) – flag to forgive (warn) for issues. Default is False

Note

checks for duplicate names, atleast 1 adjustable parameter and at least 1 non-zero-weighted observation

Not nearly as comprehensive as pestchek

Example:

pst = pyemu.Pst("pest.pst")
pst.sanity_checks()
_write_version2(new_filename, use_pst_path=True, pst_rel_path='.')

private method to write a version 2 control file

write(new_filename, version=None, check_interface=False)

main entry point to write a pest control file.

Parameters:

  • new_filename (str) – name of the new pest control file

  • version (int) – flag for which version of control file to write (must be 1 or 2). if None, uses the number of pars to decide: if number of pars iis greater than 10,000, version 2 is used.

  • check_interface (bool) – flag to check the control file par and obs names against the names found in the template and instruction files. Default is False

Example:

pst = pyemu.Pst("my.pst")
pst.parrep("my.par")
pst.write(my_new.pst")
#write a version 2 control file
pst.write("my_new_v2.pst",version=2)
_rectify_parchglim()

private method to just fix the parchglim vs cross zero issue

_write_version1(new_filename)

private method to write a version 1 pest control file

bounds_report(iterations=None)

report how many parameters are at bounds. If ensemble, the base enbsemble member is evaluated

Parameters:

iterations ([int]) – a list of iterations for which a bounds report is requested If None, all iterations for which par files are located are reported. Default is None

Returns:

a pandas DataFrame object with rows being parameter groups and columns

<iter>_num_at_ub, <iter>_num_at_lb, and <iter>_total_at_bounds row 0 is total at bounds, subsequent rows correspond with groups

Return type:

df

Example

pst = pyemu.Pst(“my.pst”) df = pst.bound_report(iterations=[0,2,3])

get(par_names=None, obs_names=None)

get a new pst object with subset of parameters and/or observations

Parameters:

  • par_names ([str]) – a list of parameter names to have in the new Pst instance. If None, all parameters are in the new Pst instance. Default is None

  • obs_names ([str]) – a list of observation names to have in the new Pst instance. If None, all observations are in teh new Pst instance. Default is None

Returns:

a new Pst instance

Return type:

Pst

Note

passing par_names as None and obs_names as None effectively generates a copy of the current Pst

Does not modify model i/o files - this is just a method for performing pyemu operations

Example:

pst = pyemu.Pst("pest.pst")
new_pst = pst.get(pst.adj_par_names[0],pst.obs_names[:10])
parrep(parfile=None, enforce_bounds=True, real_name=None, noptmax=0, binary_ens_file=False)
replicates the pest parrep util. replaces the parval1 field in the

parameter data section dataframe with values in a PEST parameter file or a single realization from an ensemble parameter csv file

Parameters:

  • parfile (str, optional) – parameter file to use. If None, try to find and use a parameter file that corresponds to the case name. If parfile has extension ‘.par’ a single realization parameter file is used If parfile has extention ‘.csv’ an ensemble parameter file is used which invokes real_name If parfile has extention ‘.jcb’ a binary ensemble parameter file is used which invokes real_name Default is None

  • enforce_bounds (bool, optional) – flag to enforce parameter bounds after parameter values are updated. This is useful because PEST and PEST++ round the parameter values in the par file, which may cause slight bound violations. Default is True

  • real_name (str or int, optional) – name of the ensemble realization to use for updating the parval1 value in the parameter data section dataframe. If None, try using “base”. If “base” not present, use the real_name with smallest index number. Ignored if parfile is of the PEST parameter file format (e.g. not en ensemble)

  • noptmax (int, optional) – Value with which to update the pst.control_data.noptmax value Default is 0.

  • binary_ens_file (bool) – If True, use binary format to load ensemble file, else assume it’s a CSV file

Example:

pst = pyemu.Pst("pest.pst")
pst.parrep("pest.1.base.par")
pst.control_data.noptmax = 0
pst.write("pest_1.pst")
adjust_weights_discrepancy(resfile=None, original_ceiling=True, bygroups=False)

adjusts the weights of each non-zero weight observation based on the residual in the pest residual file so each observations contribution to phi is 1.0 (e.g. Mozorov’s discrepancy principal)

Parameters:

  • resfile (str) – residual file name. If None, try to use a residual file with the Pst case name. Default is None

  • original_ceiling (bool) – flag to keep weights from increasing - this is generally a good idea. Default is True

  • bygroups (bool) – flag to adjust weights by groups. If False, the weight of each non-zero weighted observation is adjusted individually. If True, intergroup weighting is preserved (the contribution to each group is used) but this may result in some strangeness if some observations in a group have a really low phi already.

Example:

pst = pyemu.Pst("my.pst")
print(pst.phi) #assumes "my.res" is colocated with "my.pst"
pst.adjust_weights_discrepancy()
print(pst.phi) # phi should equal number of non-zero observations
_adjust_weights_by_phi_components(components, original_ceiling)

private method that resets the weights of observations by group to account for residual phi components.

Parameters:

  • components (dict) – a dictionary of obs group:phi contribution pairs

  • original_ceiling (bool) – flag to keep weights from increasing.

__reset_weights(target_phis, res_idxs, obs_idxs)

private method to reset weights based on target phi values for each group. This method should not be called directly

Parameters:

  • target_phis (dict) – target phi contribution for groups to reweight

  • res_idxs (dict) – the index positions of each group of interest in the res dataframe

  • obs_idxs (dict) – the index positions of each group of interest in the observation data dataframe

_adjust_weights_by_list(obslist, weight)

a private method to reset the weight for a list of observation names. Supports the data worth analyses in pyemu.Schur class. This method only adjusts observation weights in the current weight is nonzero. User beware!

Parameters:

  • obslist ([str]) – list of observation names

  • weight (float) – new weight to assign

adjust_weights(obs_dict=None, obsgrp_dict=None)

reset the weights of observations or observation groups to contribute a specified amount to the composite objective function

Parameters:

  • obs_dict (dict, optional) – dictionary of observation name,new contribution pairs

  • obsgrp_dict (dict, optional) – dictionary of obs group name,contribution pairs

Notes

If a group is assigned a contribution of 0, all observations in that group will be assigned zero weight.

If a group is assigned a nonzero contribution AND all observations in that group start with zero weight, the observations will be assigned weight of 1.0 to allow balancing.

If groups obsgrp_dict is not passed, all nonzero

Example:

pst = pyemu.Pst("my.pst")

# adjust a single observation
pst.adjust_weights(obs_dict={"obs1":10})

# adjust a single observation group
pst.adjust_weights(obsgrp_dict={"group1":100.0})

# make all non-zero weighted groups have a contribution of 100.0
balanced_groups = {grp:100 for grp in pst.nnz_obs_groups}
pst.adjust_weights(obsgrp_dict=balanced_groups)
proportional_weights(fraction_stdev=1.0, wmax=100.0, leave_zero=True)

setup weights inversely proportional to the observation value

Parameters:

  • fraction_stdev (float, optional) – the fraction portion of the observation val to treat as the standard deviation. set to 1.0 for inversely proportional. Default is 1.0

  • wmax (float, optional) – maximum weight to allow. Default is 100.0

  • leave_zero (bool, optional) – flag to leave existing zero weights. Default is True

Example:

pst = pyemu.Pst("pest.pst")
# set the weights of the observations to 20% of the observed value
pst.proportional_weights(fraction_stdev=0.2,wmax=10)
pst.write("pest_propo.pst")
calculate_pertubations()

experimental method to calculate finite difference parameter pertubations.

Note

The pertubation values are added to the Pst.parameter_data attribute - user beware!

build_increments()

experimental method to calculate parameter increments for use in the finite difference pertubation calculations

Note

user beware!

add_transform_columns()

add transformed values to the Pst.parameter_data attribute

Note

adds parval1_trans, parlbnd_trans and parubnd_trans to Pst.parameter_data

Example:

pst = pyemu.Pst("pest.pst")
pst.add_transform_columns()
print(pst.parameter_data.parval1_trans
enforce_bounds()

enforce bounds violation

Note

cheap enforcement of simply bringing violators back in bounds

Example:

pst = pyemu.Pst("pest.pst")
pst.parrep("random.par")
pst.enforce_bounds()
pst.write("pest_rando.pst")
classmethod from_io_files(tpl_files, in_files, ins_files, out_files, pst_filename=None, pst_path=None)

create a Pst instance from model interface files.

Parameters:

  • tpl_files ([str]) – list of template file names

  • in_files ([str]) – list of model input file names (pairs with template files)

  • ins_files ([str]) – list of instruction file names

  • out_files ([str]) – list of model output file names (pairs with instruction files)

  • pst_filename (str) – name of control file to write. If None, no file is written. Default is None

  • pst_path ('str') – the path from the control file to the IO files. For example, if the control will be in the same directory as the IO files, then pst_path should be ‘.’. Default is None, which doesnt do any path manipulation on the I/O file names

Returns:

new control file instance with parameter and observation names found in tpl_files and ins_files, repsectively.

Return type:

Pst

Note

calls pyemu.helpers.pst_from_io_files()

Assigns generic values for parameter info. Tries to use INSCHEK to set somewhat meaningful observation values

all file paths are relatively to where python is running.

Example:

tpl_files = ["my.tpl"]
in_files = ["my.in"]
ins_files = ["my.ins"]
out_files = ["my.out"]
pst = pyemu.Pst.from_io_files(tpl_files,in_files,ins_files,out_files)
pst.control_data.noptmax = 0
pst.write("my.pst)
add_parameters(template_file, in_file=None, pst_path=None)

add new parameters to an existing control file

Parameters:

  • template_file (str) – template file with (possibly) some new parameters

  • in_file (str) – model input file. If None, template_file.replace(‘.tpl’,’’) is used. Default is None.

  • pst_path (str) – the path to append to the template_file and in_file in the control file. If not None, then any existing path in front of the template or in file is split off and pst_path is prepended. If python is being run in a directory other than where the control file will reside, it is useful to pass pst_path as .. Default is None

Returns:

the data for the new parameters that were added. If no new parameters are in the new template file, returns None

Return type:

pandas.DataFrame

Note

populates the new parameter information with default values

Example:

pst = pyemu.Pst(os.path.join("template","my.pst"))
pst.add_parameters(os.path.join("template","new_pars.dat.tpl",pst_path=".")
pst.write(os.path.join("template","my_new.pst")
drop_observations(ins_file, pst_path=None)

remove observations in an instruction file from the control file

Parameters:

  • ins_file (str) – instruction file to remove

  • pst_path (str) – the path to append to the instruction file in the control file. If not None, then any existing path in front of the instruction is split off and pst_path is prepended. If python is being run in a directory other than where the control file will reside, it is useful to pass pst_path as .. Default is None

Returns:

the observation data for the observations that were removed.

Return type:

pandas.DataFrame

Example:

pst = pyemu.Pst(os.path.join("template", "my.pst"))
pst.remove_observations(os.path.join("template","some_obs.dat.ins"), pst_path=".")
pst.write(os.path.join("template", "my_new_with_less_obs.pst")
drop_parameters(tpl_file, pst_path=None)

remove parameters in a template file from the control file

Parameters:

  • tpl_file (str) – template file to remove

  • pst_path (str) – the path to append to the template file in the control file. If not None, then any existing path in front of the template or in file is split off and pst_path is prepended. If python is being run in a directory other than where the control file will reside, it is useful to pass pst_path as .. Default is None

Returns:

the parameter data for the parameters that were removed.

Return type:

pandas.DataFrame

Note

This method does not check for multiple occurences of the same parameter name(s) in across template files so if you have the same parameter in multiple template files, this is not the method you are looking for

Example:

pst = pyemu.Pst(os.path.join("template", "my.pst"))
pst.remove_parameters(os.path.join("template","boring_zone_pars.dat.tpl"), pst_path=".")
pst.write(os.path.join("template", "my_new_with_less_pars.pst")
add_observations(ins_file, out_file=None, pst_path=None, inschek=True)

add new observations to a control file

Parameters:

  • ins_file (str) – instruction file with exclusively new observation names

  • out_file (str) – model output file. If None, then ins_file.replace(“.ins”,””) is used. Default is None

  • pst_path (str) – the path to append to the instruction file and out file in the control file. If not None, then any existing path in front of the template or in file is split off and pst_path is prepended. If python is being run in a directory other than where the control file will reside, it is useful to pass pst_path as .. Default is None

  • inschek (bool) – flag to try to process the existing output file using the pyemu.InstructionFile class. If successful, processed outputs are used as obsvals

Returns:

the data for the new observations that were added

Return type:

pandas.DataFrame

Note

populates the new observation information with default values

Example:

pst = pyemu.Pst(os.path.join("template", "my.pst"))
pst.add_observations(os.path.join("template","new_obs.dat.ins"), pst_path=".")
pst.write(os.path.join("template", "my_new.pst")
write_input_files(pst_path='.')

writes model input files using template files and current parval1 values.

Parameters:

pst_path (str) – the path to where control file and template files reside. Default is ‘.’

Note

adds “parval1_trans” column to Pst.parameter_data that includes the effect of scale and offset

Example:

pst = pyemu.Pst("my.pst")

# load final parameter values
pst.parrep("my.par")

# write new model input files with final parameter values
pst.write_input_files()
process_output_files(pst_path='.')

processing the model output files using the instruction files and existing model output files.

Parameters:

pst_path (str) – relative path from where python is running to where the control file, instruction files and model output files are located. Default is “.” (current python directory)

Returns:

model output values

Return type:

pandas.Series

Note

requires a complete set of model input files at relative path from where python is running to pst_path

Example:

pst = pyemu.Pst("pest.pst")
obsvals = pst.process_output_files()
print(obsvals)
get_res_stats(nonzero=True)

get some common residual stats by observation group.

Parameters:

nonzero (bool) – calculate stats using only nonzero-weighted observations. This may seem obsvious to most users, but you never know….

Returns:

a dataframe with columns for groups names and indices of statistic name.

Return type:

pd.DataFrame

Note

Stats are derived from the current obsvals, weights and grouping in Pst.observation_data and the modelled values in Pst.res. The key here is ‘current’ because if obsval, weights and/or groupings have changed in Pst.observation_data since the residuals file was generated then the current values for obsval, weight and group are used

the normalized RMSE is normalized against the obsval range (max - min)

Example:

pst = pyemu.Pst("pest.pst")
stats_df = pst.get_res_stats()
print(stats_df.loc["mae",:])
static _stats_rss(df)
static _stats_mean(df)
static _stats_mae(df)
static _stats_rmse(df)
static _stats_nrmse(df)
plot(kind=None, **kwargs)

method to plot various parts of the control. This is sweet as!

Parameters:

  • kind (str) – options are ‘prior’ (prior parameter histograms, ‘1to1’ (line of equality and sim vs res), ‘obs_v_sim’ (time series using datetime suffix), ‘phi_pie’ (pie chart of phi components)

  • kwargs (dict) – optional args for plots that are passed to pyemu plot helpers and ultimately to matplotlib

Note

Depending on ‘kind’ argument, a multipage pdf is written

Example:

pst = pyemu.Pst("my.pst")
pst.plot(kind="1to1") # requires Pst.res
pst.plot(kind="prior")
pst.plot(kind="phi_pie")
write_par_summary_table(filename=None, group_names=None, sigma_range=4.0, report_in_linear_space=False)

write a stand alone parameter summary latex table or Excel sheet

Parameters:

  • filename (str) – filename. If None, use <case>.par.tex to write as LaTeX. If filename extention is ‘.xls’ or ‘.xlsx’, tries to write as an Excel file. If filename is “none”, no table is written Default is None

  • group_names (dict) – par group names : table names. For example {“w0”:”well stress period 1”}. Default is None

  • sigma_range (float) – number of standard deviations represented by parameter bounds. Default is 4.0, implying 95% confidence bounds

  • report_in_linear_space (bool) – flag, if True, that reports all logtransformed values in linear space. This renders standard deviation meaningless, so that column is skipped

Returns:

the summary parameter group dataframe

Return type:

pandas.DataFrame

Example:

pst = pyemu.Pst("my.pst")
pst.write_par_summary_table(filename="par.tex")
write_obs_summary_table(filename=None, group_names=None)
write a stand alone observation summary latex table or Excel shet
filename (str): filename. If None, use <case>.par.tex to write as LaTeX. If filename extention is ‘.xls’ or ‘.xlsx’,

tries to write as an Excel file. If filename is “none”, no table is written Default is None

Parameters:

  • filename (str) – filename. If filename is “none”, no table is written. If None, use <case>.obs.tex. If filename extention is ‘.xls’ or ‘.xlsx’, tries to write as an Excel file. Default is None

  • group_names (dict) – obs group names : table names. For example {“hds”:”simulated groundwater level”}. Default is None

Returns:

the summary observation group dataframe

Return type:

pandas.DataFrame

Example:

pst = pyemu.Pst("my.pst")
pst.write_obs_summary_table(filename="obs.tex")
get_par_change_limits()

calculate the various parameter change limits used in pest.

Returns:

a copy of Pst.parameter_data with columns for relative and factor change limits

Return type:

pandas.DataFrame

Note

does not yet support absolute parameter change limits!

Works in control file values space (not log transformed space). Also adds columns for effective upper and lower which account for par bounds and the value of parchglim

example:

pst = pyemu.Pst("pest.pst")
df = pst.get_par_change_limits()
print(df.chg_lower)
get_adj_pars_at_bounds(frac_tol=0.01)

get list of adjustable parameter at/near bounds

Parameters:

frac_tol ('float`) – fractional tolerance of distance to bound. For upper bound, the value parubnd * (1-frac_tol) is used, lower bound uses parlbnd * (1.0 + frac_tol)

Returns:

  • [`str`]: list of parameters at/near lower bound

  • [`str`]: list of parameters at/near upper bound

Return type:

tuple containing

Example:

pst = pyemu.Pst("pest.pst")
at_lb,at_ub = pst.get_adj_pars_at_bounds()
print("pars at lower bound",at_lb)
try_parse_name_metadata()

try to add meta data columns to parameter and observation data based on item names. Used with the PstFrom process.

Note

metadata is identified in key-value pairs that are separated by a colon. each key-value pair is separated from others by underscore

This works with PstFrom style long names

This method is called programmtically during Pst.load()

rename_parameters(name_dict, pst_path='.', tplmap=None)

rename parameters in the control and template files

Parameters:

  • name_dict (dict) – mapping of current to new names.

  • pst_path (str) – the path to the control file from where python is running. Default is “.” (python is running in the same directory as the control file)

Note

no attempt is made to maintain the length of the marker strings in the template files, so if your model is sensitive to changes in spacing in the template file(s), this is not a method for you

This does a lot of string compare, so its gonna be slow as…

Example:

pst = pyemu.Pst(os.path.join("template","pest.pst"))
name_dict = {"par1":"par1_better_name"}
pst.rename_parameters(name_dict,pst_path="template")
rename_observations(name_dict, pst_path='.', insmap=None)

rename observations in the control and instruction files

Parameters:

  • name_dict (dict) – mapping of current to new names.

  • pst_path (str) – the path to the control file from where python is running. Default is “.” (python is running in the same directory as the control file)

Note

This does a lot of string compare, so its gonna be slow as…

Example:

pst = pyemu.Pst(os.path.join("template","pest.pst"))
name_dict = {"obs1":"obs1_better_name"}
pst.rename_observations(name_dict,pst_path="template")
pyemu.pst.pst_handler._replace_str_in_files(filelist, name_dict, file_obsparmap=None, pst_path='.')
pyemu.pst.pst_handler._multiprocess_obspar_rename(sys_file, map_dict, rex=None)
pyemu.pst.pst_handler._multiprocess_obspar_rename_v3(sys_file, map_dict, rex=None)