pyemu.utils.pst_from

Module Contents

Classes

PstFrom

construct high-dimensional PEST(++) interfaces with all the bells and whistles

Functions

_get_datetime_from_str(sdt)

_check_var_len(var, n[, fill])

_load_array_get_fmt(fname[, sep, fullfile])

write_list_tpl(filenames, dfs, name, tpl_filename, ...)

Write template files for a list style input.

_write_direct_df_tpl(in_filename, tpl_filename, df, ...)

Private method to auto-generate parameter or obs names from tabular

_get_use_rows(tpldf, idxcolvals, use_rows, zero_based, ...)

private function to get use_rows index within df based on passed use_rows

_get_index_strfmt(index_cols)

_get_index_strings(df, fmt, zero_based)

_getxy_from_idx(df, get_xy, xy_in_idx, ij_in_idx)

_build_parnames(df, typ, zone_array, index_cols, ...)

_get_tpl_or_ins_df(dfs, name, index_cols, typ[, ...])

Private method to auto-generate parameter or obs names from tabular

write_array_tpl(name, tpl_filename, suffix, par_type)

write a template file for a 2D array.

_check_diff(org_arr, input_filename[, zval])

get_filepath(folder, filename)

Return a path to a file within a folder,

get_relative_filepath(folder, filename)

Like get_filepath(), except

Attributes

DIRECT_PAR_PERCENT_DIFF_TOL

pyemu.utils.pst_from.DIRECT_PAR_PERCENT_DIFF_TOL = 1.0
pyemu.utils.pst_from._get_datetime_from_str(sdt)
pyemu.utils.pst_from._check_var_len(var, n, fill=None)
pyemu.utils.pst_from._load_array_get_fmt(fname, sep=None, fullfile=False)
class pyemu.utils.pst_from.PstFrom(original_d, new_d, longnames=True, remove_existing=False, spatial_reference=None, zero_based=True, start_datetime=None, tpl_subfolder=None, chunk_len=50, echo=True)

Bases: object

construct high-dimensional PEST(++) interfaces with all the bells and whistles

Parameters:

  • original_d (str or Path) – the path to a complete set of model input and output files

  • new_d (str or Path) – the path to where the model files and PEST interface files will be copied/built

  • longnames (bool) – flag to use longer-than-PEST-likes parameter and observation names. Default is True

  • remove_existing (bool) – flag to destroy any existing files and folders in new_d. Default is False

  • spatial_reference (varies) – an object that faciliates geo-locating model cells based on index. Default is None

  • zero_based (bool) – flag if the model uses zero-based indices, Default is True

  • start_datetime (str or Timestamp) – a string that can be case to a datatime instance the represents the starting datetime of the model

  • tpl_subfolder (str) – option to write template files to a subfolder within new_d. Default is False (write template files to new_d).

  • chunk_len (int) – the size of each “chunk” of files to spawn a multiprocessing.Pool member to process. On windows, beware setting this much smaller than 50 because of the overhead associated with spawning the pool. This value is added to the call to apply_list_and_array_pars. Default is 50

  • echo (bool) – flag to echo logger messages to the screen. Default is True

Note

This is the way…

Example:

pf = PstFrom("path_to_model_files","new_dir_with_pest_stuff",start_datetime="1-1-2020")
pf.add_parameters("hk.dat")
pf.add_observations("heads.csv")
pf.build_pst("pest.pst")
pe = pf.draw(100)
pe.to_csv("prior.csv")
property parfile_relations

build up a container of parameter file information. Called programmatically…

_generic_get_xy(args, **kwargs)
_dict_get_xy(arg, **kwargs)
_flopy_sr_get_xy(args, **kwargs)
_flopy_mg_get_xy(args, **kwargs)
parse_kij_args(args, kwargs)

parse args into kij indices. Called programmatically

initialize_spatial_reference()

process the spatial reference argument. Called programmatically

write_forward_run()

write the forward run script. Called by build_pst()

_pivot_par_struct_dict()
_rename_par_struct_dict(mapdict)
build_prior(fmt='ascii', filename=None, droptol=None, chunk=None, sigma_range=6)

Build the prior parameter covariance matrix

Parameters:

  • fmt (str) – the file format to save to. Default is “ASCII”, can be “binary”, “coo”, or “none”

  • filename (str) – the filename to save the cov to

  • droptol (float) – absolute value of prior cov entries that are smaller than droptol are treated as zero.

  • chunk (int) – number of entries to write to binary/coo at once. Default is None (write all elements at once

  • sigma_range (int) – number of standard deviations represented by parameter bounds. Default is 6 (99% confidence). 4 would be approximately 95% confidence bounds

Returns:

the prior parameter covariance matrix

Return type:

pyemu.Cov

Note

This method processes parameters by group names

For really large numbers of parameters (>30K), this method will cause memory errors. Luckily, in most cases, users only want this matrix to generate a prior parameter ensemble and the PstFrom.draw() is a better choice…

draw(num_reals=100, sigma_range=6, use_specsim=False, scale_offset=True)

Draw a parameter ensemble from the distribution implied by the initial parameter values in the control file and the prior parameter covariance matrix.

Parameters:

  • num_reals (int) – the number of realizations to draw

  • sigma_range (int) – number of standard deviations represented by parameter bounds. Default is 6 (99% confidence). 4 would be approximately 95% confidence bounds

  • use_specsim (bool) – flag to use spectral simulation for grid-scale pars (highly recommended). Default is False

  • scale_offset (bool) – flag to apply scale and offset to parameter bounds before calculating prior variance. Dfault is True. If you are using non-default scale and/or offset and you get an exception during draw, try changing this value to False.

Returns:

a prior parameter ensemble

Return type:

pyemu.ParameterEnsemble

Note

This method draws by parameter group

If you are using grid-style parameters, please use spectral simulation (use_specsim=True)

build_pst(filename=None, update=False, version=1)

Build control file from i/o files in PstFrom object. Warning: This builds a pest control file from scratch, overwriting anything already in self.pst object and anything already writen to filename

Parameters:

  • filename (str) – the filename to save the control file to. If None, the name is formed from the PstFrom.original_d ,the orginal directory name from which the forward model was extracted. Default is None. The control file is saved in the PstFrom.new_d directory.

  • update (bool) or (str) – flag to add to existing Pst object and rewrite. If string {‘pars’, ‘obs’} just update respective components of Pst. Default is False - build from PstFrom components.

  • version (int) – control file version to write, Default is 1. If None, option to not write pst to file at pst_build() call – handy when control file is huge pst object will be modified again before running.

Note

This builds a pest control file from scratch, overwriting anything already in self.pst object and anything already writen to filename

The new pest control file is assigned an NOPTMAX value of 0

_setup_dirs()
_par_prep(filenames, index_cols, use_cols, fmts=None, seps=None, skip_rows=None, c_char=None)
_next_count(prefix)
add_py_function(file_name, call_str=None, is_pre_cmd=True, function_name=None)

add a python function to the forward run script

Parameters:

  • file_name (str) – a python source file

  • call_str (str) – the call string for python function in file_name. call_str will be added to the forward run script, as is.

  • is_pre_cmd (bool or None) – flag to include call_str in PstFrom.pre_py_cmds. If False, call_str is added to PstFrom.post_py_cmds instead. If passed as None, then the function call_str is added to the forward run script but is not called. Default is True.

  • function_name (str) – DEPRECATED, used call_str

Returns:

None

Note

call_str is expected to reference standalone a function that contains all the imports it needs or these imports should have been added to the forward run script through the PstFrom.extra_py_imports list.

This function adds the call_str call to the forward run script (either as a pre or post command or function not directly called by main). It is up to users to make sure call_str is a valid python function call that includes the parentheses and requisite arguments

This function expects “def “ + function_name to be flushed left at the outer most indentation level

Example:

pf = PstFrom()
# add the function "mult_well_function" from the script file "preprocess.py" as a
# command to run before the model is run
pf.add_py_function("preprocess.py",
                   "mult_well_function(arg1='userarg')",
                   is_pre_cmd = True)
# add the post processor function "made_it_good" from the script file "post_processors.py"
pf.add_py_function("post_processors.py","make_it_good()",is_pre_cmd=False)
# add the function "another_func" from the script file "utils.py" as a
# function not called by main
pf.add_py_function("utils.py","another_func()",is_pre_cmd=None)
_process_array_obs(out_filename, ins_filename, prefix, ofile_sep, ofile_skip, zone_array)

private method to setup observations for an array-style file

Parameters:

  • out_filename (str) – the output array file

  • ins_filename (str) – the instruction file to create

  • prefix (str) – the prefix to add to the observation names and also to use as the observation group name.

  • ofile_sep (str) – the separator in the output file. This is currently just a placeholder arg, only whitespace-delimited files are supported

  • ofile_skip (int) – number of header and/or comment lines to skip at the top of the output file

  • zone_array (numpy.ndarray) – an integer array used to identify positions to skip in the output array

Returns:

None

Note

This method is called programmatically by PstFrom.add_observations()

add_observations(filename, insfile=None, index_cols=None, use_cols=None, use_rows=None, prefix='', ofile_skip=None, ofile_sep=None, rebuild_pst=False, obsgp=None, zone_array=None, includes_header=True)

Add values in output files as observations to PstFrom object

Parameters:

  • filename (str) – model output file name(s) to set up as observations. By default filename should give relative loction from top level of pest template directory (new_d as passed to PstFrom()).

  • insfile (str) – desired instructions file filename

  • index_cols (list-like or int) – columns to denote are indices for obs

  • use_cols (list-like or int) – columns to set up as obs. If None, and index_cols is not None (i.e list-style obs assumed), observations will be set up for all columns in filename that are not in index_cols.

  • use_rows (list-like or int) – select only specific row of file for obs

  • prefix (str) – prefix for obsnmes

  • ofile_skip (int) – number of lines to skip in model output file

  • ofile_sep (str) – delimiter in output file. If None, the delimiter is eventually governed by the file extension (, for .csv).

  • rebuild_pst (bool) – (Re)Construct PstFrom.pst object after adding new obs

  • obsgp (str of list-like) – observation group name(s). If type str (or list of len == 1) and use_cols is None (i.e. all non-index cols are to be set up as obs), the same group name will be mapped to all obs in call. If None the obs group name will be derived from the base of the constructed observation name. If passed as list (and len(list) = n > 1), the entries in obsgp will be interpreted to explicitly define the grouped for the first n cols in use_cols, any remaining columns will default to None and the base of the observation name will be used. Default is None.

  • zone_array (np.ndarray) – array defining spatial limits or zones for array-style observations. Default is None

  • includes_header (bool) – flag indicating that the list-style file includes a header row. Default is True.

Returns:

dataframe with info for new observations

Return type:

Pandas.DataFrame

Note

This is the main entry for adding observations to the pest interface

If index_cols and use_cols are both None, then it is assumed that array-style observations are being requested. In this case, filenames must be only one filename.

zone_array is only used for array-style observations. Zone values less than or equal to zero are skipped (using the “dum” option)

Example:

# setup observations for the 2nd thru 5th columns of the csv file
# using the first column as the index
df = pf.add_observations("heads.csv",index_col=0,use_cols=[1,2,3,4],
                         ofile_sep=",")
# add array-style observations, skipping model cells with an ibound
# value less than or equal to zero
df = pf.add_observations("conce_array.dat,index_col=None,use_cols=None,
                         zone_array=ibound)
add_observations_from_ins(ins_file, out_file=None, pst_path=None, inschek=True)

add new observations to a control file from an existing instruction file

Parameters:

  • ins_file (str) – instruction file with exclusively new observation names. N.B. if ins_file just contains base filename string (i.e. no directory name), the path to PEST directory will be automatically appended.

  • out_file (str) – model output file. If None, then ins_file.replace(“.ins”,””) is used. Default is None. If out_file just contains base filename string (i.e. no directory name), the path to PEST directory will be automatically appended.

  • pst_path (str) – the path to append to the instruction file and out file in the control file. If not None, then any existing path in front of the template or ins file is split off and pst_path is prepended. If python is being run in a directory other than where the control file will reside, it is useful to pass pst_path as .. Default is None

  • inschek (bool) – flag to try to process the existing output file using the pyemu.InstructionFile class. If successful, processed outputs are used as obsvals

Returns:

the data for the new observations that were

added

Return type:

pandas.DataFrame

Note

populates the new observation information with default values

Example:

pf = pyemu.PstFrom("temp","template")
pf.add_observations_from_ins(os.path.join("template","new_obs.dat.ins"),
                     pst_path=".")
add_parameters(filenames, par_type, zone_array=None, dist_type='gaussian', sigma_range=4.0, upper_bound=None, lower_bound=None, transform=None, par_name_base='p', index_cols=None, use_cols=None, use_rows=None, pargp=None, pp_space=10, use_pp_zones=False, num_eig_kl=100, spatial_reference=None, geostruct=None, datetime=None, mfile_fmt='free', mfile_skip=None, mfile_sep=None, ult_ubound=None, ult_lbound=None, rebuild_pst=False, alt_inst_str='inst', comment_char=None, par_style='multiplier', initial_value=None)

Add list or array style model input files to PstFrom object. This method is the main entry point for adding parameters to the pest interface

Parameters:

  • filenames (str) – Model input filenames to parameterize. By default filename should give relative loction from top level of pest template directory (new_d as passed to PstFrom()).

  • par_type (str) – One of grid - for every element, constant - for single parameter applied to every element, zone - for zone-based parameterization or pilotpoint - for pilot-point base parameterization of array style input files. Note kl not yet implemented # TODO

  • zone_array (np.ndarray) – array defining spatial limits or zones for parameterization.

  • dist_type – not yet implemented # TODO

  • sigma_range – not yet implemented # TODO

  • upper_bound (float) – PEST parameter upper bound. If None, then 1.0e+10 is used. Default is None #

  • lower_bound (float) – PEST parameter lower bound. If None and transform is “log”, then 1.0e-10 is used. Otherwise, if None, -1.0e+10 is used. Default is None

  • transform (str) – PEST parameter transformation. Must be either “log”,”none” or “fixed. The “tied” transform must be used after calling PstFrom.build_pst().

  • par_name_base (str or list-like) – basename for parameters that are set up. If parameter file is tabular list-style file (index_cols is not None) then : len(par_name_base) must equal len(use_cols)

  • index_cols (list-like) – if not None, will attempt to parameterize expecting a tabular-style model input file. index_cols defines the unique columns used to set up pars. If passed as a list of str, stings are expected to denote the columns headers in tabular-style parameter files; if i and j in list, these columns will be used to define spatial position for spatial correlations (if required). WARNING: If passed as list of int, i and j will be assumed to be in last two entries in the list. Can be passed as a dictionary using the keys i and j to explicitly speficy the columns that relate to model rows and columns to be identified and processed to x,y.

  • use_cols (list-like or int) – for tabular-style model input file, defines the columns to be parameterised

  • use_rows (list or tuple) – Setup parameters for only specific rows in list-style model input file. Action is dependent on the the dimensions of use_rows. If ndim(use_rows) < 2: use_rows is assumed to represent the row number, index slicer (equiv df.iloc), for all passed files (after headers stripped). So use_rows=[0,3,5], will parameterise the 1st, 4th and 6th rows of each passed list-like file. If ndim(use_rows) = 2: use_rows represent the index value to paramterise according to index_cols. e.g. [(3,5,6)] or [[3,5,6]] would attempt to set parameters where the model file values for 3 index_cols are 3,5,6. N.B. values in tuple are the actual model file entry values. If no rows in the model input file match use_rows, parameters will be set up for all rows. Only valid/effective if index_cols is not None. Default is None – setup parameters for all rows.

  • pargp (str) – Parameter group to assign pars to. This is PESTs pargp but is also used to gather correlated parameters set up using multiple add_parameters() calls (e.g. temporal pars) with common geostructs.

  • pp_space (float, int,`str` or pd.DataFrame) – Spatial pilot point information. If float or int, AND spatial_reference is of type VertexGrid, it is the spacing in model length untis between pilot points. If int it is the spacing in rows and cols of where to place pilot points. If pd.DataFrame, then this arg is treated as a prefined set of pilot points and in this case, the dataframe must have “name”, “x”, “y”, and optionally “zone” columns. If str, then an attempt is made to load a dataframe from a csv file (if pp_space ends with “.csv”), shapefile (if pp_space ends with “.shp”) or from a pilot points file. If pp_space is None, an integer spacing of 10 is used. Default is None

  • use_pp_zones (bool) – a flag to use the greater-than-zero values in the zone_array as pilot point zones. If False, zone_array values greater than zero are treated as a single zone. This argument is only used if pp_space is None or int. Default is False.

  • num_eig_kl – TODO - impliment with KL pars

  • spatial_reference (pyemu.helpers.SpatialReference) – If different spatial reference required for pilotpoint setup. If None spatial reference passed to PstFrom() will be used for pilot-points

  • geostruct (pyemu.geostats.GeoStruct()) – For specifying correlation geostruct for pilot-points and par covariance.

  • datetime (str) – optional %Y%m%d string or datetime object for setting up temporally correlated pars. Where datetime is passed correlation axis for pars will be set to timedelta.

  • mfile_fmt (str) – format of model input file - this will be preserved

  • mfile_skip (int or str) – header in model input file to skip when reading and reapply when writing. Can optionally be str in which case mf_skip will be treated as a comment_char.

  • mfile_sep (str) – separator/delimiter in model input file. If None, separator will be interpretted from file name extension. .csv is assumed to be comma separator. Default is None

  • ult_ubound (float) – Ultimate upper bound for model input parameter once all mults are applied - ensure physical model par vals. If not passed, it is set to 1.0e+30

  • ult_lbound (float) – Ultimate lower bound for model input parameter once all mults are applied. If not passed, it is set to 1.0e-30 for log transform and -1.0e+30 for non-log transform

  • rebuild_pst (bool) – (Re)Construct PstFrom.pst object after adding new parameters

  • alt_inst_str (str) – Alternative to default inst string in parameter names. Specify None or "" to exclude the instance information from parameter names. For example, if parameters that apply to more than one input/template file are desired.

  • comment_char (str) – option to skip comment lines in model file. This is not additive with mfile_skip option. Warning: currently comment lines within list-style tabular data will be lost.

  • par_style (str) – either “m”/”mult”/”multiplier”, “a”/”add”/”addend”, or “d”/”direct” where the former setups up a multiplier and addend parameters process against the existing model input array and the former setups a template file to write the model input file directly. Default is “multiplier”.

  • initial_value (float) – the value to set for the parval1 value in the control file Default is 1.0

Returns:

dataframe with info for new parameters

Return type:

pandas.DataFrame

Example:

# setup grid-scale direct parameters for an array of numbers
df = pf.add_parameters("hk.dat",par_type="grid",par_style="direct")
# setup pilot point multiplier parameters for an array of numbers
# with a pilot point being set in every 5th active model cell
df = pf.add_parameters("recharge.dat",par_type="pilotpoint",pp_space=5,
                       zone_array="ibound.dat")
# setup a single multiplier parameter for the 4th column
# of a column format (list/tabular type) file
df = pf.add_parameters("wel_list_1.dat",par_type="constant",
                       index_cols=[0,1,2],use_cols=[3])
_load_listtype_file(filename, index_cols, use_cols, fmt=None, sep=None, skip=None, c_char=None)
_prep_arg_list_lengths(filenames, fmts=None, seps=None, skip_rows=None, index_cols=None, use_cols=None)

Private wrapper function to align filenames, formats, delimiters, reading options and setup columns for passing sequentially to load_listtype :param filenames: names for files ot eventually read :type filenames: str) or (list :param fmts: of column formaters for input file.

If None, free-formatting is assumed

Parameters:

  • seps (str) or (list) – column separator free formatter files. If None, a list of None`s is returned and the delimiter is eventually governed by the file extension (,` for .csv)

  • skip_rows (str) or (list) – Number of rows in file header to not form part of the dataframe

  • index_cols (int) or (list) – Columns in tabular file to use as indicies

  • use_cols (int) or (list) – Columns in tabular file to use as par or obs cols

Returns:

filenames, fmts, seps, skip_rows, index_cols, use_cols for squentially passing to _load_listtype_file()

Return type:

algined lists of

pyemu.utils.pst_from.write_list_tpl(filenames, dfs, name, tpl_filename, index_cols, par_type, use_cols=None, use_rows=None, suffix='', zone_array=None, gpname=None, get_xy=None, ij_in_idx=None, xy_in_idx=None, zero_based=True, input_filename=None, par_style='m', headerlines=None, fill_value=1.0, logger=None)

Write template files for a list style input.

Parameters:

  • filenames (str of container of str) – original input filenames

  • dfs (pandas.DataFrame or container of pandas.DataFrames) – pandas representations of input file.

  • name (str or container of str) – parameter name prefixes. If more that one column to be parameterised, must be a container of strings providing the prefix for the parameters in the different columns.

  • tpl_filename (str) – Path (from current execution directory) for desired template file

  • index_cols (list) – column names to use as indices in tabular input dataframe

  • par_type (str) – ‘constant’,’zone’, or ‘grid’ used in parname generation. If constant, one par is set up for each use_cols. If zone, one par is set up for each zone for each use_cols. If grid, one par is set up for every unique index combination (from index_cols) for each use_cols.

  • use_cols (list) – Columns in tabular input file to paramerterise. If None, pars are set up for all columns apart from index cols.

  • use_rows (list of int or tuple) –

    Setup parameters for only specific rows in list-style model input file. If list of int – assumed to be a row index selction (zero-based). If list of tuple – assumed to be selection based index_cols

    values. e.g. [(3,5,6)] would attempt to set parameters where the model file values for 3 index_cols are 3,5,6. N.B. values in tuple are actual model file entry values.

    If no rows in the model input file match use_rows – parameters

    will be set up for all rows.

    Only valid/effective if index_cols is not None. Default is None – setup parameters for all rows.

  • suffix (str) – Optional par name suffix

  • zone_array (np.ndarray) – Array defining zone divisions. If not None and par_type is grid or zone it is expected that index_cols provide the indicies for querying zone_array. Therefore, array dimension should equal len(index_cols).

  • get_xy (pyemu.PstFrom method) – Can be specified to get real-world xy from index_cols passed (to assist correlation definition)

  • ij_in_idx (list or array) – defining which index_cols contain i,j

  • xy_in_idx (list or array) – defining which index_cols contain x,y

  • zero_based (boolean) – IMPORTANT - pass as False if index_cols are NOT zero-based indicies (e.g. MODFLOW row/cols). If False 1 with be subtracted from index_cols.

  • input_filename (str) – Path to input file (paired with tpl file)

  • par_style (str) – either ‘d’,’a’, or ‘m’

  • headerlines ([str]) – optional header lines in the original model file, used for direct style parameters

Returns:

dataframe with info for the new parameters

Return type:

pandas.DataFrame

Note

This function is called by PstFrom programmatically

pyemu.utils.pst_from._write_direct_df_tpl(in_filename, tpl_filename, df, name, index_cols, typ, use_cols=None, use_rows=None, suffix='', zone_array=None, get_xy=None, ij_in_idx=None, xy_in_idx=None, zero_based=True, gpname=None, headerlines=None, logger=None)

Private method to auto-generate parameter or obs names from tabular model files (input or output) read into pandas dataframes :param tpl_filename: template filename :type tpl_filename: str :param df: DataFrame of list-style input file :type df: pandas.DataFrame :param name: Parameter name prefix :type name: str :param index_cols: columns of dataframes to use as indicies :type index_cols: str or list :param typ: ‘constant’,’zone’, or ‘grid’ used in parname generation.

If constant, one par is set up for each use_cols. If zone, one par is set up for each zone for each use_cols. If grid, one par is set up for every unique index combination (from index_cols) for each use_cols.

Parameters:

  • use_cols (list) – Columns to parameterise. If None, pars are set up for all columns apart from index cols

  • suffix (str) – Optional par name suffix.

  • zone_array (np.ndarray) – Array defining zone divisions. If not None and par_type is grid or zone it is expected that index_cols provide the indicies for querying zone_array. Therefore, array dimension should equal len(index_cols).

  • get_xy (pyemu.PstFrom method) – Can be specified to get real-world xy from index_cols passed (to include in obs/par name)

  • ij_in_idx (list or array) – defining which index_cols contain i,j

  • xy_in_idx (list or array) – defining which index_cols contain x,y

  • zero_based (boolean) – IMPORTANT - pass as False if index_cols are NOT zero-based indicies (e.g. MODFLOW row/cols). If False 1 with be subtracted from index_cols.

Returns:

pandas.DataFrame with paranme and pargroup define for each use_col

Note

This function is called by PstFrom programmatically

pyemu.utils.pst_from._get_use_rows(tpldf, idxcolvals, use_rows, zero_based, fnme, logger=None)

private function to get use_rows index within df based on passed use_rows option, which could be in various forms… :param tpldf: :param idxcolvals: :param use_rows: :param zero_based: :param fname: :param logger:

Returns:

pyemu.utils.pst_from._get_index_strfmt(index_cols)
pyemu.utils.pst_from._get_index_strings(df, fmt, zero_based)
pyemu.utils.pst_from._getxy_from_idx(df, get_xy, xy_in_idx, ij_in_idx)
pyemu.utils.pst_from._build_parnames(df, typ, zone_array, index_cols, use_cols, basename, gpname, suffix, par_style, init_df=None, init_fname=None, fill_value=1.0)
pyemu.utils.pst_from._get_tpl_or_ins_df(dfs, name, index_cols, typ, use_cols=None, suffix='', zone_array=None, get_xy=None, ij_in_idx=None, xy_in_idx=None, zero_based=True, gpname=None, par_fill_value=1.0, par_style='m')

Private method to auto-generate parameter or obs names from tabular model files (input or output) read into pandas dataframes :param dfs: DataFrames (can be list of DataFrames)

to set up parameters or observations

Parameters:

  • name (str) – Parameter name or Observation name prefix

  • index_cols (str or list) – columns of dataframes to use as indicies

  • typ (str) – ‘obs’ to set up observation names or, ‘constant’,’zone’, or ‘grid’ used in parname generation. If constant, one par is set up for each use_cols. If zone, one par is set up for each zone for each use_cols. If grid, one par is set up for every unique index combination (from index_cols) for each use_cols.

  • use_cols (list) – Columns to parameterise. If None, pars are set up for all columns apart from index cols. Not used if typ`==`obs.

  • suffix (str) – Optional par name suffix. Not used if typ`==`obs.

  • zone_array (np.ndarray) – Only used for paremeters (typ != obs). Array defining zone divisions. If not None and par_type is grid or zone it is expected that index_cols provide the indicies for querying zone_array. Therefore, array dimension should equal len(index_cols).

  • get_xy (pyemu.PstFrom method) – Can be specified to get real-world xy from index_cols passed (to include in obs/par name)

  • ij_in_idx (list or array) – defining which index_cols contain i,j

  • xy_in_idx (list or array) – defining which index_cols contain x,y

  • zero_based (boolean) – IMPORTANT - pass as False if index_cols are NOT zero-based indicies (e.g. MODFLOW row/cols). If False 1 with be subtracted from index_cols.=

  • par_fill_value (float) – value to use as parval1,Default is 1.0

Returns:

pandas.DataFrame with index strings for setting up obs names when passing through to pyemu.pst_utils.csv_to_ins_file(df.set_index(‘idx_str’)

else: pandas.DataFrame with paranme and pargroup define for each use_col

Return type:

if typ`==`obs

pyemu.utils.pst_from.write_array_tpl(name, tpl_filename, suffix, par_type, zone_array=None, gpname=None, shape=None, fill_value=1.0, get_xy=None, input_filename=None, par_style='m')

write a template file for a 2D array.

Args:

name (str): the base parameter name tpl_filename (str): the template file to write - include path suffix (str): suffix to append to par names par_type (str): type of parameter zone_array (numpy.ndarray): an array used to skip inactive cells. Values less than 1 are

not parameterized and are assigned a value of fill_value. Default is None.

gpname (str): pargp filed in dataframe shape (tuple): dimensions of array to write fill_value: get_xy: input_filename: par_style (str): either ‘d’,’a’, or ‘m’

Returns:

a dataframe with parameter information

Return type:

df (pandas.DataFrame)

Note

This function is called by PstFrom programmatically

pyemu.utils.pst_from._check_diff(org_arr, input_filename, zval=None)
pyemu.utils.pst_from.get_filepath(folder, filename)

Return a path to a file within a folder, without repeating the folder in the output path, if the input filename (path) already contains the folder.

pyemu.utils.pst_from.get_relative_filepath(folder, filename)

Like get_filepath(), except return path for filename relative to folder.