pyemu.mat
This module contains classes for handling matrices in a linear algebra setting. The primary objects are the Matrix and Cov. These objects overload most numerical operators to autoalign the elements based on row and column names.
Submodules
Package Contents
Classes
Easy linear algebra in the PEST(++) realm |
|
Diagonal and/or dense Covariance matrices |
|
a thin wrapper class to get more intuitive attribute names. Functions |
Functions
|
Concatenate Matrix objects. Tries either axis. |
|
write a PEST-compatible binary file. The data format is |
- class pyemu.mat.Matrix(x=None, row_names=[], col_names=[], isdiagonal=False, autoalign=True)
Bases:
object
Easy linear algebra in the PEST(++) realm
- Parameters:
x (numpy.ndarray) – numeric values
row_names ([str]) – list of row names
col_names (['str']) – list of column names
isdigonal (bool) – flag if the Matrix is diagonal
autoalign (bool) – flag to control the autoalignment of Matrix during linear algebra operations
Example:
data = np.random.random((10,10)) row_names = ["row_{0}".format(i) for i in range(10)] col_names = ["col_{0}".format(j) for j in range(10)] mat = pyemu.Matrix(x=data,row_names=row_names,col_names=col_names) mat.to_binary("mat.jco") # load an existing jacobian matrix jco = pyemu.Jco.from_binary("pest.jco") # form an observation noise covariance matrix from weights obscov = pyemu.Cov.from_observation_data(pst) # form the normal matrix, aligning rows and cols on-the-fly xtqx = jco * obscov.inv * jco.T
Note
this class makes heavy use of property decorators to encapsulate private attributes
- property newx
return a copy of Matrix.x attribute
- Returns:
a copy Matrix.x
- Return type:
numpy.ndarray
- property x
return a reference to Matrix.x
- Returns:
reference to Matrix.x
- Return type:
numpy.ndarray
- property as_2d
get a 2D numeric representation of Matrix.x. If not isdiagonal, simply return reference to Matrix.x, otherwise, constructs and returns a 2D, diagonal ndarray
- Returns:
numpy.ndarray
- Return type:
numpy.ndarray
Exmaple:
# A diagonal cov cov = pyemu.Cov.from_parameter_data x2d = cov.as_2d # a numpy 2d array print(cov.shape,cov.x.shape,x2d.shape)
- property shape
get the implied, 2D shape of Matrix
- Returns:
length of 2 tuple
- Return type:
int
Exmaple:
jco = pyemu.Jco.from_binary("pest.jcb") shape = jco.shape nrow,ncol = shape #unpack to ints
- property ncol
length of second dimension
- Returns:
number of columns
- Return type:
int
- property nrow
length of first dimension
- Returns:
number of rows
- Return type:
int
- property T
wrapper function for Matrix.transpose() method
- Returns:
transpose of Matrix
- Return type:
Matrix
Note
returns a copy of self
A syntatic-sugar overload of Matrix.transpose()
Example:
jcb = pyemu.Jco.from_binary("pest.jcb") jcbt = jcb.T
- property transpose
transpose operation of self
- Returns:
transpose of Matrix
- Return type:
Matrix
Example:
jcb = pyemu.Jco.from_binary("pest.jcb") jcbt = jcb.T
- property inv
inversion operation of Matrix
- Returns:
inverse of Matrix
- Return type:
Matrix
Note
uses numpy.linalg.inv for the inversion
Example:
mat = pyemu.Matrix.from_binary("my.jco") mat_inv = mat.inv mat_inv.to_binary("my_inv.jco")
- property sqrt
element-wise square root operation
- Returns:
element-wise square root of Matrix
- Return type:
Matrix
Note
uses numpy.sqrt
Example:
cov = pyemu.Cov.from_uncfile("my.unc") sqcov = cov.sqrt sqcov.to_uncfile("sqrt_cov.unc")
- property full_s
Get the full singular value matrix
- Returns:
full singular value matrix. Shape is (max(Matrix.shape),max(Matrix.shape)) with zeros along the diagonal from min(Matrix.shape) to max(Matrix.shape)
- Return type:
Matrix
Example:
jco = pyemu.Jco.from_binary("pest.jcb") jco.full_s.to_ascii("full_sing_val_matrix.mat")
- property s
the singular value (diagonal) Matrix
- Returns:
singular value matrix. shape is (min(Matrix.shape),min(Matrix.shape))
- Return type:
Matrix
Example:
# plot the singular spectrum of the jacobian import matplotlib.pyplot as plt jco = pyemu.Jco.from_binary("pest.jcb") plt.plot(jco.s.x) plt.show()
- property u
the left singular vector Matrix
- Returns:
left singular vectors. Shape is (Matrix.shape[0], Matrix.shape[0])
- Return type:
Matrix
Example:
jco = pyemu.Jco.from_binary("pest.jcb") jco.u.to_binary("u.jcb")
- property v
the right singular vector Matrix
- Returns:
right singular vectors. Shape is (Matrix.shape[1], Matrix.shape[1])
- Return type:
Matrix
Example:
jco = pyemu.Jco.from_binary("pest.jcb") jco.v.to_binary("v.jcb")
- property zero2d
get an 2D instance of self with all zeros
- Returns:
Matrix of zeros
- Return type:
Matrix
Example:
jco = pyemu.Jco.from_binary("pest.jcb") zero_jco = jco.zero2d
- integer
- double
- char
- binary_header_dt
- binary_rec_dt
- coo_rec_dt
- par_length = 12
- obs_length = 20
- new_par_length = 200
- new_obs_length = 200
- reset_x(x, copy=True)
reset self.__x private attribute
- Parameters:
x (numpy.ndarray) – the new numeric data
copy (bool) – flag to make a copy of ‘x’. Default is True
- Returns:
None
Note
operates in place
- __str__()
overload of object.__str__()
- Returns:
string representation
- Return type:
str
- __getitem__(item)
a very crude overload of object.__getitem__().
- Parameters:
item (object) – something that can be used as an index
- Returns:
an object that is a sub-matrix of Matrix
- Return type:
Matrix
- __pow__(power)
overload of numpy.ndarray.__pow__() operator
- Parameters:
power (float) – interpreted as follows: -1 = inverse of self, -0.5 = sqrt of inverse of self, 0.5 = sqrt of self. All other positive ints = elementwise self raised to power
- Returns:
a new Matrix object raised to the power power
- Return type:
Matrix
Example:
cov = pyemu.Cov.from_uncfile("my.unc") sqcov = cov**2
- __sub__(other)
- numpy.ndarray.__sub__() overload. Tries to speedup by
checking for scalars of diagonal matrices on either side of operator
- Parameters:
other – (int,`float`,`numpy.ndarray`,`Matrix`): the thing to subtract
- Returns:
the result of subtraction
- Return type:
Matrix
Note
if Matrix and other (if applicable) have autoalign set to True, both Matrix and other are aligned based on row and column names. If names are not common between the two, this may result in a smaller returned Matrix. If no names are shared, an exception is raised
Example:
jco1 = pyemu.Jco.from_binary("pest.1.jcb") jco2 = pyemu.Jco.from_binary("pest.2.jcb") diff = jco1 - jco2
- __add__(other)
- Overload of numpy.ndarray.__add__() - elementwise addition. Tries to speedup by checking for
scalars of diagonal matrices on either side of operator
- Parameters:
other – (int,`float`,`numpy.ndarray`,`Matrix`): the thing to add
- Returns:
the result of addition
- Return type:
Matrix
Note
if Matrix and other (if applicable) have autoalign set to True, both Matrix and other are aligned based on row and column names. If names are not common between the two, this may result in a smaller returned Matrix. If no names are shared, an exception is raised. The addition of a scalar does not expand non-zero elements.
Example:
m1 = pyemu.Cov.from_parameter_data(pst) m1_plus_on1 = m1 + 1.0 #add 1.0 to all non-zero elements m2 = m1.copy() m2_plus_m1 = m1 + m2
- hadamard_product(other)
Overload of numpy.ndarray.__mult__(): element-wise multiplication. Tries to speedup by checking for scalars of diagonal matrices on either side of operator
- Parameters:
other – (int,`float`,`numpy.ndarray`,`Matrix`): the thing to multiply
- Returns:
the result of multiplication
- Return type:
Matrix
Note
if Matrix and other (if applicable) have autoalign set to True, both Matrix and other are aligned based on row and column names. If names are not common between the two, this may result in a smaller returned Matrix. If not common elements are shared, an excetion is raised
Example:
cov = pyemu.Cov.from_parameter_data(pst) cov2 = cov * 10 cov3 = cov * cov2
- __mul__(other)
Dot product multiplication overload. Tries to speedup by checking for scalars or diagonal matrices on either side of operator
- Parameters:
other – (int,`float`,`numpy.ndarray`,`Matrix`): the thing to dot product
- Returns:
the result of dot product
- Return type:
Matrix
Note
if Matrix and other (if applicable) have autoalign set to True, both Matrix and other are aligned based on row and column names. If names are not common between the two, this may result in a smaller returned Matrix. If not common elements are found, an exception is raised
Example:
jco = pyemu.Jco.from_binary("pest.jcb") cov = pyemu.Cov.from_parmaeter_data(pst) # get the forecast prior covariance matrix forecast_cov = jco.get(pst.forecast_names).T * cov * jco.get(pst.forecast_names)
- __rmul__(other)
Reverse order Dot product multiplication overload.
- Parameters:
other – (int,`float`,`numpy.ndarray`,`Matrix`): the thing to dot product
- Returns:
the result of dot product
- Return type:
Matrix
Note
if Matrix and other (if applicable) have autoalign set to True, both Matrix and other are aligned based on row and column names. If names are not common between the two, this may result in a smaller returned Matrix. If not common elements are found, an exception is raised
Example:
# multiply by a scalar jco = pyemu.Jco.from_binary("pest.jcb") jco_times_10 = 10 * jco
- __set_svd()
private method to set SVD components.
Note: this should not be called directly
- mult_isaligned(other)
check if matrices are aligned for dot product multiplication
- Parameters:
other (Matrix) – the other matrix to check for alignment with
- Returns:
True if aligned, False if not aligned
- Return type:
bool
- element_isaligned(other)
check if matrices are aligned for element-wise operations
- Parameters:
other (Matrix) – the other matrix to check for alignment with
- Returns:
True if aligned, False if not aligned
- Return type:
bool
- to_2d()
- get a 2D Matrix representation of Matrix. If not Matrix.isdiagonal, simply
return a copy of Matrix, otherwise, constructs and returns a new Matrix instance that is stored as diagonal
- Returns:
non-diagonal form of Matrix
- Return type:
Martrix
Exmaple:
# A diagonal cov cov = pyemu.Cov.from_parameter_data cov2d = cov.as_2d # a numpy 2d array print(cov.shape,cov.x.shape,cov2d.shape,cov2d.x.shape)
- static get_maxsing_from_s(s, eigthresh=1e-05)
static method to work out the maxsing for a given singular spectrum
- Parameters:
s (numpy.ndarray) – 1-D array of singular values. This array should come from calling either numpy.linalg.svd or from the pyemu.Matrix.s.x attribute
eigthresh (float) – the ratio of smallest to largest singular value to retain. Since it is assumed that s is sorted from largest to smallest, once a singular value is reached that yields a ratio with the first (largest) singular value, the index of this singular is returned.
- Returns:
the index of the singular value whos ratio with the first singular value is less than or equal to eigthresh
- Return type:
int
Example:
jco = pyemu.Jco.from_binary("pest.jco") max_sing = pyemu.Matrix.get_maxsing_from_s(jco.s,eigthresh=pst.svd_data.eigthresh)
- get_maxsing(eigthresh=1e-05)
Get the number of singular components with a singular value ratio greater than or equal to eigthresh
- Args:
- eigthresh (float): the ratio of smallest to largest
singular value to retain. Since it is assumed that s is sorted from largest to smallest, once a singular value is reached that yields a ratio with the first (largest) singular value, the index of this singular is returned.
- Returns:
the index of the singular value whos ratio with the first singular value is less than or equal to eigthresh
- Return type:
int
Note
this method calls the static method Matrix.get_maxsing_from_s() with Matrix.s.x
Example:
jco = pyemu.Jco.from_binary("pest.jco") max_sing = jco.get_maxsing(eigthresh=pst.svd_data.eigthresh)
- pseudo_inv_components(maxsing=None, eigthresh=1e-05, truncate=True)
Get the (optionally) truncated SVD components
- Parameters:
maxsing (int, optional) – the number of singular components to use. If None, maxsing is calculated using Matrix.get_maxsing() and eigthresh
eigthresh – (float, optional): the ratio of largest to smallest singular components to use for truncation. Ignored if maxsing is not None. Default is 1.0e-5
truncate (bool) – flag to truncate components. If False, U, s, and V will be zeroed out at locations greater than maxsing instead of truncated. Default is True
- Returns:
tuple containing
Matrix: (optionally truncated) left singular vectors
Matrix: (optionally truncated) singular value matrix
Matrix: (optionally truncated) right singular vectors
Example:
mat = pyemu.Matrix.from_binary("my.jco") u1,s1,v1 = mat.pseudo_inv_components(maxsing=10) resolution_matrix = v1 * v1.T resolution_matrix.to_ascii("resol.mat")
- pseudo_inv(maxsing=None, eigthresh=1e-05)
The pseudo inverse of self. Formed using truncated singular value decomposition and Matrix.pseudo_inv_components
- Parameters:
maxsing (int, optional) – the number of singular components to use. If None, maxsing is calculated using Matrix.get_maxsing() and eigthresh
eigthresh – (float, optional): the ratio of largest to smallest singular components to use for truncation. Ignored if maxsing is not None. Default is 1.0e-5
- Returns:
the truncated-SVD pseudo inverse of Matrix (V_1 * s_1^-1 * U^T)
- Return type:
Matrix
Example:
jco = pyemu.Jco.from_binary("pest.jcb") jco_psinv = jco.pseudo_inv(pst.svd_data.maxsing,pst.svd_data.eigthresh) jco_psinv.to_binary("pseudo_inv.jcb")
- static find_rowcol_indices(names, row_names, col_names, axis=None)
fast(er) look of row and colum names indices
- Parameters:
names ([str]) – list of names to look for in row_names and/or col_names names
row_names ([str]) – list of row names
col_names ([str]) – list of column names
axis (int, optional) – axis to search along. If None, search both. Default is None
- Returns:
array of (integer) index locations. If axis is None, a 2 numpy.ndarrays of both row and column name indices is returned
- Return type:
numpy.ndarray
- indices(names, axis=None)
- get the row and col indices of names. If axis is None, two ndarrays
are returned, corresponding the indices of names for each axis
- Parameters:
names ([str]) – list of names to look for in row_names and/or col_names names
row_names ([str]) – list of row names
col_names ([str]) – list of column names
axis (int, optional) – axis to search along. If None, search both. Default is None
- Returns:
array of (integer) index locations. If axis is None, a 2 numpy.ndarrays of both row and column name indices is returned
- Return type:
numpy.ndarray
Note
thin wrapper around Matrix.find_rowcol_indices static method
- align(names, axis=None)
reorder Matrix by names in place. If axis is None, reorder both indices
- Parameters:
names (['str']) – names in Matrix.row_names and or Matrix.col_names
axis (int, optional) – the axis to reorder. if None, reorder both axes
Note
Works in place. Is called programmatically during linear algebra operations
Example:
# load a jco that has more obs (rows) than non-zero weighted obs # in the control file jco = pyemu.Jco.from_binary("pest.jcb") # get an obs noise cov matrix obscov = pyemu.Cov.from_observation_data(pst) jco.align(obscov.row_names,axis=0)
- get(row_names=None, col_names=None, drop=False)
get a new Matrix instance ordered on row_names or col_names
- Parameters:
row_names (['str'], optional) – row_names for new Matrix. If None, all row_names are used.
col_names (['str'], optional) – col_names for new Matrix. If None, all col_names are used.
drop (bool) – flag to remove row_names and/or col_names from this Matrix
- Returns:
a new Matrix
- Return type:
Matrix
- Example::
# load a jco that has more obs (rows) than non-zero weighted obs # in the control file jco = pyemu.Jco.from_binary(“pest.jcb”) # get an obs noise cov matrix obscov = pyemu.Cov.from_observation_data(pst) nnz_jco = jco.get(row_names = obscov.row_names)
- copy()
get a copy of Matrix
- Returns:
copy of this Matrix
- Return type:
Matrix
- drop(names, axis)
drop elements from Matrix in place
- Parameters:
names (['str']) – list of names to drop
axis (int) – the axis to drop from. must be in [0,1]
- extract(row_names=None, col_names=None)
wrapper method that Matrix.gets() then Matrix.drops() elements. one of row_names or col_names must be not None.
- Parameters:
row_names (['str'], optional) – row_names to extract. If None, all row_names are retained.
col_names (['str'], optional) – col_names to extract. If None, all col_names are retained.
- Returns:
the extract sub-matrix defined by row_names and/or col_names
- Return type:
Matrix
Example:
cov = pyemu.Cov.from_parameter_data(pst) hk_cov = cov.extract(row_names=["hk1","hk2","hk3"])
- get_diagonal_vector(col_name='diag')
Get a new Matrix instance that is the diagonal of self. The shape of the new matrix is (self.shape[0],1). Self must be square
- Parameters:
col_name (str) – the name of the single column in the new Matrix
- Returns:
vector-shaped Matrix instance of the diagonal of this Matrix
- Return type:
Matrix
Example:
cov = pyemu.Cov.from_unc_file("my.unc") cov_diag = cov.get_diagonal_vector() print(cov_diag.col_names)
- to_coo(filename, droptol=None, chunk=None)
write an extended PEST-format binary file. The data format is [int,int,float] for i,j,value. It is autodetected during the read with Matrix.from_binary().
- Parameters:
filename (str) – filename to save binary file
droptol (float) – absolute value tolerance to make values smaller droptol than zero. Default is None (no dropping)
chunk (int) – number of elements to write in a single pass. Default is None, which writes the entire numeric part of the Matrix at once. This is faster but requires more memory.
Note
This method is needed when the number of dimensions times 2 is larger than the max value for a 32-bit integer. happens! This method is used by pyemu.Ensemble.to_binary()
- to_dense(filename, close=True)
experimental new dense matrix storage format to support faster I/O with ensembles
- Parameters:
filename (str) – the filename to save to
close (bool) – flag to close the filehandle after saving
- Returns:
the file handle. Only returned if close is False
- Return type:
f (file)
Note
calls Matrix.write_dense()
- static write_dense(filename, row_names, col_names, data, close=False)
experimental new dense matrix storage format to support faster I/O with ensembles
- Parameters:
filename (str or file) – the file to write to
row_names ([str]) – row names of the matrix
col_names ([str]) – col names of the matrix
data (np.ndarray) – matrix elements
close (bool) – flag to close the file after writing
- Returns:
the file handle. Only returned if close is False
- Return type:
f (file)
- to_binary(filename, droptol=None, chunk=None)
write a PEST-compatible binary file. The format is the same as the format used to storage a PEST Jacobian matrix
- Parameters:
filename (str) – filename to save binary file
droptol (float) – absolute value tolerance to make values smaller droptol than zero. Default is None (no dropping)
chunk (int) – number of elements to write in a single pass. Default is None, which writes the entire numeric part of the Matrix at once. This is faster but requires more memory.
- static read_dense(filename, forgive=False, close=True)
read a dense-format binary file.
- Parameters:
filename (str) – the filename or the open filehandle
forgive (bool) – flag to forgive incomplete records. If True and an incomplete record is encountered, only the previously read records are returned. If False, an exception is raised for an incomplete record
close (bool) – flag to close the filehandle. Default is True
- Returns:
tuple containing
numpy.ndarray: the numeric values in the file
[‘str’]: list of row names
[`str`]: list of col_names
- classmethod from_binary(filename, forgive=False)
class method load from PEST-compatible binary file into a Matrix instance
- Parameters:
filename (str) – filename to read
forgive (bool) – flag to forgive incomplete data records. Only applicable to dense binary format. Default is False
- Returns:
Matrix loaded from binary file
- Return type:
Matrix
Example:
mat = pyemu.Matrix.from_binary("my.jco") cov = pyemu.Cov.from_binary("large_cov.jcb")
- static read_binary_header(filename)
read the first elements of a PEST(++)-style binary file to get format and dimensioning information.
- Parameters:
filename (str) – the filename of the binary file
- Returns:
tuple containing
int: the itemp1 value
int: the itemp2 value
int: the icount value
- static read_binary(filename, forgive=False)
static method to read PEST-format binary files
- Parameters:
filename (str) – filename to read
forgive (bool) – flag to forgive incomplete data records. Only applicable to dense binary format. Default is False
- Returns:
tuple containing
numpy.ndarray: the numeric values in the file
[‘str’]: list of row names
[`str`]: list of col_names
- static from_fortranfile(filename)
- a binary load method to accommodate one of the many
bizarre fortran binary writing formats
- Parameters:
filename (str) – name of the binary matrix file
- Returns:
tuple containing
numpy.ndarray: the numeric values in the file
[‘str’]: list of row names
[`str`]: list of col_names
- to_ascii(filename, icode=2)
write a PEST-compatible ASCII Matrix/vector file
- Parameters:
filename (str) – filename to write to
icode (int, optional) – PEST-style info code for matrix style. Default is 2.
Note
if icode == -1, a 1-d vector is written that represents a diagonal matrix. An exception is raised if icode == -1 and isdiagonal is False
- classmethod from_ascii(filename)
load a PEST-compatible ASCII matrix/vector file into a Matrix instance
- Parameters:
filename (str) – name of the file to read
- Returns:
Matrix loaded from ASCII file
- Return type:
Matrix
Example:
mat = pyemu.Matrix.from_ascii("my.mat") cov = pyemi.Cov.from_ascii("my.cov")
- static read_ascii(filename)
read a PEST-compatible ASCII matrix/vector file
- Parameters:
filename (str) – file to read from
- Returns:
tuple containing
numpy.ndarray: numeric values
[‘str’]: list of row names
[`str`]: list of column names
bool: diagonal flag
- df()
wrapper of Matrix.to_dataframe()
- classmethod from_dataframe(df)
- class method to create a new Matrix instance from a
pandas.DataFrame
- Parameters:
df (pandas.DataFrame) – dataframe
- Returns:
Matrix instance derived from df.
- Return type:
Matrix
Example:
df = pd.read_csv("my.csv") mat = pyemu.Matrix.from_dataframe(df)
- classmethod from_names(row_names, col_names, isdiagonal=False, autoalign=True, random=False)
class method to create a new Matrix instance from row names and column names, filled with trash
- Parameters:
row_names (['str']) – row names for the new Matrix
col_names (['str']) – col_names for the new matrix
isdiagonal (bool, optional) – flag for diagonal matrix. Default is False
autoalign (bool, optional) – flag for autoaligning new matrix during linear algebra calcs. Default is True
random (bool) – flag for contents of the trash matrix. If True, fill with random numbers, if False, fill with zeros Default is False
- Returns:
the new Matrix instance
- Return type:
Matrix
Example:
row_names = ["row_1","row_2"] col_names = ["col_1,"col_2"] m = pyemu.Matrix.from_names(row_names,col_names)
- to_dataframe()
return a pandas.DataFrame representation of Matrix
- Returns:
a dataframe derived from Matrix
- Return type:
pandas.DataFrame
Note
if self.isdiagonal is True, the full matrix is used to fill the dataframe - lots of zeros.
- extend(other)
extend Matrix with the elements of other, returning a new matrix.
Args: other (Matrix): the Matrix to extend self by
- Returns:
new, extended Matrix
- Return type:
Matrix
Note
No row or column names can be shared between self and other
Example:
jco1 = pyemu.Jco.from_binary("pest_history.jco") jco2 = pyemu.Jco.from_binary("pest_forecast.jco") jco_ext = jco1.extend(jco2)
- class pyemu.mat.Cov(x=None, names=[], row_names=[], col_names=[], isdiagonal=False, autoalign=True)
Bases:
Matrix
Diagonal and/or dense Covariance matrices
- Parameters:
x (numpy.ndarray) – numeric values
names ([str]) – list of row and column names
isdigonal (bool) – flag if the Matrix is diagonal
autoalign (bool) – flag to control the autoalignment of Matrix during linear algebra operations
Example:
data = np.random.random((10,10)) names = ["par_{0}".format(i) for i in range(10)] mat = pyemu.Cov(x=data,names=names) mat.to_binary("mat.jco")
Note
row_names and col_names args are supported in the contructor so support inheritance. However, users should only pass names
- property identity
get an identity Cov of the same shape
- Returns:
new Cov instance with identity matrix
- Return type:
Cov
Note
the returned identity matrix has the same row-col names as self
- property zero
get a diagonal instance of Cov with all zeros on the diagonal
- Returns:
new Cov instance with zeros
- Return type:
Cov
- property names
wrapper for getting row_names. row_names == col_names for Cov
- Returns:
list of names
- Return type:
[str]
- condition_on(conditioning_elements)
get a new Covariance object that is conditional on knowing some elements. uses Schur’s complement for conditional Covariance propagation
- Parameters:
conditioning_elements (['str']) – list of names of elements to condition on
- Returns:
new conditional Cov that assumes conditioning_elements have become known
- Return type:
Cov
Example:
prior_cov = pyemu.Cov.from_parameter_data(pst) now_known_pars = pst.adj_par_names[:5] post_cov = prior_cov.condition_on(now_known_pars)
- replace(other)
replace elements in the covariance matrix with elements from other. if other is not diagonal, then this Cov becomes non diagonal
- Parameters:
Cov – the Cov to replace elements in this Cov with
Note
operates in place. Other must have the same row-col names as self
- to_uncfile(unc_file, covmat_file='cov.mat', var_mult=1.0, include_path=False)
write a PEST-compatible uncertainty file
- Parameters:
unc_file (str) – filename of the uncertainty file
covmat_file (str) – covariance matrix filename. Default is “Cov.mat”. If None, and Cov.isdiaonal, then a standard deviation form of the uncertainty file is written. Exception raised if covmat_file is None and not Cov.isdiagonal
var_mult (float) – variance multiplier for the covmat_file entry
include_path (bool) – flag to include the path of unc_file in the name of covmat_file. Default is False - not sure why you would ever make this True…
Example:
cov = pyemu.Cov.from_parameter_data(pst) cov.to_uncfile("my.unc")
- classmethod from_obsweights(pst_file)
instantiates a Cov instance from observation weights in a PEST control file.
- Parameters:
pst_file (str) – pest control file name
- Returns:
a diagonal observation noise covariance matrix derived from the weights in the pest control file. Zero-weighted observations are included with a weight of 1.0e-30
- Return type:
Cov
Note
Calls Cov.from_observation_data()
Example:
obscov = pyemu.Cov.from_obsweights("my.pst")
- classmethod from_observation_data(pst)
instantiates a Cov from pyemu.Pst.observation_data
- Parameters:
pst (pyemu.Pst) – control file instance
- Returns:
a diagonal observation noise covariance matrix derived from the weights in the pest control file. Zero-weighted observations are included with a weight of 1.0e-30
- Return type:
Cov
Example:
obscov = pyemu.Cov.from_observation_data(pst)
- classmethod from_parbounds(pst_file, sigma_range=4.0, scale_offset=True)
Instantiates a Cov from a pest control file parameter data section using parameter bounds as a proxy for uncertainty.
- Parameters:
pst_file (str) – pest control file name
sigma_range (float) – defines range of upper bound - lower bound in terms of standard deviation (sigma). For example, if sigma_range = 4, the bounds represent 4 * sigma. Default is 4.0, representing approximately 95% confidence of implied normal distribution
scale_offset (bool) – flag to apply scale and offset to parameter upper and lower bounds before calculating varaince. In some cases, not applying scale and offset can result in undefined (log) variance. Default is True.
- Returns:
diagonal parameter Cov matrix created from parameter bounds
- Return type:
Cov
Note
Calls Cov.from_parameter_data()
- classmethod from_parameter_data(pst, sigma_range=4.0, scale_offset=True, subset=None)
Instantiates a Cov from a pest control file parameter data section using parameter bounds as a proxy for uncertainty.
- Parameters:
pst_file (str) – pest control file name
sigma_range (float) – defines range of upper bound - lower bound in terms of standard deviation (sigma). For example, if sigma_range = 4, the bounds represent 4 * sigma. Default is 4.0, representing approximately 95% confidence of implied normal distribution
scale_offset (bool) – flag to apply scale and offset to parameter upper and lower bounds before calculating varaince. In some cases, not applying scale and offset can result in undefined (log) variance. Default is True.
subset (list-like, optional) – Subset of parameters to draw
- Returns:
diagonal parameter Cov matrix created from parameter bounds
- Return type:
Cov
Note
Calls Cov.from_parameter_data()
- classmethod from_uncfile(filename, pst=None)
instaniates a Cov from a PEST-compatible uncertainty file
- Parameters:
filename (str) – uncertainty file name
pst ('pyemu.Pst`) – a control file instance. this is needed if “first_parameter” and “last_parameter” keywords. Default is None
- Returns:
Cov instance from uncertainty file
- Return type:
Cov
Example:
cov = pyemu.Cov.from_uncfile("my.unc")
- static _get_uncfile_dimensions(filename)
quickly read an uncertainty file to find the dimensions
- classmethod identity_like(other)
Get an identity matrix Cov instance like other Cov
- Parameters:
other (Matrix) – other matrix - must be square
- Returns:
new identity matrix Cov with shape of other
- Return type:
Cov
Note
the returned identity cov matrix is treated as non-diagonal
- to_pearson()
Convert Cov instance to Pearson correlation coefficient matrix
- Returns:
A Matrix of correlation coefs. Return type is Matrix on purpose so that it is clear the returned instance is not a Cov
- Return type:
Matrix
Example:
# plot the posterior parameter correlation matrix import matplotlib.pyplot as plt cov = pyemu.Cov.from_ascii("pest.post.cov") cc = cov.to_pearson() cc.x[cc.x==1.0] = np.NaN plt.imshow(cc)
- class pyemu.mat.Jco(x=None, row_names=[], col_names=[], isdiagonal=False, autoalign=True)
Bases:
Matrix
a thin wrapper class to get more intuitive attribute names. Functions exactly like Matrix
- property par_names
thin wrapper around Matrix.col_names
- Returns:
a list of parameter names
- Return type:
[str]
- property obs_names
thin wrapper around Matrix.row_names
- Returns:
a list of observation names
- Return type:
[‘str’]
- property npar
number of parameters in the Jco
- Returns:
number of parameters (columns)
- Return type:
int
- property nobs
number of observations in the Jco
- Returns:
number of observations (rows)
- Return type:
int
- __init(**kwargs)
Jco constuctor takes the same arguments as Matrix.
- Parameters:
**kwargs (dict) – constructor arguments for Matrix
Example
jco = pyemu.Jco.from_binary(“my.jco”)
- classmethod from_pst(pst, random=False)
construct a new empty Jco from a control file optionally filled with trash
- Parameters:
pst (pyemu.Pst) – a pest control file instance. If type is ‘str’, pst is loaded from filename
random (bool) – flag for contents of the trash matrix. If True, fill with random numbers, if False, fill with zeros Default is False
- Returns:
the new Jco instance
- Return type:
Jco
- pyemu.mat.concat(mats)
Concatenate Matrix objects. Tries either axis.
- Parameters:
mats ([Matrix]) – list of Matrix objects
- Returns:
a concatenated Matrix instance
- Return type:
pyemu.Matrix
- pyemu.mat.save_coo(x, row_names, col_names, filename, chunk=None)
write a PEST-compatible binary file. The data format is [int,int,float] for i,j,value. It is autodetected during the read with Matrix.from_binary().
- Parameters:
x (numpy.sparse) – coo sparse matrix
row_names ([str]) – list of row names
col_names (['str]) – list of col_names
filename (str) – filename
droptol (float) – absolute value tolerance to make values smaller than droptol zero. Default is None (no dropping)
chunk (int) – number of elements to write in a single pass. Default is None