pyemu.mat package

Submodules

pyemu.mat.mat_handler module

class pyemu.mat.mat_handler.Cov(x=None, names=[], row_names=[], col_names=[], isdiagonal=False, autoalign=True)

Bases: Matrix

Diagonal and/or dense Covariance matrices

Parameters:

x (numpy.ndarray) – numeric values
names ([str]) – list of row and column names
isdigonal (bool) – flag if the Matrix is diagonal
autoalign (bool) – flag to control the autoalignment of Matrix during linear algebra operations

Example:

data = np.random.random((10,10))
names = ["par_{0}".format(i) for i in range(10)]
mat = pyemu.Cov(x=data,names=names)
mat.to_binary("mat.jco")

Note

row_names and col_names args are supported in the contructor so support inheritance. However, users should only pass names

condition_on(conditioning_elements)

get a new Covariance object that is conditional on knowing some elements. uses Schur’s complement for conditional Covariance propagation

Parameters:: conditioning_elements (['str']) – list of names of elements to condition on
Returns:: new conditional Cov that assumes conditioning_elements have become known
Return type:: Cov

Example:

prior_cov = pyemu.Cov.from_parameter_data(pst)
now_known_pars = pst.adj_par_names[:5]
post_cov = prior_cov.condition_on(now_known_pars)

classmethod from_observation_data(pst)

instantiates a Cov from pyemu.Pst.observation_data

Parameters:: pst (pyemu.Pst) – control file instance
Returns:: a diagonal observation noise covariance matrix derived from the weights in the pest control file. Zero-weighted observations are included with a weight of 1.0e-30
Return type:: Cov

Example:

obscov = pyemu.Cov.from_observation_data(pst)

classmethod from_obsweights(pst_file)

instantiates a Cov instance from observation weights in a PEST control file.

Parameters:: pst_file (str) – pest control file name
Returns:: a diagonal observation noise covariance matrix derived from the weights in the pest control file. Zero-weighted observations are included with a weight of 1.0e-30
Return type:: Cov

Note

Calls Cov.from_observation_data()

Example:

obscov = pyemu.Cov.from_obsweights("my.pst")

classmethod from_parameter_data(pst, sigma_range=4.0, scale_offset=True, subset=None)

Instantiates a Cov from a pest control file parameter data section using parameter bounds as a proxy for uncertainty.

Parameters:

pst_file (str) – pest control file name
sigma_range (float) – defines range of upper bound - lower bound in terms of standard deviation (sigma). For example, if sigma_range = 4, the bounds represent 4 * sigma. Default is 4.0, representing approximately 95% confidence of implied normal distribution
scale_offset (bool) – flag to apply scale and offset to parameter upper and lower bounds before calculating varaince. In some cases, not applying scale and offset can result in undefined (log) variance. Default is True.
subset (list-like, optional) – Subset of parameters to draw

Returns:

diagonal parameter Cov matrix created from parameter bounds

Return type:

Cov

Note

Calls Cov.from_parameter_data()

classmethod from_parbounds(pst_file, sigma_range=4.0, scale_offset=True)

Instantiates a Cov from a pest control file parameter data section using parameter bounds as a proxy for uncertainty.

Parameters:

pst_file (str) – pest control file name
sigma_range (float) – defines range of upper bound - lower bound in terms of standard deviation (sigma). For example, if sigma_range = 4, the bounds represent 4 * sigma. Default is 4.0, representing approximately 95% confidence of implied normal distribution
scale_offset (bool) – flag to apply scale and offset to parameter upper and lower bounds before calculating varaince. In some cases, not applying scale and offset can result in undefined (log) variance. Default is True.

Returns:

diagonal parameter Cov matrix created from parameter bounds

Return type:

Cov

Note

Calls Cov.from_parameter_data()

classmethod from_uncfile(filename, pst=None)

instaniates a Cov from a PEST-compatible uncertainty file

Parameters:

filename (str) – uncertainty file name
pst ('pyemu.Pst`) – a control file instance. this is needed if “first_parameter” and “last_parameter” keywords. Default is None

Returns:

Cov instance from uncertainty file

Return type:

Cov

Example:

cov = pyemu.Cov.from_uncfile("my.unc")

property identity

get an identity Cov of the same shape

Returns:: new Cov instance with identity matrix
Return type:: Cov

Note

the returned identity matrix has the same row-col names as self

classmethod identity_like(other)

Get an identity matrix Cov instance like other Cov

Parameters:: other (Matrix) – other matrix - must be square
Returns:: new identity matrix Cov with shape of other
Return type:: Cov

Note

the returned identity cov matrix is treated as non-diagonal

property names

wrapper for getting row_names. row_names == col_names for Cov

Returns:: list of names
Return type:: [str]

replace(other)

replace elements in the covariance matrix with elements from other. if other is not diagonal, then this Cov becomes non diagonal

Parameters:: Cov – the Cov to replace elements in this Cov with

Note

operates in place. Other must have the same row-col names as self

to_pearson()

Convert Cov instance to Pearson correlation coefficient matrix

Returns:: A Matrix of correlation coefs. Return type is Matrix on purpose so that it is clear the returned instance is not a Cov
Return type:: Matrix

Example:

# plot the posterior parameter correlation matrix
import matplotlib.pyplot as plt
cov = pyemu.Cov.from_ascii("pest.post.cov")
cc = cov.to_pearson()
cc.x[cc.x==1.0] = np.NaN
plt.imshow(cc)

to_uncfile(unc_file, covmat_file='cov.mat', var_mult=1.0, include_path=False)

write a PEST-compatible uncertainty file

Parameters:

unc_file (str) – filename of the uncertainty file
covmat_file (str) – covariance matrix filename. Default is “Cov.mat”. If None, and Cov.isdiaonal, then a standard deviation form of the uncertainty file is written. Exception raised if covmat_file is None and not Cov.isdiagonal
var_mult (float) – variance multiplier for the covmat_file entry
include_path (bool) – flag to include the path of unc_file in the name of covmat_file. Default is False - not sure why you would ever make this True…

Example:

cov = pyemu.Cov.from_parameter_data(pst)
cov.to_uncfile("my.unc")

property zero

get a diagonal instance of Cov with all zeros on the diagonal

Returns:: new Cov instance with zeros
Return type:: Cov

class pyemu.mat.mat_handler.Jco(x=None, row_names=[], col_names=[], isdiagonal=False, autoalign=True)

Bases: Matrix

a thin wrapper class to get more intuitive attribute names. Functions exactly like Matrix

classmethod from_pst(pst, random=False)

construct a new empty Jco from a control file optionally filled with trash

Parameters:

pst (pyemu.Pst) – a pest control file instance. If type is ‘str’, pst is loaded from filename
random (bool) – flag for contents of the trash matrix. If True, fill with random numbers, if False, fill with zeros Default is False

Returns:

the new Jco instance

Return type:

Jco

property nobs

number of observations in the Jco

Returns:: number of observations (rows)
Return type:: int

property npar

number of parameters in the Jco

Returns:: number of parameters (columns)
Return type:: int

property obs_names

thin wrapper around Matrix.row_names

Returns:: a list of observation names
Return type:: [‘str’]

property par_names

thin wrapper around Matrix.col_names

Returns:: a list of parameter names
Return type:: [str]

class pyemu.mat.mat_handler.Matrix(x=None, row_names=[], col_names=[], isdiagonal=False, autoalign=True)

Bases: object

Easy linear algebra in the PEST(++) realm

Parameters:

x (numpy.ndarray) – numeric values
row_names ([str]) – list of row names
col_names (['str']) – list of column names
isdigonal (bool) – flag if the Matrix is diagonal
autoalign (bool) – flag to control the autoalignment of Matrix during linear algebra operations

Example:

data = np.random.random((10,10))
row_names = ["row_{0}".format(i) for i in range(10)]
col_names = ["col_{0}".format(j) for j in range(10)]
mat = pyemu.Matrix(x=data,row_names=row_names,col_names=col_names)
mat.to_binary("mat.jco")

# load an existing jacobian matrix
jco = pyemu.Jco.from_binary("pest.jco")
# form an observation noise covariance matrix from weights
obscov = pyemu.Cov.from_observation_data(pst)
# form the normal matrix, aligning rows and cols on-the-fly
xtqx = jco * obscov.inv * jco.T

Note

this class makes heavy use of property decorators to encapsulate private attributes

property T

wrapper function for Matrix.transpose() method

Returns:: transpose of Matrix
Return type:: Matrix

Note

returns a copy of self

A syntatic-sugar overload of Matrix.transpose()

Example:

jcb = pyemu.Jco.from_binary("pest.jcb")
jcbt = jcb.T

align(names, axis=None)

reorder Matrix by names in place. If axis is None, reorder both indices

Parameters:

names (['str']) – names in Matrix.row_names and or Matrix.col_names
axis (int, optional) – the axis to reorder. if None, reorder both axes

Note

Works in place. Is called programmatically during linear algebra operations

Example:

# load a jco that has more obs (rows) than non-zero weighted obs
# in the control file
jco = pyemu.Jco.from_binary("pest.jcb")
# get an obs noise cov matrix
obscov = pyemu.Cov.from_observation_data(pst)
jco.align(obscov.row_names,axis=0)

property as_2d

get a 2D numeric representation of Matrix.x. If not isdiagonal, simply return reference to Matrix.x, otherwise, constructs and returns a 2D, diagonal ndarray

Returns:: numpy.ndarray
Return type:: numpy.ndarray

Exmaple:

# A diagonal cov
cov = pyemu.Cov.from_parameter_data
x2d = cov.as_2d # a numpy 2d array
print(cov.shape,cov.x.shape,x2d.shape)

binary_header_dt = dtype([('itemp1', '<i4'), ('itemp2', '<i4'), ('icount', '<i4')])

binary_rec_dt = dtype([('j', '<i4'), ('dtemp', '<f8')])

char: alias of uint8

coo_rec_dt = dtype([('i', '<i4'), ('j', '<i4'), ('dtemp', '<f8')])

copy()

get a copy of Matrix

Returns:: copy of this Matrix
Return type:: Matrix

df(): wrapper of Matrix.to_dataframe()

double: alias of float64

drop(names, axis)

drop elements from Matrix in place

Parameters:

names (['str']) – list of names to drop
axis (int) – the axis to drop from. must be in [0,1]

element_isaligned(other)

check if matrices are aligned for element-wise operations

Parameters:: other (Matrix) – the other matrix to check for alignment with
Returns:: True if aligned, False if not aligned
Return type:: bool

extend(other)

extend Matrix with the elements of other, returning a new matrix.

Args: other (Matrix): the Matrix to extend self by

Returns:: new, extended Matrix
Return type:: Matrix

Note

No row or column names can be shared between self and other

Example:

jco1 = pyemu.Jco.from_binary("pest_history.jco")
jco2 = pyemu.Jco.from_binary("pest_forecast.jco")

jco_ext = jco1.extend(jco2)

extract(row_names=None, col_names=None)

wrapper method that Matrix.gets() then Matrix.drops() elements. one of row_names or col_names must be not None.

Parameters:

row_names (['str'], optional) – row_names to extract. If None, all row_names are retained.
col_names (['str'], optional) – col_names to extract. If None, all col_names are retained.

Returns:

the extract sub-matrix defined by row_names and/or col_names

Return type:

Matrix

Example:

cov = pyemu.Cov.from_parameter_data(pst)
hk_cov = cov.extract(row_names=["hk1","hk2","hk3"])

static find_rowcol_indices(names, row_names, col_names, axis=None)

fast(er) look of row and colum names indices

Parameters:

names ([str]) – list of names to look for in row_names and/or col_names names
row_names ([str]) – list of row names
col_names ([str]) – list of column names
axis (int, optional) – axis to search along. If None, search both. Default is None

Returns:

array of (integer) index locations. If axis is None, a 2 numpy.ndarrays of both row and column name indices is returned

Return type:

numpy.ndarray

classmethod from_ascii(filename)

load a PEST-compatible ASCII matrix/vector file into a Matrix instance

Parameters:: filename (str) – name of the file to read
Returns:: Matrix loaded from ASCII file
Return type:: Matrix

Example:

mat = pyemu.Matrix.from_ascii("my.mat")
cov = pyemi.Cov.from_ascii("my.cov")

classmethod from_binary(filename, forgive=False)

class method load from PEST-compatible binary file into a Matrix instance

Parameters:

filename (str) – filename to read
forgive (bool) – flag to forgive incomplete data records. Only applicable to dense binary format. Default is False

Returns:

Matrix loaded from binary file

Return type:

Matrix

Example:

mat = pyemu.Matrix.from_binary("my.jco")
cov = pyemu.Cov.from_binary("large_cov.jcb")

classmethod from_dataframe(df)

class method to create a new Matrix instance from a: pandas.DataFrame

Parameters:: df (pandas.DataFrame) – dataframe
Returns:: Matrix instance derived from df.
Return type:: Matrix

Example:

df = pd.read_csv("my.csv")
mat = pyemu.Matrix.from_dataframe(df)

static from_fortranfile(filename)

a binary load method to accommodate one of the many: bizarre fortran binary writing formats

Parameters:

filename (str) – name of the binary matrix file

Returns:

tuple containing

numpy.ndarray: the numeric values in the file
[‘str’]: list of row names
[`str`]: list of col_names

classmethod from_names(row_names, col_names, isdiagonal=False, autoalign=True, random=False)

class method to create a new Matrix instance from row names and column names, filled with trash

Parameters:

row_names (['str']) – row names for the new Matrix
col_names (['str']) – col_names for the new matrix
isdiagonal (bool, optional) – flag for diagonal matrix. Default is False
autoalign (bool, optional) – flag for autoaligning new matrix during linear algebra calcs. Default is True
random (bool) – flag for contents of the trash matrix. If True, fill with random numbers, if False, fill with zeros Default is False

Returns:

the new Matrix instance

Return type:

Matrix

Example:

row_names = ["row_1","row_2"]
col_names = ["col_1,"col_2"]
m = pyemu.Matrix.from_names(row_names,col_names)

property full_s

Get the full singular value matrix

Returns:: full singular value matrix. Shape is (max(Matrix.shape),max(Matrix.shape)) with zeros along the diagonal from min(Matrix.shape) to max(Matrix.shape)
Return type:: Matrix

Example:

jco = pyemu.Jco.from_binary("pest.jcb")
jco.full_s.to_ascii("full_sing_val_matrix.mat")

get(row_names=None, col_names=None, drop=False)

get a new Matrix instance ordered on row_names or col_names

Parameters:

row_names (['str'], optional) – row_names for new Matrix. If None, all row_names are used.
col_names (['str'], optional) – col_names for new Matrix. If None, all col_names are used.
drop (bool) – flag to remove row_names and/or col_names from this Matrix

Returns:

a new Matrix

Return type:

Matrix

Example::: # load a jco that has more obs (rows) than non-zero weighted obs # in the control file jco = pyemu.Jco.from_binary(“pest.jcb”) # get an obs noise cov matrix obscov = pyemu.Cov.from_observation_data(pst) nnz_jco = jco.get(row_names = obscov.row_names)

get_diagonal_vector(col_name='diag')

Get a new Matrix instance that is the diagonal of self. The shape of the new matrix is (self.shape[0],1). Self must be square

Parameters:: col_name (str) – the name of the single column in the new Matrix
Returns:: vector-shaped Matrix instance of the diagonal of this Matrix
Return type:: Matrix

Example:

cov = pyemu.Cov.from_unc_file("my.unc")
cov_diag = cov.get_diagonal_vector()
print(cov_diag.col_names)

get_maxsing(eigthresh=1e-05)

Get the number of singular components with a singular value ratio greater than or equal to eigthresh

Args:

eigthresh (float): the ratio of smallest to largest
singular value to retain. Since it is assumed that s is sorted from largest to smallest, once a singular value is reached that yields a ratio with the first (largest) singular value, the index of this singular is returned.

Returns:: the index of the singular value whos ratio with the first singular value is less than or equal to eigthresh
Return type:: int

Note

this method calls the static method Matrix.get_maxsing_from_s() with Matrix.s.x

Example:

jco = pyemu.Jco.from_binary("pest.jco")
max_sing = jco.get_maxsing(eigthresh=pst.svd_data.eigthresh)

static get_maxsing_from_s(s, eigthresh=1e-05)

static method to work out the maxsing for a given singular spectrum

Parameters:

s (numpy.ndarray) – 1-D array of singular values. This array should come from calling either numpy.linalg.svd or from the pyemu.Matrix.s.x attribute
eigthresh (float) – the ratio of smallest to largest singular value to retain. Since it is assumed that s is sorted from largest to smallest, once a singular value is reached that yields a ratio with the first (largest) singular value, the index of this singular is returned.

Returns:

the index of the singular value whos ratio with the first singular value is less than or equal to eigthresh

Return type:

int

Example:

jco = pyemu.Jco.from_binary("pest.jco")
max_sing = pyemu.Matrix.get_maxsing_from_s(jco.s,eigthresh=pst.svd_data.eigthresh)

hadamard_product(other)

Overload of numpy.ndarray.__mult__(): element-wise multiplication. Tries to speedup by checking for scalars of diagonal matrices on either side of operator

Parameters:: other – (int,`float`,`numpy.ndarray`,`Matrix`): the thing to multiply
Returns:: the result of multiplication
Return type:: Matrix

Note

if Matrix and other (if applicable) have autoalign set to True, both Matrix and other are aligned based on row and column names. If names are not common between the two, this may result in a smaller returned Matrix. If not common elements are shared, an excetion is raised

Example:

cov = pyemu.Cov.from_parameter_data(pst)
cov2 = cov * 10
cov3 = cov * cov2

indices(names, axis=None)

get the row and col indices of names. If axis is None, two ndarrays: are returned, corresponding the indices of names for each axis

Parameters:

names ([str]) – list of names to look for in row_names and/or col_names names
row_names ([str]) – list of row names
col_names ([str]) – list of column names
axis (int, optional) – axis to search along. If None, search both. Default is None

Returns:

array of (integer) index locations. If axis is None, a 2 numpy.ndarrays of both row and column name indices is returned

Return type:

numpy.ndarray

Note

thin wrapper around Matrix.find_rowcol_indices static method

integer: alias of int32

property inv

inversion operation of Matrix

Returns:: inverse of Matrix
Return type:: Matrix

Note

uses numpy.linalg.inv for the inversion

Example:

mat = pyemu.Matrix.from_binary("my.jco")
mat_inv = mat.inv
mat_inv.to_binary("my_inv.jco")

mult_isaligned(other)

check if matrices are aligned for dot product multiplication

Parameters:: other (Matrix) – the other matrix to check for alignment with
Returns:: True if aligned, False if not aligned
Return type:: bool

property ncol

length of second dimension

Returns:: number of columns
Return type:: int

new_obs_length = 200

new_par_length = 200

property newx

return a copy of Matrix.x attribute

Returns:: a copy Matrix.x
Return type:: numpy.ndarray

property nrow

length of first dimension

Returns:: number of rows
Return type:: int

obs_length = 20

par_length = 12

pseudo_inv(maxsing=None, eigthresh=1e-05)

The pseudo inverse of self. Formed using truncated singular value decomposition and Matrix.pseudo_inv_components

Parameters:

maxsing (int, optional) – the number of singular components to use. If None, maxsing is calculated using Matrix.get_maxsing() and eigthresh
eigthresh – (float, optional): the ratio of largest to smallest singular components to use for truncation. Ignored if maxsing is not None. Default is 1.0e-5

Returns:

the truncated-SVD pseudo inverse of Matrix (V_1 * s_1^-1 * U^T)

Return type:

Matrix

Example:

jco = pyemu.Jco.from_binary("pest.jcb")
jco_psinv = jco.pseudo_inv(pst.svd_data.maxsing,pst.svd_data.eigthresh)
jco_psinv.to_binary("pseudo_inv.jcb")

pseudo_inv_components(maxsing=None, eigthresh=1e-05, truncate=True)

Get the (optionally) truncated SVD components

Parameters:

maxsing (int, optional) – the number of singular components to use. If None, maxsing is calculated using Matrix.get_maxsing() and eigthresh
eigthresh – (float, optional): the ratio of largest to smallest singular components to use for truncation. Ignored if maxsing is not None. Default is 1.0e-5
truncate (bool) – flag to truncate components. If False, U, s, and V will be zeroed out at locations greater than maxsing instead of truncated. Default is True

Returns:

tuple containing

Matrix: (optionally truncated) left singular vectors
Matrix: (optionally truncated) singular value matrix
Matrix: (optionally truncated) right singular vectors

Example:

mat = pyemu.Matrix.from_binary("my.jco")
u1,s1,v1 = mat.pseudo_inv_components(maxsing=10)
resolution_matrix = v1 * v1.T
resolution_matrix.to_ascii("resol.mat")

static read_ascii(filename)

read a PEST-compatible ASCII matrix/vector file

Parameters:

filename (str) – file to read from

Returns:

tuple containing

numpy.ndarray: numeric values
[‘str’]: list of row names
[`str`]: list of column names
bool: diagonal flag

static read_binary(filename, forgive=False)

static method to read PEST-format binary files

Parameters:

filename (str) – filename to read
forgive (bool) – flag to forgive incomplete data records. Only applicable to dense binary format. Default is False

Returns:

tuple containing

numpy.ndarray: the numeric values in the file
[‘str’]: list of row names
[`str`]: list of col_names

static read_binary_header(filename)

read the first elements of a PEST(++)-style binary file to get format and dimensioning information.

Parameters:

filename (str) – the filename of the binary file

Returns:

tuple containing

int: the itemp1 value
int: the itemp2 value
int: the icount value

static read_dense(filename, forgive=False, close=True)

read a dense-format binary file.

Parameters:

filename (str) – the filename or the open filehandle
forgive (bool) – flag to forgive incomplete records. If True and an incomplete record is encountered, only the previously read records are returned. If False, an exception is raised for an incomplete record
close (bool) – flag to close the filehandle. Default is True

Returns:

tuple containing

numpy.ndarray: the numeric values in the file
[‘str’]: list of row names
[`str`]: list of col_names

reset_x(x, copy=True)

reset self.__x private attribute

Parameters:

x (numpy.ndarray) – the new numeric data
copy (bool) – flag to make a copy of ‘x’. Default is True

Returns:

None

Note

operates in place

property s

the singular value (diagonal) Matrix

Returns:: singular value matrix. shape is (min(Matrix.shape),min(Matrix.shape))
Return type:: Matrix

Example:

# plot the singular spectrum of the jacobian
import matplotlib.pyplot as plt
jco = pyemu.Jco.from_binary("pest.jcb")
plt.plot(jco.s.x)
plt.show()

property shape

get the implied, 2D shape of Matrix

Returns:: length of 2 tuple
Return type:: int

Exmaple:

jco = pyemu.Jco.from_binary("pest.jcb")
shape = jco.shape
nrow,ncol = shape #unpack to ints

property sqrt

element-wise square root operation

Returns:: element-wise square root of Matrix
Return type:: Matrix

Note

uses numpy.sqrt

Example:

cov = pyemu.Cov.from_uncfile("my.unc")
sqcov = cov.sqrt
sqcov.to_uncfile("sqrt_cov.unc")

to_2d()

get a 2D Matrix representation of Matrix. If not Matrix.isdiagonal, simply: return a copy of Matrix, otherwise, constructs and returns a new Matrix instance that is stored as diagonal

Returns:: non-diagonal form of Matrix
Return type:: Martrix

Exmaple:

# A diagonal cov
cov = pyemu.Cov.from_parameter_data
cov2d = cov.as_2d # a numpy 2d array
print(cov.shape,cov.x.shape,cov2d.shape,cov2d.x.shape)

to_ascii(filename, icode=2)

write a PEST-compatible ASCII Matrix/vector file

Parameters:

filename (str) – filename to write to
icode (int, optional) – PEST-style info code for matrix style. Default is 2.

Note

if icode == -1, a 1-d vector is written that represents a diagonal matrix. An exception is raised if icode == -1 and isdiagonal is False

to_binary(filename, droptol=None, chunk=None)

write a PEST-compatible binary file. The format is the same as the format used to storage a PEST Jacobian matrix

Parameters:

filename (str) – filename to save binary file
droptol (float) – absolute value tolerance to make values smaller droptol than zero. Default is None (no dropping)
chunk (int) – number of elements to write in a single pass. Default is None, which writes the entire numeric part of the Matrix at once. This is faster but requires more memory.

to_coo(filename, droptol=None, chunk=None)

write an extended PEST-format binary file. The data format is [int,int,float] for i,j,value. It is autodetected during the read with Matrix.from_binary().

Parameters:

filename (str) – filename to save binary file
droptol (float) – absolute value tolerance to make values smaller droptol than zero. Default is None (no dropping)
chunk (int) – number of elements to write in a single pass. Default is None, which writes the entire numeric part of the Matrix at once. This is faster but requires more memory.

Note

This method is needed when the number of dimensions times 2 is larger than the max value for a 32-bit integer. happens! This method is used by pyemu.Ensemble.to_binary()

to_dataframe()

return a pandas.DataFrame representation of Matrix

Returns:: a dataframe derived from Matrix
Return type:: pandas.DataFrame

Note

if self.isdiagonal is True, the full matrix is used to fill the dataframe - lots of zeros.

to_dense(filename, close=True)

experimental new dense matrix storage format to support faster I/O with ensembles

Parameters:

filename (str) – the filename to save to
close (bool) – flag to close the filehandle after saving

Returns:

the file handle. Only returned if close is False

Return type:

f (file)

Note

calls Matrix.write_dense()

property transpose

transpose operation of self

Returns:: transpose of Matrix
Return type:: Matrix

Example:

jcb = pyemu.Jco.from_binary("pest.jcb")
jcbt = jcb.T

property u

the left singular vector Matrix

Returns:: left singular vectors. Shape is (Matrix.shape[0], Matrix.shape[0])
Return type:: Matrix

Example:

jco = pyemu.Jco.from_binary("pest.jcb")
jco.u.to_binary("u.jcb")

property v

the right singular vector Matrix

Returns:: right singular vectors. Shape is (Matrix.shape[1], Matrix.shape[1])
Return type:: Matrix

Example:

jco = pyemu.Jco.from_binary("pest.jcb")
jco.v.to_binary("v.jcb")

static write_dense(filename, row_names, col_names, data, close=False)

experimental new dense matrix storage format to support faster I/O with ensembles

Parameters:

filename (str or file) – the file to write to
row_names ([str]) – row names of the matrix
col_names ([str]) – col names of the matrix
data (np.ndarray) – matrix elements
close (bool) – flag to close the file after writing

Returns:

the file handle. Only returned if close is False

Return type:

f (file)

property x

return a reference to Matrix.x

Returns:: reference to Matrix.x
Return type:: numpy.ndarray

property zero2d

get an 2D instance of self with all zeros

Returns:: Matrix of zeros
Return type:: Matrix

Example:

jco = pyemu.Jco.from_binary("pest.jcb")
zero_jco = jco.zero2d

pyemu.mat.mat_handler.concat(mats)

Concatenate Matrix objects. Tries either axis.

Parameters:: mats ([Matrix]) – list of Matrix objects
Returns:: a concatenated Matrix instance
Return type:: pyemu.Matrix

pyemu.mat.mat_handler.get_common_elements(list1, list2)

find the common elements in two lists. used to support auto align: might be faster with sets

Parameters:

list1 ([str]) – a list of strings (could be either row or col names, depending on calling function)
list2 ([str]) – a list of strings (could be either row or col names, depending on calling function)

Returns:

list of common strings shared by list1 and list2

Return type:

[str]

Note

result is not ordered WRT list1 or list2

pyemu.mat.mat_handler.save_coo(x, row_names, col_names, filename, chunk=None)

write a PEST-compatible binary file. The data format is [int,int,float] for i,j,value. It is autodetected during the read with Matrix.from_binary().

Parameters:

x (numpy.sparse) – coo sparse matrix
row_names ([str]) – list of row names
col_names (['str]) – list of col_names
filename (str) – filename
droptol (float) – absolute value tolerance to make values smaller than droptol zero. Default is None (no dropping)
chunk (int) – number of elements to write in a single pass. Default is None

Module contents

This module contains classes for handling matrices in a linear algebra setting. The primary objects are the Matrix and Cov. These objects overload most numerical operators to autoalign the elements based on row and column names.