navdata module

Base class for moving values between different modules/functions

class navdata.NavData(csv_path=None, pandas_df=None, numpy_array=None, **kwargs)

Bases: object

gnss_lib_py specific class for handling data.

Uses numpy for speed combined with pandas like intuitive indexing.

Can either be initialized empty, with a csv file by setting csv_path, a Pandas DataFrame by setting pandas_df or by a Numpy array by setting numpy_array.

Parameters:

• csv_path (string or path-like) – Path to csv file containing data

• pandas_df (pd.DataFrame) – Data used to initialize NavData instance.

• numpy_array (np.ndarray) – Numpy array containing data used to initialize NavData instance.

• **kwargs (args) – Additional arguments (e.g. sep or header) passed into pd.read_csv if csv_path is not None.

arr_dtype

Type of values stored in data array

Type:

numpy.dtype

orig_dtypes

Type of each original column if reading from a csv or Pandas dataframe.

Type:

pandas.core.series.Series

array

Array containing data, dimension M x N

Type:

np.ndarray

map

Map of the form {pandas column name : array row number }

Type:

Dict

str_map

Map of the form {pandas column name : {array value : string}}. Map is of the form {pandas column name : {}} for non string rows.

Type:

Dict

curr_cols

Current number of column for iterator, set to 0 by default

Type:

int

_build_navdata()

Build attributes for NavData.

_get_set_str_rows(rows, new_value)

Checks which output rows contain string elements given input.

If the row used to be string values but now is numeric, this function also empties the corresponding dictionary in str_map.

Parameters:

• rows (slice/list) – Rows to check for string elements

• new_value (np.ndarray/list/int) – Values to be added to self.array attribute

Returns:

• row_list (list) – Input rows, strictly in list format

• row_str_new (list) – List of boolean values indicating which of the new rows contain strings.

_get_str_rows(rows)

Checks which input rows contain string elements

Parameters:

rows (slice/list) – Rows to check for string elements

Returns:

• row_list (list) – Input rows, strictly in list format

• row_str (list) – List of boolean values indicating which rows contain strings

_get_strings(key)

Return list of strings for given key

Parameters:

key (string for column name required as string) –

Returns:

values_str – 1D array with string entries corresponding to dataset

Return type:

np.ndarray

_parse_key_idx(key_idx)

Break down input queries to relevant row and column indices

Parameters:

key_idx (str/list/tuple/slice/int) – Query for array items

Returns:

• rows (slice/list) – Rows to extract from the array

• cols (slice/list) – Columns to extract from the array

property _row_idx_str_bool

if data entry is string.

Row has string values if the string map is nonempty for a given row.

Returns:

_row_idx_str_bool – Dictionary of whether data at row number key is string or not

Return type:

Dict

Type:

Dictionary of index

static _row_map()

Map of row names from loaded to gnss_lib_py standard

Initializes as an emptry dictionary, must be reimplemented for custom parsers.

Returns:

row_map – Dictionary of the form {old_name : new_name}

Return type:

Dict

_str_2_val(new_str_vals, new_value, key)

Convert string valued arrays to values for storing in array

Parameters:

• new_str_vals (np.ndarray) – Array of dtype=self.arr_dtype where numeric values are to be stored

• new_value (np.ndarray) – Array of dtype=object, containing string values that are to be converted

• key (string) – Key indicating row where string to numeric conversion is required

argwhere(key_idx, value, condition='eq')

Return columns where conditions are met for the given row.

For string rows, only the “eq” and “neq” conditions are valid. The “value” argument can contain either a string, np.nan or an array-like object of strings. If an array-like object of strings is passed in then np.isin() is used to check the condition meaning that the returned subset will contain one of the values in the “value” array-like object for the “eq” condition or none of the values in the “value” array-like object for the “neq” condition.

For non-string rows, all valid conditions are listed in the “condition” argument description. The “value” argument can either contain a numeric or an array-like object of numerics for both the “eq” and “neq” conditions. If an array-like object is passed then the returned subset will contain one of the values in the “value” array-like object for the “eq” condition or none of the values in the “value” array-like object for the “neq” condition. For the “between” condition, the two limit values must be passed into the “value” argument as an array-like object.

Parameters:

• key_idx (string/int) – Key or index of the row in which conditions will be checked

• value (float/int/str/array-like) – Value that the row is checked against, array-like object possible for “eq”, “neq”, or “between” conditions.

• condition (string) – Condition type (greater than (“greater”)/ less than (“lesser”)/ equal to (“eq”)/ greater than or equal to (“geq”)/ lesser than or equal to (“leq”) / in between (“between”) inclusive of the provided limits / not equal to (“neq”))

Returns:

new_cols – Columns in NavData where given condition is satisfied for specified row

Return type:

list

copy(rows=None, cols=None)

Return copy of NavData keeping specified rows and columns

If None is passed into either argument, all rows or cols respectively are returned.

If no arguments are added .copy() returns a full copy of the entire NavData class.

Parameters:

• rows (None/list/np.ndarray) – Strings or integers indicating rows to keep in copy. Defaults to None meaning all rows are copied.

• cols (None/list/np.ndarray) – Integers indicating columns to keep in copy. Defaults to None meaning all cols are copied.

Returns:

new_navdata – Copy of original NavData with desired rows and columns

Return type:

gnss_lib_py.navdata.navdata.NavData

from_csv_path(csv_path, **kwargs)

Build attributes of NavData using csv file.

Parameters:

• csv_path (string or path-like) – Path to csv file containing data

• header (string, int, or None) – “infer” uses the first row as column names, setting to None will add int names for the columns.

• sep (char) – Delimiter to use when reading in csv file.

from_numpy_array(numpy_array)

Build attributes of NavData using np.ndarray.

Parameters:

numpy_array (np.ndarray) – Numpy array containing data used to initialize NavData instance.

from_pandas_df(pandas_df)

Build attributes of NavData using pd.DataFrame.

Parameters:

pandas_df (pd.DataFrame) – Data used to initialize NavData instance.

in_rows(rows)

Checks whether the given rows are in NavData.

If the rows are not in NavData, it creates a KeyError and lists all non-existent rows.

Parameters:

rows (string or list/np.ndarray/tuple of strings) – Indexes to check whether they are rows in NavData

property inv_map

Inverse dictionary map for label and row_number map

Returns:

inv_map – Dictionary of row_number : label

Return type:

Dict

is_str(row_name)

Check whether a row contained string values.

Parameters:

row_name (string) – Name of the row to check whether it contains string values.

Returns:

contains_str – True if the row contains string values, False otherwise.

Return type:

bool

property num_cols

Return the number of columns in the NavData instance.

Returns:

num_cols – Number of columns in the NavData instance.

Return type:

int

pandas_df()

Return pandas DataFrame equivalent to class

Returns:

df – DataFrame with data, including strings as strings

Return type:

pd.DataFrame

postprocess()

Postprocess loaded data. Optional in subclass

remove(rows=None, cols=None, inplace=False)

Reset NavData to remove specified rows and columns

Parameters:

• rows (None/list/np.ndarray/tuple) – Rows to remove from NavData

• cols (None/list/np.ndarray/tuple) – Columns to remove from NavData

• inplace (bool) – If False, will return new NavData instance with specified rows and columns removed. If True, will remove rows and columns from the current NavData instance.

Returns:

new_navdata – If inplace is False, returns NavData instance after removing specified rows and columns. If inplace is True, returns None.

Return type:

gnss_lib_py.navdata.navdata.NavData or None

rename(mapper=None, inplace=False)

Rename rows of NavData class.

Row names must be strings.

Parameters:

• mapper (dict) – Pairs of {“old_name” : “new_name”} for each row to be renamed.

• inplace (bool) – If False, will return new NavData instance with rows renamed. If True, will rename data rows in the current NavData instance.

Returns:

new_navdata – If inplace is False, returns NavData instance after renaming specified rows. If inplace is True, returns None.

Return type:

gnss_lib_py.navdata.navdata.NavData or None

replace(mapper=None, rows=None, inplace=False)

Replace data within rows or row names of NavData class.

Row names must be strings.

Parameters:

• mapper (dict) – Pairs of {“old_name” : “new_name”} for each value to replace. Values are replaced for each row in “rows” if “rows” is specified or for all rows if “rows” is left defaulted to None.

• rows (dict, or array-like) – If a dictionary is passed, then rows will be renamed according to pairs of {“old_name” : “new_name”}. If mapper is not None, then an array-like input may be passed to indicate which rows of values should be remapped.

• inplace (bool) – If False, will return new NavData instance with data replaced. If True, will replace data in the current NavData instance.

Returns:

new_navdata – If inplace is False, returns NavData instance after replacing specified data. If inplace is True, returns None.

Return type:

gnss_lib_py.navdata.navdata.NavData or None

property rows

Return all row names in instance as a list

Returns:

rows – List of row names in NavData

Return type:

list

property shape

Return shape of class

Returns:

shp – (M, N), M is number of rows and N number of time steps

Return type:

tuple

to_csv(output_path=None, index=False, prefix='', **kwargs)

Save data as csv.

Saves to a “results” directory if the output_path varaible is not set.

Parameters:

• output_path (string) – Path where csv should be saved

• index (bool) – If True, will write csv row names (index).

• prefix (string) – File prefix to add to filename if output_path not specified.

• header (bool) – If True (default), will list out names as columns

• sep (string) – Delimiter string of length 1, defaults to ‘,’

where(key_idx, value, condition='eq')

Return NavData where conditions are met for the given row.

For string rows, only the “eq” and “neq” conditions are valid. The “value” argument can contain either a string, np.nan or an array-like object of strings. If an array-like object of strings is passed in then np.isin() is used to check the condition meaning that the returned subset will contain one of the values in the “value” array-like object for the “eq” condition or none of the values in the “value” array-like object for the “neq” condition.

For non-string rows, all valid conditions are listed in the “condition” argument description. The “value” argument can either contain a numeric or an array-like object of numerics for both the “eq” and “neq” conditions. If an array-like object is passed then the returned subset will contain one of the values in the “value” array-like object for the “eq” condition or none of the values in the “value” array-like object for the “neq” condition. For the “between” condition, the two limit values must be passed into the “value” argument as an array-like object.

Parameters:

• key_idx (string/int) – Key or index of the row in which conditions will be checked

• value (float/int/str/array-like) – Value that the row is checked against, array-like object possible for “eq”, “neq”, or “between” conditions.

• condition (string) – Condition type (greater than (“greater”)/ less than (“lesser”)/ equal to (“eq”)/ greater than or equal to (“geq”)/ lesser than or equal to (“leq”) / in between (“between”) inclusive of the provided limits / not equal to (“neq”))

Returns:

new_navdata – NavData with columns where given condition is satisfied for specified row

Return type:

gnss_lib_py.navdata.navdata.NavData