otoole.preprocess package#

Submodules#

otoole.preprocess.longify_data module#

Read in a folder of irregular wide-format csv files and write them out as narrow csvs

otoole.preprocess.longify_data.check_datatypes(df: DataFrame, config_details: Dict, parameter: str) DataFrame[source]#

Checks a parameters datatypes

Parameters
  • df (pandas.DataFrame) – The parameter data

  • config_details (dict) – The configuration dictionary

  • parameter (str) – The name of the parameter

otoole.preprocess.longify_data.check_set_datatype(narrow: DataFrame, config_details: Dict, set_name: str) DataFrame[source]#

Checks the datatypes of a set_name dataframe

Parameters
  • narrow (pandas.DataFrame) – The set data

  • config_details (dict) – The configuration dictionary

  • set_name (str) – The name of the set

otoole.preprocess.setup module#

Module to create template data

otoole.preprocess.setup.get_config_setup_data() Dict[str, Any][source]#

Reads in template config data

otoole.preprocess.setup.get_csv_setup_data(config: Dict[str, Any]) Tuple[Dict[str, DataFrame], Dict[str, Any]][source]#

Gets template dataframe data to write as csvs

Parameters

config (Dict[str,Any]) – Configuration data to get CSV data to match against

Returns

  • Dict[str (pd.DataFrame]) – Parameters with empty template dataframes

  • Dict[str (Any]) – Default values extracted from config file

otoole.preprocess.validate_config module#

Validation methods for the user configuration file.

class otoole.preprocess.validate_config.UserDefinedParameter(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#

Bases: UserDefinedValue

Represents a parameter

classmethod check_dtype(value, info)[source]#
check_dtype_default()[source]#
check_index_in_set()[source]#
classmethod check_required_inputs(values)[source]#
model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class otoole.preprocess.validate_config.UserDefinedResult(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#

Bases: UserDefinedValue

Represents a result

classmethod check_deprecated_values(values)[source]#
classmethod check_dtype(value, info)[source]#
check_dtype_default()[source]#
check_index_in_set()[source]#
classmethod check_required_inputs(values)[source]#
model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class otoole.preprocess.validate_config.UserDefinedSet(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#

Bases: UserDefinedValue

Represents a set

classmethod check_dtype(value, info)[source]#
model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class otoole.preprocess.validate_config.UserDefinedValue(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#

Bases: BaseModel

Represents any user defined value

calculated: Optional[bool]#
classmethod check_name_for_numbers(value)[source]#
classmethod check_name_for_spaces(value)[source]#
classmethod check_name_for_special_chars(value)[source]#
classmethod check_name_length(values)[source]#
classmethod check_param_type(value, info)[source]#
default: Optional[Union[int, float]]#
defined_sets: Optional[List[str]]#
dtype: str#
indices: Optional[List[str]]#
model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str#
short_name: Optional[str]#
type: str#

Module contents#