otoole.preprocess package#
Submodules#
otoole.preprocess.longify_data module#
Read in a folder of irregular wide-format csv files and write them out as narrow csvs
- otoole.preprocess.longify_data.check_datatypes(df: DataFrame, config_details: Dict, parameter: str) DataFrame [source]#
Checks a parameters datatypes
- Parameters
df (pandas.DataFrame) – The parameter data
config_details (dict) – The configuration dictionary
parameter (str) – The name of the parameter
- otoole.preprocess.longify_data.check_set_datatype(narrow: DataFrame, config_details: Dict, set_name: str) DataFrame [source]#
Checks the datatypes of a set_name dataframe
- Parameters
narrow (pandas.DataFrame) – The set data
config_details (dict) – The configuration dictionary
set_name (str) – The name of the set
otoole.preprocess.setup module#
Module to create template data
- otoole.preprocess.setup.get_config_setup_data() Dict[str, Any] [source]#
Reads in template config data
- otoole.preprocess.setup.get_csv_setup_data(config: Dict[str, Any]) Tuple[Dict[str, DataFrame], Dict[str, Any]] [source]#
Gets template dataframe data to write as csvs
- Parameters
config (Dict[str,Any]) – Configuration data to get CSV data to match against
- Returns
Dict[str (pd.DataFrame]) – Parameters with empty template dataframes
Dict[str (Any]) – Default values extracted from config file
otoole.preprocess.validate_config module#
Validation methods for the user configuration file.
- class otoole.preprocess.validate_config.UserDefinedParameter(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#
Bases:
UserDefinedValue
Represents a parameter
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].
This replaces Model.__fields__ from Pydantic V1.
- class otoole.preprocess.validate_config.UserDefinedResult(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#
Bases:
UserDefinedValue
Represents a result
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].
This replaces Model.__fields__ from Pydantic V1.
- class otoole.preprocess.validate_config.UserDefinedSet(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#
Bases:
UserDefinedValue
Represents a set
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].
This replaces Model.__fields__ from Pydantic V1.
- class otoole.preprocess.validate_config.UserDefinedValue(*, name: str, type: str, dtype: str, defined_sets: Optional[List[str]] = None, indices: Optional[List[str]] = None, default: Optional[Union[int, float]] = None, calculated: Optional[bool] = None, short_name: Optional[str] = None)[source]#
Bases:
BaseModel
Represents any user defined value
- model_config: ClassVar[ConfigDict] = {'extra': 'forbid'}#
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- model_fields: ClassVar[dict[str, FieldInfo]] = {'calculated': FieldInfo(annotation=Union[bool, NoneType], required=False), 'default': FieldInfo(annotation=Union[int, float, NoneType], required=False), 'defined_sets': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'dtype': FieldInfo(annotation=str, required=True), 'indices': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'name': FieldInfo(annotation=str, required=True), 'short_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'type': FieldInfo(annotation=str, required=True)}#
Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].
This replaces Model.__fields__ from Pydantic V1.