API Reference

Main Classes

class record_convertor.RecordConvertor(rule_source: str, field_convertor: type[FieldConvertorProtocol] | None = None, date_formatter: type[DateFormatProtocol] | None = None, data_classes: list[type[DataclassInstance]] | None = None, command_class: type[ProcessCommand] | None = None)

Bases: object

COMMAND_CLASS

alias of ProcessCommand

DATA_CLASS_PROCESSOR: DataClassProcessor = <record_convertor.dataclass_processor.DataClassProcessor object>

DEFAULT_DATE_FORMAT_CLASS

alias of DateFieldConvertor

DEFAULT_FIELD_CONVERTOR_CLASS

alias of BaseFieldConvertor

DEFAULT_VALUE: dict = {}

EVALUATE_CLASS

alias of EvaluateConditions

KEYS_IN_LOWER_CASE: bool = False

RULE_CLASS

alias of RulesFromYAML

convert(record: dict) → dict

Primary public method to run the actual conversion of the record.

Parameters:

record (dict) – input record

Returns:

dict – converted record

get_record_convertor_copy_with_new_rules(new_rules: BaseRuleDict | FormatDateRuleDict | SkipRuleDict | DataClassRuleDict | dict[str, str | dict | BaseRuleDict | FormatDateRuleDict | SkipRuleDict | DataClassRuleDict]) → RecordConvertor

Return a copy of the current record convertor instance with new rules.

class record_convertor.RecordConvertorWithRulesDict(rule_dict: dict, field_convertor: type[FieldConvertorProtocol] | None = None, date_formatter: type[DateFormatProtocol] | None = None, data_classes: list[type[DataclassInstance]] | None = None, command_class: type[ProcessCommand] | None = None)

Bases: RecordConvertor

RULE_CLASS

alias of RulesFromDict

Field Convertors

class record_convertor.field_convertors.base_convertor.BaseFieldConvertor

Bases: DataStructureConversions, GenericConversions, InPlaceBasicConversions, KeyValueConversions

class record_convertor.field_convertors.date_convertor.DateFieldConvertor

Bases: object

A class dedicated to converting date fields within records according to specified conversion rules.

This class applies a series of predefined actions to transform the format of date fields in a given record. It supports both direct and nested fields within the record’s dictionary structure. The conversion process is contingent upon the fulfillment of specified conditions, allowing for versatile date field manipulation.

Parameters: - record (Dict):

The record containing the date field to be converted. This argument expects a dictionary.

• conversion_rule (FormatDateRuleDict):

  A dictionary specifying the conversion rules. The rules dictionary should include the following keys:

  • ‘fieldname’:

    The name of the field to be converted.

  • ‘format’:

    The name of the input format.

  • ‘conditions’ (optional):

    Conditions that must be met for the conversion to proceed. If not provided, the conversion is assumed to be unconditional.

Available Conversions: - unix_dt_stamp: Converts Unix timestamp (string) to ‘YYYY-MM-DD’. - year_month_date: Converts ‘YYYY-MM-DD’ to ‘YYYY-MM-DD’ (identity transformation). - year_month_day_time: Extracts ‘YYYY-MM-DD’ from ‘YYYY-MM-DD:time’. - year_month_day: Converts ‘YYYY_MM_DD’ to ‘YYYY-MM-DD’. - day_month_year: Converts ‘DD-MM-YYYY’ to ‘YYYY-MM-DD’. - day_month_year_dotted: Converts ‘DD.MM.YYYY’ to ‘YYYY-MM-DD’.

Methods: - convert_date:

Converts the date field in the record to the ‘YYYY-MM-DD’ format based on the provided conversion rule.

• all_conditions_true:

  Evaluates if all specified conditions are met for the given date field value.

• update_field_with_date:

  Updates the specified field in the record with the new date format.

• _get_field:

  Retrieves the value from a potentially nested field within the record.

The conversion process supports handling both flat and nested dictionary structures, with the ability to navigate through nested fields specified by a dot-separated path.

Example

To convert a date field within a record, instantiate the DateFieldConvertor with the target record and a rule dict outlining the conversion specifics. Then, call the convert_date method to apply the conversion.

all_conditions_true(date_field_value: str, conversion_rule: FormatDateRuleDict) → bool

Returns True if all provided conditions are satisfied.

static day_month_year(date_str: str) → str

convert DD-MM-YYYY to YYYY-MM-DD

static day_month_year_dotted(date_str: str) → str

convert DD.MM.YYYY to YYYY-MM-DD

format_date_field(record: dict, conversion_rule: FormatDateRuleDict) → dict

Method to convert a date field in a record into into a ‘YYYY-MM-DD’ string date format.

Parameters:

• - record (Dict) – The record containing the date field to be converted.

• - conversion_rule (RuleDict) – The conversion rules specifying the fieldname, actions, and optional conditions.

static unix_dt_stamp(unix_dt_stamp: str) → str

convert Unix date time stamp to YYYY-MM-DD

update_field_with_date(date_in_new_format: str) → None

updates the datefield in the record to date_in_new_format

Parameters:

- date_in_new_format (str)

static year_month_date(date_str: str) → str

convert YYYY-MM-DD to YYYY-MM-DD

static year_month_day(date_str: str) → str

convert YYYY-MM-DD to YYYY-MM-DD

static year_month_day_time(date_str: str) → str

convert YYYY-MM-DD:time to YYYY-MM-DD

Command Processor

Module to provide DateFieldConvertor class.

This class allows you to do a number of date conversions on a record. This is usually done prior to creating a new record from this existing record, thus ensuring a well formatted record prior to processing.

Conditions can be included and conversions will only be executed if all conditions comply.

Availale conversion commands

• $fixed_value

  Returns the fixed value given in the arguments

• $split_field

  Splits a field based upon a given seperator and return a given entry (index) from the resulting split list Args: field_name (str), seperator (str), index (int)

• $int_from_string

  returns a numerical value (as a string) from the input string based upon the first sequence of numerical charraters found in the string. Characters can be removed first by adding them to the list in the seperators argument. ‘123 456.00 EUR’ can be converted to ‘123456’ with seperators arg [’ ‘] Args: field_name (str), seperators (list of str)

• $join

  return a joined set of fields and optionally fixed values with optionally a seperator in between the fields Args: [list of str] where str’s are

  • `$seperatorX’ where X is the seperator. Field is optional

    but if used should always be at index 0 of the list

  • <field_name> value will be retrieved from entry in record with field_name

  • <$fixed_value> value fixed_value will be used

• $normalized_address

  returns a normalized address as string using an external Geo API Args: address (str), zip_code (str), city (str),

  iso3116_country_code (str)

  -> all mandatory, representing the (nestred) field in the

  record where the actual values are to be retrieved

• $point

  returns a geojson Point dict Args: lat (str), lon (str)

  -> all mandatory, representing the (nested) field in the record

  where the actual values are to be retrieved

• $full_record

  returns the full record Args: none

• $join_key_value

  returns a key value pair from two record field defined in the args. If with retrieved values no key value pair can be made None is returned Args: key(str), value(str)

  -> both mandatory, representing the (nested) field in the record

  where the actual values are to be retrieved

• $get_coordinates

  returns a geoJson point dict representing the address in lon, lat. Args: address_keys(list of str), zip_code(str), city (str),

  iso3116_country_code (str)

  -> all are optional representing the (nested) field in the record

  where the actual values are to be retrieved

  -> address field is created by joining the different values for

  retrieved with the list of address_keys

• $from_list

  returns a list of dicts created from specific fields in a list of dicts provided by the record. This method can be used to transform a list of dicts into the correct format. Args: list_field_name (str) -> key retrieve the input list of dicts

  <keys> (str) -> keys to be used in new list of dicts. Value for

  this key is retrieved with the value of this key i.e. ‘target_field1’: ‘field1’ will result in ‘target_field1’: list_item[‘field1’]

• $to_list

  returns a list with values retrieved from the record. None values are skipped. Args: List of keys (str) for which the values need to be returned in

  the return list

• $to_list_dynamic

  returns a list with the results of the rule sets provided in the input list. This allows to create a list of more complex objects Args: List of rule dicts that result in the required data objects

  when processed against the input record. Each rule dict will result in a single entry in the list

• $to_int

  returns a string from the record with all strings removed indiacted by the skip list in the arguments. Args: field_name (str) -> field name from where to retrieve the string

  skip (list of str or str) -> strings that need to be removed from

  the input string

• $set_to_none_value

  return a None value

• $allow_none_value

  retrieves a value from the record but if not found leaves a None value instead of skipping the field

• $current_year

  sets the field value to the current year as a str

class record_convertor.command_processor.ProcessCommand(record: dict, process_command: str, process_args: dict | list | str, record_convertor, add_process_commands=None)

Bases: object

Class to create a value for the output record, mostly based upon one or more fields from the input record.

Parameters:

• record (dict) – record that needs some conversion action

• process_command (str) – process command to be executed to obtain the correct value from the record

• process_args (str, dict) – arguments needed to run the process command

• add_process_commands (dict) – dict with process names and custom lambda’s

Returns:

value (dict, list, str, int, float) – output of teh conversion

allow_none_value()

Returns value for field and None if no field can be found

current_year()

Returns current year in 4 decimals

first_item_from_list()

fixed_value()

return the (fixed) value given in the conversion args

from_list()

converts a list of dicts from the input record to a new cleaned list of dicts

full_record()

returns the full record

get_value()

calls the actual process command

first command is looked up in default commands. If not found there it is looked up in custom commands

int_from_string()

returns the value represented in a string in a string format

join()

joins the record values for the list of keys to a single string

join_key_value()

key_value()

point()

Retrieves lat, lon fields from record and returns them in point format.

set_to_none_value()

Returns None value

split_field()

Split the requested field and returns a specific entry from the split result

to_int()

turn a string into an int

to_list()

retrieve the values for a list of fields and returns them as a list

to_list_dynamic()

Dataclass Processor

This module provides functionality for using a dataclasses to generate a dict by using an input record and a set of rules. The rules determine how the input record will be convertod so that the dataclass instance can be generated. The dataclass might hold internal methods that can be called based upon the rules with arguments also based upon the rules.

The class support regestering custom dataclasses to be used.

The class uses an injected record convertor to process the input record bases upon the provided rules.

class record_convertor.dataclass_processor.DataClassProcessor

Bases: object

Processes dataclasses by applying a set of rules and converting dictionaries to dataclass instances.

This class provides methods to register dataclasses, apply conversion and transformation rules, and generate dataclass instances from dictionaries. It is designed to work with a customizable set of rules and leverages a protocol for converting records to dataclass-compatible formats.

data_from_dataclass(record: dict, rules: DataClassRuleDict, record_convertor: RecordConvertorProtocol) → dict

Uses a record as input to return a dict created by a dataclass. The dataclass in socpe is defined in the rules dict as well as how to convert the input record to be converted before is used to create the dataclass instance.

Parameters:

• record (dict) – The input dict used to create the dataclass instance (after conversion).

• rules (DataClassRuleDict) – The rules to apply during the conversion of the input dict and the creation of the dataclass instance.

• record_convertor (RecordConvertorProtocol) – The record convertor to use for data preparation.

Returns:

dict – The converted dataclass instance as a dictionary.

register_data_classes(dataclasses: list[Type[DataclassInstance]])

Registers a list of dataclasses.

Parameters:

dataclasses (list[Type[DataclassInstance]]) – A list of dataclass types to register.

register_dataclass(dataclass: Type[DataclassInstance])

Registers a single dataclass.

Parameters:

dataclass (Type[DataclassInstance]) – The dataclass type to register.

register_dict_of_data_classes(dataclasses: dict[str, Type[DataclassInstance]])

Registers multiple dataclasses provided in a dictionary where keys are the dataclass names.

Parameters:

dataclasses (dict[str, Type[DataclassInstance]]) – A dictionary of dataclass names to dataclass types.

Rules Generator

Conditions

Types and Protocols

class record_convertor.package_settings.package_types.BaseConvertorKeys

Bases: object

ACTIONS: Literal['actions'] = 'actions'

ACTIONTARGET: Literal['target_field_name'] = 'target_field_name'

ACTIONTYPE: Literal['action_type'] = 'action_type'

ACTIONVALUE: Literal['action_value'] = 'action_value'

CONDITION: Literal['condition'] = 'condition'

FIELDNAME: Literal['fieldname'] = 'fieldname'

class record_convertor.package_settings.package_types.BaseRuleDict

Bases: TypedDict

action_type: str | None

action_value: str | dict

actions: dict | None

condition: ConditionsDict | None

fieldname: str

format: str | None

class record_convertor.package_settings.package_types.DataClassRuleDict

Bases: TypedDict

data_class_name: str

methods: list[dict]

params: dict

class record_convertor.package_settings.package_types.DataClassRuleKeys

Bases: object

METHODS: Literal['methods'] = 'methods'

NAME: Literal['data_class_name'] = 'data_class_name'

RECORD_CONVERSION_ARGUMENTS: Literal['params'] = 'params'

class record_convertor.package_settings.package_types.FormatDateConvKeys

Bases: object

CONDITION: Literal['condition'] = 'condition'

DATEFIELD: Literal['date_field'] = 'date_field'

FORMAT: Literal['format'] = 'format'

class record_convertor.package_settings.package_types.FormatDateRuleDict

Bases: TypedDict

condition: ConditionsDict | None

date_field: str

format: str

class record_convertor.package_settings.package_types.RecConvKeys

Bases: object

CONVERT: Literal['$convert'] = '$convert'

SKIP: Literal['$skip'] = '$skip'

class record_convertor.package_settings.package_types.SkipConvKeys

Bases: object

CONDITION: Literal['condition'] = 'condition'

FIELDNAME: Literal['fieldname'] = 'fieldname'

class record_convertor.package_settings.package_types.SkipRuleDict

Bases: TypedDict

condition: ConditionsDict

fieldname: str

class record_convertor.package_settings.protocol.DataclassInstance(*args, **kwargs)

Bases: Protocol

class record_convertor.package_settings.protocol.DateFormatProtocol(*args, **kwargs)

Bases: Protocol

format_date_field(record: dict, conversion_rule: FormatDateRuleDict) → dict

class record_convertor.package_settings.protocol.FieldConvertorProtocol(*args, **kwargs)

Bases: Protocol

convert_field(record: dict, conversion_rule: BaseRuleDict) → dict

class record_convertor.package_settings.protocol.RecordConvertorProtocol(*args, **kwargs)

Bases: Protocol

convert(record: dict) → dict

get_record_convertor_copy_with_new_rules(new_rules: BaseRuleDict | FormatDateRuleDict | SkipRuleDict | DataClassRuleDict | dict[str, str | dict | BaseRuleDict | FormatDateRuleDict | SkipRuleDict | DataClassRuleDict]) → RecordConvertorProtocol