Transform Pipeline Components

This module exposes the primary class that contains all DataPath objects and is responsible for running all transformations.

class gretel_client.transformers.data_transform_pipeline.DataTransformPipeline(data_paths: List[gretel_client.transformers.data_pipeline.DataPath])

This class is a container for data paths describing a records transformations.

It constructs a data pipeline from a list of DataPath objects and is used to process records based on the order of the data path list.

Parameters

data_paths – A list of data_path instancess

transform_df(df: None) → None

Helper method that can consume a DataFrame and iterate over each record as a dictionary, then run the transform pipeline on each record.