scoped_processing_operator¶

Module that defines the SequenceOperator class

class tasrif.processing_pipeline.scoped_processing_operator.ScopedProcessingOperator(operation, observers=None)¶

Class represented a scoped operation where the operation is strictly defined as a lambda function taking any arguments and returning a ProcessingOperator

__init__(operation, observers=None)¶

Constructs a scoped processing operator where the operation is a lambda function

Parameters

(lambda Any (operation) – ProcessingOperator): Lambda returning a ProcessingOperator
observers (list[Observer]) – Python list of observers

Raises

ValueError – Occurs when operation is not a lambda function

Examples

>>> df1 = pd.DataFrame({'id': [1, 2, 3], 'cities': ['Rome', 'Barcelona', 'Stockholm']})
>>> df2 = pd.DataFrame({'id': [4, 5, 6], 'cities': ['Doha', 'Vienna', 'Belo Horizonte']})

>>> class TrainingOperator(ProcessingOperator):
>>>     def __init__(self, model, x=1, y=2):
>>>         super().__init__()
>>>         self.model = model
>>>         self.x = x
>>>         self.y = y
>>>     def _process(self, *args):
>>>         self.model.value = {'x': self.x, 'y': self.y}
>>>         return args
>>> class PredictionOperator(ProcessingOperator):
>>>     def __init__(self, model):
>>>         super().__init__()
>>>         self.model = model
>>>     def _process(self, *args):
>>>         print(self.model.value)
>>>         return args
>>> modela = Variable(None)
>>> s = SequenceOperator(processing_operators=[
>>>     TrainingOperator(model=modela),
>>>     PredictionOperator(model=modela),
>>>     ScopedProcessingOperator(lambda modelb=Variable():
>>>         SequenceOperator(processing_operators=[
>>>             TrainingOperator(model=modelb),
>>>             PredictionOperator(model=modelb),
>>>             ScopedProcessingOperator(lambda :
>>>                 SequenceOperator(processing_operators=[
>>>                     TrainingOperator(model=modelb),
>>>                     PredictionOperator(model=modelb)]))
>>>         ])
>>>     )
>>> ])

set_observers(observers)¶

Function to store the observers for the given operator.

Parameters: observers (list of Observer) – Observer objects that observe the operator

is_functional()¶

Function that returns whether the operator is functional or infrastructure

Returns: whether is_functional
Return type: is_functional (bool)

reduce_processing_operator sequence_operator