Pipeline configs

A pipeline configuration is necessary to instantiate a Pipeline. To create a PipelineConfig, you can use the Config.configure_pipeline() method with the following parameters:

  • id: The id of this new pipeline configuration. This id is mandatory and must be a unique and valid Python identifier.
  • tasks: The list of task configurations.
  • properties: A dictionary of additional properties.

Reserved keys

Note that we cannot use the word "_entity_owner" as a key in the properties as it has been reserved for internal use.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from taipy import Config


def double(nb):
    return nb * 2


input_data_node_cfg = Config.configure_data_node("input",
                                                 default_data=21)
output_data_node_cfg = Config.configure_data_node("output")
task_cfg = Config.configure_task("double_task",
                                 double,
                                 input_data_node_cfg,
                                 output_data_node_cfg)

pipeline_cfg = Config.configure_pipeline("my_pipeline", [task_cfg])

In the previous code example, in line 16, we create a pipeline configuration with the id "my_pipeline" and made of a single task configuration task_config.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
from taipy import Config


def double(nb):
    return nb * 2


input_data_node_cfg = Config.configure_data_node("input",
                                                 default_data=21)
intermediate_data_node_cfg = Config.configure_data_node("intermediate")
output_data_node_cfg = Config.configure_data_node("output")
first_task_cfg = Config.configure_task("first_double_task",
                                       double,
                                       input_data_node_cfg,
                                       intermediate_data_node_cfg)
second_task_cfg = Config.configure_task("second_double_task",
                                        double,
                                        intermediate_data_node_cfg,
                                        output_data_node_cfg)

other_pipeline_cfg = Config.configure_pipeline("another_pipeline",
                                               [first_task_cfg, second_task_cfg])

In this second code example, in line 21, we create a pipeline configuration with the id "another_pipeline" and made of the two task configurations created in lines 12 and 16 first_task_cfg and second_task_cfg.

Note

Note that the order of the task_config in the list does not matter. The following lines are equivalent.

pipeline_cfg = Config.configure_pipeline("pipeline",
                                         [first_task_cfg, second_task_cfg])
pipeline_cfg = Config.configure_pipeline("pipeline",
                                         [second_task_cfg, first_task_cfg])

The next section introduces the scenario configuration.