Skip to content

Execution flow

Reminder: Config vs Entities

The data nodes, tasks, pipelines, and scenarios concepts have two types of Taipy objects related to them: configs and runtime entities.

Remember that each entity is created from a config (e.g. a Data node is created from a Data node config, a Task from a Task Config, a Pipeline from a Pipeline config, a Scenarios from a Scenario config, etc). Remember also that the same config can be used to instantiate multiple entities (e.g., a scenario config can be used to instantiate two different scenarios).

Let's take some scenario entity use cases to illustrate the logic behind the execution flow. For this purpose, we will use the configuration graph below.

Configuration Graph

The picture above represents the graph of configs objects (DataNodeConfig and TaskConfig objects).

Single Scenario Execution

Let’s assume we instantiate a new scenario entity from the scenario configuration above (see the previous configuration graph). Let’s call it Scenario 1. All the entities of Scenario 1 will be instantiated from the various configuration objects.

Scenario 1 can also be represented as the entity graph below:

Scenario 1 Graph

Thanks to this graph representation, Taipy automatically understand the execution precedence constraints. When a scenario/pipeline is submitted for execution, the tasks are smartly scheduled and executed in the correct sequence.

Taipy also optimizes the execution of pipelines and scenarios by not recomputing tasks that do not need to be re-executed. This is the concept of caching.

Let’s assume that scenario 1 has already been executed. If the end-user decides to re-execute the same scenario, then only three situations can occur:

  • Situation 1: Taipy re-executes all the tasks. This is the default behavior.
  • Situation 2: None of the data nodes have been modified since the last run. The data nodes sales predictions and production orders are cached, then they don't need to be re-executed. Taipy does not re-execute the tasks. (Please refer to DataNodeConfig documentation to activate the "cacheable" feature).
  • Situation 3: If at least one of the input data nodes has been modified since the last run, Taipy only executes the "appropriate" tasks. It implies that:
    • If the predict task entity has any of its two input data nodes (trained model or current month) changed since the last run, then in the second run, Taipy re-executes both the predict and the planning tasks.
    • If trained model and current month have not been modified but capacity has, then Taipy only re-executes the planning task.

Multiple Scenario Execution

Let’s continue with the previous example by creating a second scenario from the same config. Let’s call it Scenario 2. In the case of two scenarios instantiated from the same configuration, what could be the impact of executing scenario 1 over the execution of scenario 2.

Two options are possible with Taipy:

  • Option 1 : Taipy assumes that the entities of each scenario are "local" to each scenario and are not shared with other scenarios. Similar to the first scenario, the following entity graph is created for Scenario 2.

Scenario 2 Graph option 1

Note that even if both scenarios share the same configuration graph, all their data nodes and tasks are separate instances. Scenario 2's data nodes and tasks are separate instances and are prefixed by 2.

  • Option 2 : Taipy assumes that some entities of the scenarios are "global" and are shared among scenarios. For instance, let's assume that current month can be shared by Scenario 1 and Scenario 2. In this second option, the following entity graph is created for Scenario 2.

Scenario 2 Graph option 2

To illustrate the difference between the two options, let’s assume that:

  • First, Scenario 1 has been executed with its current month entity set to "January".
  • Then, the current month data node entity of Scenario 2 has been set to "February".

If Option 1 applies, when the end-user re-executes Scenario 1, Taipy produces the same execution as before (or does not re-execute the tasks if the "cacheable" feature is activated).

If Option 2 applies, current month and current month 2 data nodes are the same entity. When the end–user re-execute Scenario 1, Taipy detects that the current month data node entity has changed. Then re-executing Scenario 1, runs the entity graph with the current month set to "February".

To enable the difference in behavior, Taipy introduces the concept of Scope. In Taipy, Option 1 corresponds to the by-default behavior: the SCOPE default value is SCENARIO. While Option 2 corresponds to setting the scope to a level above SCENARIO (either CYCLE or GLOBAL).

The next section introduces the Scope concept.