Reminder: Config vs Entities
The data nodes, tasks, pipelines, and scenarios concepts have two types of Taipy objects related to them: configs and runtime entities.
Remember that each entity is created from a config (e.g. a Data node is created from a Data node config, a Task from a Task Config, a Pipeline from a Pipeline config, a Scenarios from a Scenario config, etc). Remember also that the same config can be used to instantiate multiple entities (e.g., a scenario config can be used to instantiate two different scenarios).
Let's take some scenario entity use cases to illustrate the logic behind the execution flow. For this purpose, we will use the configuration graph below.
The picture above represents the graph of configs objects (
Single Scenario Execution¶
Let’s assume we instantiate a new scenario entity from the scenario configuration above (see the previous configuration graph). Let’s call it Scenario 1. All the entities of Scenario 1 will be instantiated from the various configuration objects.
Scenario 1 can also be represented as the entity graph below:
Thanks to this graph representation, Taipy automatically understand the execution precedence constraints. When a scenario/pipeline is submitted for execution, the tasks are smartly scheduled and executed in the correct sequence.
Taipy also optimizes the execution of pipelines and scenarios by not recomputing tasks that do not need to be re-executed. This is the concept of caching.
Let’s assume that scenario 1 has already been executed. If the end-user decides to re-execute the same scenario, then only three situations can occur:
- Situation 1: Taipy re-executes all the tasks. This is the default behavior.
- Situation 2: None of the data nodes have been modified since the last run. The data nodes sales
predictions and production orders are cached, then they don't need to be re-executed. Taipy does not
re-execute the tasks. (Please refer to
DataNodeConfigdocumentation to activate the "cacheable" feature).
- Situation 3: If at least one of the input data nodes has been modified since the last run, Taipy only
executes the "appropriate" tasks. It implies that:
- If the predict task entity has any of its two input data nodes (trained model or current month) changed since the last run, then in the second run, Taipy re-executes both the predict and the planning tasks.
- If trained model and current month have not been modified but capacity has, then Taipy only re-executes the planning task.
Multiple Scenario Execution¶
Let’s continue with the previous example by creating a second scenario from the same config. Let’s call it Scenario 2. In the case of two scenarios instantiated from the same configuration, what could be the impact of executing scenario 1 over the execution of scenario 2.
Two options are possible with Taipy:
- Option 1 : Taipy assumes that the entities of each scenario are "local" to each scenario and are not shared with other scenarios. Similar to the first scenario, the following entity graph is created for Scenario 2.
Note that even if both scenarios share the same configuration graph, all their data nodes and tasks are separate
instances. Scenario 2's data nodes and tasks are separate instances and are prefixed by
- Option 2 : Taipy assumes that some entities of the scenarios are "global" and are shared among scenarios. For instance, let's assume that current month can be shared by Scenario 1 and Scenario 2. In this second option, the following entity graph is created for Scenario 2.
To illustrate the difference between the two options, let’s assume that:
- First, Scenario 1 has been executed with its current month entity set to "January".
- Then, the current month data node entity of Scenario 2 has been set to "February".
If Option 1 applies, when the end-user re-executes Scenario 1, Taipy produces the same execution as before (or does not re-execute the tasks if the "cacheable" feature is activated).
If Option 2 applies, current month and current month 2 data nodes are the same entity. When the end–user re-execute Scenario 1, Taipy detects that the current month data node entity has changed. Then re-executing Scenario 1, runs the entity graph with the current month set to "February".
To enable the difference in behavior, Taipy introduces the concept of Scope. In Taipy, Option 1 corresponds to the
by-default behavior: the SCOPE default value is
SCENARIO. While Option 2 corresponds to setting the
scope to a level above