Skip to content

Bases: _Entity

Reference to a dataset.

A Data Node is an abstract class that holds metadata related to the dataset it refers to. In particular, a data node holds the name, the scope, the owner identifier, the last edit date, and some additional properties of the data.
A Data Node also contains information and methods needed to access the dataset. This information depends on the type of storage, and it is held by subclasses (such as SQL Data Node, CSV Data Node, ...).


It is recommended not to instantiate subclasses of DataNode directly.


Name Type Description
config_id str

Identifier of the data node configuration. It must be a valid Python identifier.

scope Scope

The scope of this data node.

id str

The unique identifier of this data node.

name str

A user-readable name of this data node.

owner_id str

The identifier of the owner (pipeline_id, scenario_id, cycle_id) or None.

parent_ids Optional[Set[str]]

The set of identifiers of the parent tasks.

last_edit_date datetime

The date and time of the last modification.

edits List[Edit]

The list of Edits (an alias for dict) containing medata about each edition of that node.

version str

The string indicates the application version of the data node to instantiate. If not provided, the current version is used.

validity_period Optional[timedelta]

The validity period of a data node. Implemented as a timedelta. If validity_period is set to None, the data_node is always up-to-date.

edit_in_progress bool

True if the data node is locked for modification. False otherwise.

kwargs bool

A dictionary of additional properties.

cacheable property writable

Deprecated. Use skippable attribute of a Task instead.

edition_in_progress property writable

Deprecated. Use edit_in_progress instead.

edits property

Get all Edits of the data node.

expiration_date: datetime property

Datetime instant of the expiration date of the data node.

is_ready_for_reading: bool property

Indicate if this data node is ready for reading.


Type Description

False if the data is locked for modification or if the data has never been written. True otherwise.

is_up_to_date property

Indicate if this data node is up-to-date.


Type Description

False if the data ever been written or the expiration date has passed. True otherwise.

job_ids property

List of the jobs having edited the data node.

last_edition_date property writable

Deprecated. Use last_edit_date instead.

parent_id property writable

Deprecated. Use owner_id instead.

parent_ids property

List of parent ids of the data node.

properties property

Dictionary of custom properties.

filter(operators, join_operator=JoinOperator.AND)

Read the data referenced by the data node, appying a filter.

The data is filtered by the provided list of 3-tuples (key, value, Operator).

If multiple filter operators are provided, filtered data will be joined based on the join operator (AND or OR).


Name Type Description Default
operators Union[List[Tuple], Tuple]


join_operator JoinOperator

The operator used to join the multiple filter 3-tuples.



Get last Edit of the data node, or None


Get all parents of the data node.


Lock the data node modification.


The data node can be unlocked with the method unlock_edit().


Deprecated. Use lock_edit instead.


Read the data referenced by this data node.


Type Description

The data referenced by this data node. None if the data has not been written yet.


Read the data referenced by this data node.


Type Description

The data referenced by this data node.


Type Description

If the data has not been written yet.

unlock_edit(at=None, job_id=None)

Unlocks the data node modification.


Name Type Description Default
at Optional[datetime]


job_id Optional[JobId]



The data node can be locked with the method lock_edit().

unlock_edition(at=None, job_id=None)

Deprecated. Use unlock_edit() instead.

write(data, job_id=None, **kwargs)

Write some data to this data node.


Name Type Description Default
data Any

The data to write to this data node.

job_id JobId

An optional identifier of the writer.


Extra information to attach to the edit document corresponding to this write.