Skip to content


Bases: Section

Configuration fields needed to instantiate a DataNode.

A Data Node config is made to be used as a generator for actual data nodes. It holds configuration information needed to create an actual data node.


Name Type Description
id str

Unique identifier of the data node config. It must be a valid Python variable name.

storage_type str

Storage type of the data nodes created from the data node config. The possible values are : "csv", "excel", "pickle", "sql_table", "sql", "mongo_collection", "generic", "json", "parquet" and "in_memory". The default value is "pickle". Note that the "in_memory" value can only be used when JobConfig.mode is "standalone".

scope Optional[Scope]

The optional Scope of the data nodes instantiated from the data node config. The default value is SCENARIO.

**properties dict[str, any]

A dictionary of additional properties.