DatasetService¶
Initialize the Elevate service.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
server
|
Optional[str]
|
The server name for the Elevate service API. If not provided, defaults to the |
None
|
client_id
|
Optional[str]
|
The client ID for authentication. If not provided, defaults to the |
None
|
secret_key
|
Optional[str]
|
The secret key for authentication. If not provided, defaults to the |
None
|
Raises:
| Type | Description |
|---|---|
ElevateConfigurationException
|
If any required parameter is missing or invalid |
add_dataset_node_to_transformation_graph
¶
Adds a new node (recipe or DataTable) to the specified transformation graph version. For DataTable nodes, the node references a specific DataTable and optionally a specific version. For recipe nodes, the node contains transformation logic and creates edges to its source nodes. Transformation recipes can only have one DataTable as a source, while structure recipes can have multiple sources. Nodes represent data processing steps or data sources within the transformation workflow. Returns only the newly added node.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
add_dataset_node_request
|
AddDatasetNodeRequest
|
Request body containing the configuration for adding a new node (recipe or DataTable) to a transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetNodeResponse
|
Returns the result object. |
create_dataset
¶
Creates a new Dataset that serves as a container for data transformation workflows. A Dataset can contain multiple versions, each representing a different configuration of the transformation graph with various DataTables and recipes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
create_dataset_request
|
CreateDatasetRequest
|
Request body containing the configuration details needed to create a new Dataset, including name, description, and initial metadata (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetResponse
|
Returns the result object. |
create_new_version_to_dataset
¶
Creates a new version of the transformation graph for the specified Dataset. This operation increments both the latest and max version numbers of the Dataset. If sourceTransformationGraphVersionNumber is provided, the new version will be cloned from that version. If sourceTransformationGraphVersionNumber is 0, an empty version is created (but only if the source version has nodes). This allows you to create different configurations and iterations of your data transformation workflow.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
create_dataset_version_request
|
CreateDatasetVersionRequest
|
Request body containing the configuration for creating a new version of a Dataset transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
TransformationGraphWrapperResponse
|
Returns the result object. |
delete_dataset
¶
Permanently deletes a Dataset and all its associated transformation graphs, versions, and metadata. This operation deletes the latest transformation graph and all its associated data. This operation cannot be undone and will remove all historical data processing configurations.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
Returns:
| Type | Description |
|---|---|
None
|
Returns the result object. |
delete_dataset_node_from_transformation_graph
¶
Deletes a specific node (recipe or dataTable) from the transformation graph version. This operation removes the node and all its associated edges from the graph.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
None
|
Returns the result object. |
delete_dataset_version
¶
Permanently deletes a specific version of the transformation graph. If the deleted version is the latest version, the Dataset's latest version number is decremented to the previous version. The last remaining version cannot be deleted. This operation cannot be undone and will remove all associated nodes, edges, and execution history for that version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
Returns:
| Type | Description |
|---|---|
None
|
Returns the result object. |
execute_recipe
¶
Executes a recipe node within the transformation graph. This triggers the data processing logic defined in the recipe and returns the execution status and job information.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
execute_recipe_request
|
ExecuteRecipeRequest
|
Request body containing the execution parameters for running a recipe node within a transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetNodeResponse
|
Returns the result object. |
export_dataset_node_to_s3
¶
Initiates an export of the processed data from the output node to Amazon S3. This operation validates that the recipe node has been successfully executed before allowing the export. The export type (sample or full data) must match the execution type of the recipe. Returns a job ID that can be used to track the export progress and retrieve results.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
export_s3_request
|
ExportS3Request
|
Request body containing S3 credentials and configuration details for exporting data to Amazon S3 (required) |
required |
Returns:
| Type | Description |
|---|---|
ExportS3Response
|
Returns the result object. |
export_dataset_version_to_s3
¶
Initiates an export of the processed data from the output node to Amazon S3. This operation validates that the recipe node has been successfully executed before allowing the export. The export type (sample or full data) must match the execution type of the recipe. Returns a job ID that can be used to track the export progress and retrieve results.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
export_s3_request
|
ExportS3Request
|
Request body containing S3 credentials and configuration details for exporting data to Amazon S3 (required) |
required |
Returns:
| Type | Description |
|---|---|
ExportS3Response
|
Returns the result object. |
get_dataset
¶
Retrieves detailed information about a specific Dataset including its latest transformation graph, version history, and metadata. Returns NOT_FOUND if the Dataset does not exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetResponse
|
Returns the result object. |
get_dataset_recipe_node_preview
¶
Retrieves a sample of the actual data content from a specific dataset recipe node for preview purposes. Returns a limited number of rows to allow users to inspect the data structure and content without downloading the entire dataset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
List[Dict[str, object]]
|
Returns the result object. |
get_latest_recipe_status
¶
Retrieves the current execution status of a recipe node from the most recent execution. Returns only the numeric status code representing the execution state (e.g., 0=PENDING, 1=IN_PROGRESS, 2=COMPLETED, 3=FAILED).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
RecipeStatusResponse
|
Returns the result object. |
get_recipe_node_column_analysis
¶
Retrieves schema analysis results for a single column of a recipe node execution result. Returns NOT_FOUND if the dataset, transformation graph, node, column, or schema analysis does not exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
column_name
|
StrictStr
|
Name of the column to retrieve analysis for (must exist in the recipe result schema) (required) |
required |
analysis_type
|
Optional[StrictStr]
|
Preferred analysis type |
None
|
Returns:
| Type | Description |
|---|---|
Dict[str, object]
|
Returns the result object. |
get_recipe_node_overview
¶
Retrieves schema overview statistics of a recipe node execution result. Returns NOT_FOUND if the dataset, transformation graph, node, or schema overview does not exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
analysis_type
|
Optional[StrictStr]
|
Preferred analysis type |
None
|
Returns:
| Type | Description |
|---|---|
Dict[str, object]
|
Returns the result object. |
get_recipe_node_schema_alerts
¶
Retrieves schema alerts of a recipe node execution result. Returns NOT_FOUND if the dataset, transformation graph, node, column, or schema alerts does not exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
analysis_type
|
Optional[StrictStr]
|
Preferred analysis type |
None
|
Returns:
| Type | Description |
|---|---|
List[Dict[str, object]]
|
Returns the result object. |
get_recipe_node_schema_analysis_status
¶
Retrieves schema analysis job status for a recipe node execution result. Returns NOT_FOUND if the dataset, transformation graph, node, column, or schema analysis does not exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
analysis_type
|
Optional[StrictStr]
|
Preferred analysis type |
None
|
Returns:
| Type | Description |
|---|---|
SchemaAnalysisStatusResponse
|
Returns the result object. |
get_transformation_graph_by_version
¶
Retrieves detailed information about a specific transformation graph version including its nodes, edges, and configuration. Returns NOT_FOUND if the Dataset or version does not exist.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
Returns:
| Type | Description |
|---|---|
TransformationGraphWrapperResponse
|
Returns the result object. |
is_dataset_name_available
¶
Validates whether a Dataset name is available for use within the current workspace. Returns true if the name can be used for creating a new Dataset, false if it already exists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
constr(strict=True, min_length=1)
|
The proposed name for the Dataset to check for availability (required) |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Returns the result object. |
is_recipe_node_name_available
¶
Validates whether a recipe node name is available for use within the specified Dataset context. This ensures that recipe names remain unique within the transformation graph scope. Returns true if the name can be used for creating a new recipe node, false if it already exists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
constr(strict=True, min_length=1)
|
The proposed name for the recipe node to check for availability (required) |
required |
Returns:
| Type | Description |
|---|---|
bool
|
Returns the result object. |
list_all_binding_candidates
¶
Returns all candidate nodes eligible for binding to a new node. These are the sink nodes (nodes with zero out-degree) in the specified transformation graph version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetNodesResponse
|
Returns the result object. |
list_binding_candidates
¶
Returns all candidate nodes eligible for binding to the specified node. Candidates are the union of the node's existing bound nodes and all sink nodes in the transformation graph, excluding the node reachable from the input node.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetNodesResponse
|
Returns the result object. |
list_datasets
¶
Retrieves a list of all Datasets. Datasets represent complex data transformation workflows that combine multiple DataTables and recipes to produce processed datasets through transformation graphs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
list_query
|
Optional[ListQuery]
|
List query parameters |
None
|
Returns:
| Type | Description |
|---|---|
DatasetsResponse
|
Returns the result object. |
list_transformation_graphs
¶
Retrieves all transformation graph versions for the specified Dataset. Each version represents a different configuration of the data transformation workflow.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
Returns:
| Type | Description |
|---|---|
List[LeanTransformationGraphResponse]
|
Returns the result object. |
release_transformation_graph
¶
Marks the transformation graph as released for production use. This operation validates that all leaf recipe nodes have successfully executed on the complete dataset (not sample data) before allowing the release. Table nodes without edges are skipped during validation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
Returns:
| Type | Description |
|---|---|
TransformationGraphWrapperResponse
|
Returns the result object. |
stop_recipe_execution
¶
stops the execution of a recipe node that is currently running. This operation allows you to cancel long-running or stuck recipe executions. It will not delete the node or its configuration, but will stop any ongoing processing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
Returns:
| Type | Description |
|---|---|
None
|
Returns the result object. |
update_dataset
¶
Updates the metadata of an existing Dataset such as name, description, and tags. This operation does not modify the transformation graph structure - use specific transformation graph operations for that purpose.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
update_dataset_request
|
UpdateDatasetRequest
|
Request body containing the updated configuration for an existing Dataset, including metadata changes such as name, description, and tags (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetResponse
|
Returns the result object. |
update_dataset_node_in_transformation_graph
¶
Updates the configuration of an existing node (recipe or dataTable) within the specified transformation graph version.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset_id
|
StrictInt
|
Unique identifier of the Dataset to operate on (required) |
required |
transformation_graph_version_number
|
StrictInt
|
Version number of the transformation graph within the Dataset (required) |
required |
dataset_node_id
|
StrictInt
|
Unique identifier of the node within the transformation graph (required) |
required |
update_dataset_node_request
|
UpdateDatasetNodeRequest
|
Request body containing the updated configuration for an existing recipe node within a transformation graph. Only recipe nodes can be updated through this endpoint (required) |
required |
Returns:
| Type | Description |
|---|---|
DatasetNodeResponse
|
Returns the result object. |