Input and output parameters are used to transmit parameter settings between ancestor and descendant nodes. This topic describes how to define and use input and output parameters.
Background information
After you define an output parameter and its value for a node, you can define an input parameter for the descendant node and configure it to reference the value of the output parameter.
Precautions
Description of parameter transmission and query result transmission:
Parameter passing: To pass parameter information from an upstream node to a downstream node, you can set the Output Parameter in the Node Context Parameters section of the upstream node as the Input Parameter in the Node Context Parameters section of the downstream node.
Pass query results: To pass query results from an upstream node to a downstream node, you can set the Assignment Parameter in the Node Context Parameters section of the upstream node as an Input Parameter in the Node Context Parameters section of the downstream node. You can also use an assignment node to pass the query results.
NotePassing query results is supported only for specific nodes, such as EMR Hive, EMR Spark SQL, ODPS Script, Hologres SQL, AnalyticDB for PostgreSQL, Click House SQL, and MySQL nodes. This feature functions similarly to an assignment node. For configuration instructions, see Assignment node.
Limits
The Use Assignment Parameters feature is supported only in DataWorks Standard Edition or more advanced editions.
Access the parameter configuration section
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
In the Data Development pane, double-click the desired node to open the edit page.
Click Scheduling Configuration on the right, and in the Node Context Parameters section, click the
icon on the right. This opens the configuration section for node context parameters.NoteFor more information about how to configure scheduling parameters, see SQL-type nodes and offline synchronization nodes.
Configure output parameters
When you configure output parameters for the current node in the Node Context Parameters section, you can set them as needed.
Configure standard output parameters: You can pass the constant or variable output parameters that are configured for the current node to a downstream node.
Configure assignment output parameters: You can configure the query result of the current node as an output parameter and pass it to a downstream node for use.
Configure regular output parameters
If you want to transmit parameter information configured in an ancestor node to a descendant node, refer to the following content.
Parameter | Configuration description |
No. | The serial number, which is automatically increased by 1 each time you define an input parameter. |
Parameter Name | You need to customize the output parameter name. |
Type | Supports Constant and Variable. You can select and configure them as needed. |
Value | The value of the output parameter. The value type can be Constant or Variable:
|
Description | The description of the parameter. |
Source | The way in which the parameter is defined. Valid values: Added Automatically, Auto Parse, and Added Manually. |
Actions | Provides Save, Edit, and Delete operations. Note Editing and deleting are not supported when downstream nodes depend on this node. Before you configure a downstream node to reference an upstream node, carefully check to ensure that the upstream output is correctly defined. |
After you configure the relevant parameters, click Save in the Actions column.
Configure assignment output parameters
If you want to pass the query results of the current task as parameters to downstream tasks, refer to the following content.
Locate the section of the node.
Click Add Assignment Parameter to add the output assignment parameter.
After you click Add Assignment Parameter, the assignment parameter passes the query results of the current node to a downstream node that references this parameter. If the query returns an empty result, the current node is not blocked, but the downstream node that references the parameter may fail.
The downstream node must add the assignment parameter of the upstream node to its input parameters. The parameter is used in the same way as an assignment node that uses ODPS SQL. For more information, see Assignment node.
After you configure the Output Parameters For This Node, you can find the Output Name for which the Add Method is Added By System in the section. A downstream node can use this Output Name to create a dependency on the current node.
Configure input parameters
You can configure input parameters in the Input and Output Parameters section to obtain output parameters from upstream nodes for use in the current node.
You can define an input parameter for a node to reference an output parameter defined for its ancestor node. You can use the input parameter in the code of the node in the same way as the assignment of scheduling parameters to variables in the node code.
After you configure scheduling dependencies to allow the current node to depend on a node, the current node can reference an output parameter defined for its ancestor node as an input parameter for the current node in the Input Parameters section.
Configure output parameters for the ancestor node.
Configure dependencies.
Click Scheduling Configuration on the right side of the configuration page for the downstream node.
In the text box for Node Output under , enter and select the Output Name that you specified for the upstream node in Configure output parameters.
Click Add.
Configure input parameters for the downstream node.
In the Node Context Parameters section, click Add to add an input parameter for the current node. For information about the parameters, see the following table:
Parameter
Configuration description
No.
The serial number, which is automatically increased by 1 each time you define an input parameter.
Parameter Name
You need to customize the output parameter name. Later, you can reference the parameter information passed from upstream nodes through this parameter name on the node editing page.
Value Source
The parameter value comes from the output parameter of the upstream node, and the specific value is the actual output result of the upstream node.
Description
You can customize a brief description of the parameter.
Parent Node ID
The system parses the ID of the ancestor node.
Source
The way in which the parameter is defined.
Valid values: Added Automatically, Auto Parse, and Added Manually.
Actions
Provides Save, Edit, and Delete operations.
After you configure the relevant parameters, click Save in the Actions column.
Use the input parameter
You can use a defined input parameter in a node in the same way as other built-in scheduling parameters, by referencing it in the ${Input parameter name} format. The following example shows how to reference the parameter in a Shell node.
You must run the workflow to which the ancestor node and the current node belong to ensure that the ancestor node and the current node are run in sequence. Otherwise, the input parameter that is defined for the current node is invalid.
Supported built-in scheduling parameters
System variable
System variable
Description
${projectId}
The project ID.
${projectName}
The name of the MaxCompute project.
${nodeId}
The node ID.
${gmtdate}
The time 00:00:00 on the day when the instance is scheduled to run, in the yyyy-MM-dd 00:00:00 format.
${taskId}
The ID of the task instance.
${seq}
The sequence number of the task instance, which indicates the ranking of this instance among all instances of the same node on the same day.
${cyctime}
The time at which the node instance is scheduled to run.
${status}
The status of the instance. Valid values: SUCCESS and FAILURE.
${bizdate}
The data timestamp.
${finishTime}
The time at which the node instance finishes running.
${taskType}
The mode in which the instance is run. Valid values: NORMAL, MANUAL, PAUSE, SKIP, UNCHOOSE, and SKIP_CYCLE.
${nodeName}
The name of the node.
For more information about other parameters, see Supported formats of scheduling parameters.