Manage workflows

Prerequisites

• You have created a workspace. For more information, see Manage workspaces.

• You have developed and published the required tasks.

Create a workflow

1. Go to the Workflows page.

   1. Log on to the E-MapReduce console.

   2. In the left-side navigation pane, choose EMR Serverless > Spark.

   3. On the Spark page, click the name of the target workspace.

   4. On the EMR Serverless Spark page, click Workflows in the left-side navigation pane.

2. On the Workflows page, click Create Workflow.

3. In the Create Workflow panel, configure the following parameters and click Next.

   Parameter		Description
   Name		The workflow name must be unique within the current workspace.
   Resource Queue		Select the default resource queue in which the workflow runs. Note A node's resource queue setting overrides this default queue.
   Other Settings	Scheduling Type	The scheduling type for running the workflow in the production environment: Manual (Default): The workflow runs only when you trigger it manually. Scheduler: The workflow runs automatically on a defined schedule (minute, hour, or day). When you set Scheduling Type to Scheduler, you must also configure the Scheduler and Scheduling Started At parameters.
   	Scheduler	How often the workflow runs automatically. Required only when you set Scheduling Type to Scheduler. The following scheduling cycles are available: Daily: Runs once at a specified time each day. Hourly: Runs at a defined interval, such as every N hours, within a specified time range each day. Minute: Runs at a defined interval, such as every N minutes, within a specified time range each day.
   	Scheduling Started At	The date and time when the scheduled workflow starts. Defaults to the current time. Required only for the Scheduler type. Important For a workflow of the Scheduler type, you must enable the Scheduling Status toggle on the Workflows page to activate its schedule.
   	Retries After Failure	The number of times a node retries after a failure. Default: no retries. Note An individual node's settings can override this parameter.
   	Failure Notification	The email address to receive failure notifications.
   	Tags	Key-value pairs to tag the workflow.

4. Edit the workflow nodes.

   1. On the Edit Workflow page, click Add Node at the bottom.

   2. In the Add Node panel, configure the node parameters.

      Parameter	Description
      Path	The path to the task that this node runs. The task must be published.
      Node Type	The workflow node type. By default, the system infers the type from the task at the specified path.
      Node Name	A custom name for the node. Automatically populated from the task source.
      Upstream Nodes	The preceding node in the workflow. The first node does not require an upstream node.
      Failure Retries	Inherits the retry settings from the workflow. Default: no retries.
      Timeout(seconds)	The timeout for a single node run. Default: no limit.
      Subscriptions	Specify an email address to subscribe to notifications for specific node states.
      Tags	Tag key-value pairs for the node. Each node automatically includes two built-in tags: workflow_name and task_name.
      Resource Queue	The resource queue for this node. This setting overrides the workflow's default resource queue. Important A node-specific resource queue setting takes precedence over the workflow's default, even if the default is changed later.

      Note

      If the task source is SQL development, you must also configure task parameters. These parameters inherit values from the task template by default. You can modify the task template to change the default values. For more information about parameters, see Configuration management.

   3. Click Save.

      After configuring the initial node, you can click Add Node at the bottom of the page to add more nodes.

5. Publish the workflow.

   1. In the upper-right corner, click Publish Workflow.

   2. In the Publish dialog box, you can enter a description for the release, and then click OK.

Run a workflow

Each workflow run generates a record on the Workflow Runs tab of the workflow details page.

View run status

View the status of all workflow runs and node runs in the Workflow Runs Status and Workflow Job Runs Status columns for the target workflow.

• Workflow run status

  Status	Description
  Blue	Running
  Green	Succeeded
  Red	Failed
  Purple	Pending

• Workflow node run status

  Status	Description
  Blue	Running
  Green	Succeeded
  Red	Failed
  Yellow	Retrying
  Purple	Pending

Related topics

• For more information about key workflow concepts, see Basic concepts.

• For information about viewing workflow and node runs, see Manage workflow runs and node runs.