Develop custom steps using Flow-CLI

Background information

• Flow-CLI is a command-line interface (CLI) for Apsara DevOps Flow. You can use it to manage various resources in Flow. The `step` subcommand lets you run custom step logic in a pipeline to extend pipeline capabilities.

• In Flow, a pipeline task run consists of multiple steps. A step is the smallest runnable unit and consists of a descriptive YAML file and a TypeScript code package. The YAML file defines the input parameters for the step at runtime, and the TypeScript code defines the execution logic for the step. This document guides you through the process of creating a custom step in Flow.

Procedure

npm i -g @flow-step/flow-cli --registry=https://registry.npmmirror.com
flow-cli -h  # The help information is displayed, which indicates that the installation is successful.

flow-cli login    # Log on

flow-cli login <instance_URL>
# For example: flow-cli login https://xxx-cn-shenzhen.devops.aliyuncs.com/

Create a new step

flow-cli step init

Follow the prompts to enter the basic information for the step. Note: The Step Sign must be unique within Apsara DevOps. If the sign is already used by another step, the creation fails.

This command creates a local folder named after the Step Sign. In this example, the folder is HelloV2StepWorld. The folder contains the following files:

.
├── README.md     # Step description
├── package.json  # TypeScript project package declaration
├── src           # Step execution logic
│   ├── index.ts
│   ├── params.ts
│   └── util.ts
├── step.yaml     # A YAML file that describes all the information about the step
├── test          # Step unit tests
│   └── index.spec.ts
├── tsconfig.eslint.json
└── tsconfig.json

• step.yaml: Defines the input parameters required for step execution. These are the configuration parameters required for the step when you configure it in the pipeline.

• src/index.ts: The entry source code file that is executed at runtime. In this file, you can retrieve environment variables from the pipeline and parameters configured in step.yaml, and then define your execution logic.

The following example describes the functions of these two key files in detail.

1. The step.yaml file defines the description of the step:

   ---
   id: HelloV2StepWorld                      # Unique identifier of the step
   name: HelloV2StepWorld                    # Name of the step
   description: My awesome flow step :-)     # Description of the step
   helpUrl: https://atomgit.com/flow-step-custom/HelloV2StepWorld  # Help link for the step
   execution:
     executor: node
     main: dist/index.js
   items:                                    # Input parameter form
     - label: input parameter
       name: abc
       type: input
       description: An input parameter of the input type named abc
       value: hello world
     - label: password parameter
       name: your_password
       type: password
       description: A password parameter of the password type named your_password
     - label: textarea parameter
       name: exclusion
       type: textarea
       value: |
         test/
         build/
       description: A textarea parameter of the textarea type named exclusion
     - label: select parameter
       name: select_version
       type: select
       value: v1.0.0
       datasource:
         - label: "v1.0.0"
           value: v1.0.0
         - label: "v1.0.1"
           value: v1.0.1
         - label: "v1.0.2"
           value: v1.0.2
       description: A select parameter of the select type named select_version
     - label: Parameter edf
       name: edf
       type: input
       showCondition: ${select_version} == "v1.0.0"
       description: Configure showCondition. This parameter is displayed only when select_version is v1.0.0.
     - label: Switch
       name: toggle
       type: checkbox
       value: false

   The `items` section defines the input form for user configuration on the pipeline editing page. It supports various frontend components, such as `input` and `checkbox`. The example provides configurations for common components. For more information about the syntax, see Step YAML description language.

2. src/index.ts contains the execution logic of the step. When the step runs in a pipeline, it can retrieve system variables from the pipeline context and input parameters from users on the editing page. The script lists some of the available environment variables:

   async function runStep(): Promise<void> {
     const params = getParams()
     step.info(`PIPELINE_ID=${params.pipelineID}`)
     step.info(`PIPELINE_NAME=${params.pipelineName}`)
     step.info(`BUILD_NUMBER=${params.buildNumber}`)
     step.info(`WORK_SPACE=${params.workSpace}`)
     step.info(`PROJECT_DIR=${params.projectDir}`)
     step.info(`BUILD_JOB_ID=${params.buildJobID}`)
   
     const abc = process.env.abc ? process.env.abc as string : ''
     step.info(`Hello from flow-step HelloV2StepWorld, custom param abc=${abc}, upper value=${convertToUpperCase(abc)}`)
   
     // ...
   }

Next, run the installation and build commands for the new step. Run the commands that are displayed in the command prompt after initialization. In this example, the command is cd HelloV2StepWorld && npm install && npm run build. This command performs three operations:

1. cd HelloV2StepWorld: Switches to the folder of the new step.

2. npm install : Installs the dependencies for the step.

3. npm run build: Builds the step locally.

After the command is executed, a `dist` folder is created. The dist/index.js file is the file that the Flow step actually executes.

After you push the configuration of the custom step, the pipeline downloads the step from the step package storage service when it executes your custom step. Then, the pipeline executes the script file that you defined in the default `alinux3` container environment. You can customize the environment to use your own image in the YAML pipeline.

Publishing steps

Run one of the following commands based on your current directory:

flow-cli step publish # Publish the step

flow-cli step publish ${your-step-folder} # Publish the step

This command performs two actions:

1. Builds and packages the local TypeScript project, and then pushes it to the remote repository.

2. Publishes the step source information described in step.yaml to the Apsara DevOps server.

After the step is published, enter the step version. In this example, the version is v1.0.0.

Run flow-cli step list to view the published steps.

Usage steps

Go to the Apsara DevOps Flow pipeline page and create a pipeline from an empty template. In the stage, add a task, add a step, and then select Organization Step to find the step that you created. Note: You must select a container environment. Otherwise, you cannot use the new version of custom steps.

After you add the step, configure its content. Click Save and Run. The step starts to run in the pipeline. You can view the logs to confirm that the TypeScript logic was executed.

Update steps

To update the step, modify the step.yaml file and the TypeScript code logic. Then, run flow-cli step publish again. Flow-CLI repackages the code and uploads the step.

If you go to the pipeline editing page and find that the step parameter settings have not changed, try refreshing the cache.

Deletion steps

To delete a published step, run flow-cli step remove "your_step_sign". Replace your_step_sign with the Step Sign of the step that you want to delete.

Other operations

Run the following command to view other commands that Flow-CLI supports.

FAQ

{"userToken":{"access_token":"ACCESS_TOKEN","token_type":"Bearer"},"organization":{"error":"","message":"","id":"","createdAt":"","updatedAt":"","creatorId":"","name":"","logo":"","location":"","category":"","description":"","website":"","background":"","contactPhone":"","desiredMemberCount":"","isDeleted":"","py":"","pinyin":"","isPublic":false,"defaultRoleId":"","defaultOrgRoleId":"","okrProgressMode":"","defaultTeamId":"","openId":"","source":""}}