端云联调简介

在本地调试函数时,您可以使用s proxied的相关命令实现本地函数资源和线上一些云产品的相互调用,即端云联调,协助您提高开发效率。本文介绍端云联调的基本原理,并以调试访问RDS MySQL和NAS为例,介绍如何实现端云联调以及在端云联调下如何实现断点调试。

使用说明

端云联调功能目前仅支持x86_64架构,不支持ARM64架构。

基本原理

在Serverless领域内,调试一直是开发者的痛点。虽然某些云厂商也提供了一些工具解决该问题,但由于这些工具都集中于模拟本地执行环境和参数阶段,无法实现本地环境和线上环境的连通。为了实现本地执行环境和线上的连通,Serverless Devs为您提供了端云联调功能。

image

如上图所示,Serverless Devs会使用s.yaml文件内的ServiceConfig参数信息(例如VPC配置和NAS配置)创建辅助函数,从而实现辅助函数(C)与被调试的函数是一样的服务配置和网络环境。同时,当端云联调通道建立成功后,可以实现以下需求。

  • 本地函数执行环境容器可以直接访问公有云线上云服务,例如:

    • VPC私网,例如NAS、RDS或Kafka私网地址。

    • Internal内网,例如OSS的Internal Endpoint等。

  • 执行入流量(例如contextevent参数信息)为来自线上的真实流量。

    您可以直接使用SDK(例如端云联调的调用命令)或触发器调用辅助函数(C),请求流量将被打回到本地的调试实例(A),即本地函数执行环境容器。此时,执行入流量为来自线上的真实流量。

说明

上图中的A、B和C均已在工具层面被封装好,您只需将代码和s.yaml文件内的资源按需配置即可。代码和s.yaml文件的资源配置需满足以下信息:

  • 代码需被挂载到本地开发环境(A)即本地函数执行环境容器内。

  • 集成开发环境和本地函数执行环境容器之间的端口映射可以通过--debug-port参数指定。

代码示例

下文介绍的示例代码以Python 3为例。当您的代码需要使用私网访问RDS MySQL和NAS时,对应的函数代码和s.yaml文件的配置信息示例如下所示。

代码示例

s.yaml示例

# -*- coding: utf-8 -*-
import logging
import os
import pymysql
import pymysql.cursors

def handler(event, context):
    print("\n***test fc internal endpoint***")
    os.system("curl --connect-timeout 3 1.cn-hangzhou-internal.fc.aliyuncs.com")

    print("\n***test db internal addr ***")
    test_mysql()

    print("\n***test nas ***")
    os.system("ls -lh/mns/auto")

def test_mysql():
    # RDS在VPC的私网地址。
    connection = pymysql.connect(host='rm-xxxxxxx.mysql.rds.aliyuncs.com',
                                 user='userName',
                                 password='userPwd',
                                 db='testDB',
                                 port=3306,
                                 charset='utf8')
    try:
        with connection.cursor() as cursor:
            sql = "select * from users"
            cout = cursor.execute(sql)
            print("total:"+str(cout))

            for row in cursor.fetchall():
                print("ID:"+str(row[0])+'  Name:'+row[1])
            connection.commit()

    finally:
        connection.close()                
edition: 1.0.0          #  命令行YAML规范版本,遵循语义化版本(Semantic Versioning)规范
name: fcDeployApp       #  项目名称
access: "default"  #  密钥别名

services:
  fc-deploy-test: #  服务名称
    component: devsapp/fc  # 组件名称
    props: #  组件的属性值
      region: cn-hangzhou
      service:
        name: fc-deploy-service
        description: 'demo for proxied invoke'
        internetAccess: true
        vpcconfig:
            vpcId: vpc-bp1wv9al02opqahi****
            securityGroupId: sg-bp1h2swzeb5vgjfu****
            vswitchIds:
                - vsw-bp1kablus0jrcdeth****
        nasconfig:
            userId:10008
            groupId:10008
            mountPoints:
                - serverAddr: example.com
                nasDir: /
                fcDir: /mnt/auto
      function:
        name: event-function
        description: this is a test
        runtime: python3
        codeUri: .
        handler: index.handler
        memorySize: 128
        timeout: 60
说明

当您执行完准备端云联调需要的辅助资源和本地环境的相关命令,即成功建立端云联调的通道后,您就可以在本地直接调试函数,即代码中可以直接使用私网访问RDS MySQL VPC和NAS。

前提条件

您已完成以下操作。

操作步骤

  1. 在项目目录中,按需选择以下命令,准备端云联调需要的辅助资源和本地环境。

    • 当您的s.yaml内只有一个项目或s.yaml内包含多个项目时,无需指定某个项目,请执行以下命令。

      s proxied setup
    • 当您的s.yaml内包含多个项目时,需要指定某个项目进行测试时,请执行以下命令。

      s <projectName> proxied setup

    输出示例。

    [2021-10-08T15:55:16.653] [INFO ] [S-CLI] - Start ...
      Session created, session id: S-d847ae28-767f-4532-a2f2-307ee2b99c5f.
    ......
    [2021-10-08T15:55:24.580] [INFO ] [FC-DEPLOY] - Checking Trigger httpTrigger exists
    [2021-10-08T15:55:24.807] [INFO ] [FC-DEPLOY] - Checking Trigger httpTrigger exists
      Make service SESSION-S-d847a success.//辅助服务部署成功。
      Make function SESSION-S-d847a/http-trigger-py36 success.//辅助函数部署成功。
      Make trigger SESSION-S-d847a/http-trigger-py36/httpTrigger success.//辅助触发器部署成功。
    ......
    There is auto config in the service: SESSION-S-d847a
      Helper function is set to 1 provison and 0 elasticity.//函数计算(C)容器启动完成。
      Proxy container is running.//本地环境(A)中的代码容器启动完成。
      Session established!//Session建立成功。
    [2021-10-08T15:56:13.251] [INFO ] [FC-PROXIED-INVOKE] - Pulling image registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-python3.6:1.9.19, you can also use 'docker pull registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-python3.6:1.9.19' to pull image by yourself.
    1.9.19: Pulling from aliyunfc/runtime-python3.6
    ......
    Digest: sha256:6a4da97962dba5f6cb00c5e8e83024c4758ec358e5bf884efff897b9826d9454
    Status: Downloaded newer image for registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-python3.6:1.9.19
    [2021-10-08T15:56:15.694] [INFO ] [FC-PROXIED-INVOKE] - Checking Server in function container is up. exists
    FunctionCompute python3 runtime inited.
    FC Invoke End RequestId: unknown_request_id, Error: Unhandled function error
    [2021-10-08T15:56:16.831] [INFO ] [FC-PROXIED-INVOKE] - Server in function container is up!//本地环境(A)中函数计算的执行环境启动成功。
    End of method: proxied
    说明

    成功执行准备命令后,项目将会被阻塞在此处等待被调用,执行环境实际是一个HTTP Server。

  2. 本地调用。

    • HTTP函数

      • 当您的函数是匿名的HTTP触发器函数时,您可以使用cURL工具或Postman调用生成的临时域名实现本地调试。本文以cURL为例。

        curl -v http://http-trigger-py36.SESSION-S-d847a.188077086902****.cn-hangzhou.fc.devsapp.net
      • 当HTTP触发器的authType是function时,请按照以下步骤实现HTTP函数的调试。

        1. 启动一个新的终端。

        2. 执行相关命令,切换到项目目录内。

        3. 执行以下命令,调用函数。

          s proxied invoke -e '{"body":123,"method":"GET","headers":{"key":"value"},"queries":{"key":"value"},"path":"string"}'
    • Event函数

      • 有触发器的事件函数

        本示例以对象存储OSS触发器为例,介绍如何实现本地调用。

        1. 确认触发函数执行的方式,例如OSS触发器、CDN事件触发器等。

        2. 登录函数计算控制台

        3. 找到辅助函数,然后给该函数创建触发器。本示例中关于如何创建OSS触发器的详细操作,请参见步骤一:创建OSS触发器

        4. 登录OSS管理控制台,实际触发相应的事件。例如上传或下载文件,即可调用本地的函数计算执行环境。

      • 无触发器的事件函数

        请按照以下步骤实现无触发器普通事件函数的调用。

        1. 启动一个新的终端。

        2. 执行相关命令,切换到项目目录内。

        3. 执行以下命令,调用函数。

          s proxied invoke -e '{"key":"value"}'
        重要

        当您的函数是有触发器的事件函数时,也可以使用该方式实现本地调用。

  3. 执行以下命令,清理端云联调需要的辅助资源和本地环境。

    s proxied cleanup

    输出示例。

      Stop container succeed.
      Unset helper function provision and on-demand config done.
    [2021-10-08T16:41:19.960] [INFO ] [FC-DEPLOY] - Using region: cn-hangzhou
    [2021-10-08T16:41:19.988] [INFO ] [FC-DEPLOY] - Using access alias: default
    [2021-10-08T16:41:19.989] [INFO ] [FC-DEPLOY] - Using accessKeyID: LTAI4G4cwJkK4Rza6xd9****
    [2021-10-08T16:41:19.993] [INFO ] [FC-DEPLOY] - Using accessKeySecret: eCc0GxSpzfq1DVspnqqd6nmYNN****
    [2021-10-08T16:41:20.104] [INFO ] [FC-DEPLOY] - Checking Service SESSION-S-a9143 exists
    [2021-10-08T16:41:20.397] [INFO ] [FC-DEPLOY] - Service: SESSION-S-a9143 already exists online.
    [2021-10-08T16:41:20.400] [INFO ] [FC-DEPLOY] - Checking Function http-trigger-py36 exists
    [2021-10-08T16:41:20.583] [INFO ] [FC-DEPLOY] - Function: http-trigger-py36 already exists online.
    [2021-10-08T16:41:20.586] [INFO ] [FC-DEPLOY] - Checking Trigger httpTrigger exists
    [2021-10-08T16:41:20.820] [INFO ] [FC-DEPLOY] - Trigger: httpTrigger already exists online.
      Using fc deploy type: sdk, If you want to deploy with pulumi, you can [s cli fc-default set deploy-type pulumi] to switch.
      Delete trigger SESSION-S-a9143/http-trigger-py36/httpTrigger success.
      [TriggerNotFound], DELETE /services/SESSION-S-a9143/functions/http-trigger-py36/triggers/httpTrigger failed with 404. requestid: 9de742e1-d8f6-4db0-a4bd-02b10542147e, message: trigger 'httpTrigger' does not exist in service 'SESSION-S-a9143' and function 'http-trigger-py36'.
      Delete function SESSION-S-a9143/http-trigger-py36 success.
      Delete service SESSION-S-a9143 success.
      Delete session: S-a9143b1a-88ff-48bd-bd79-2f3755397fe2 done.
      Stop container succeed.
    End of method: proxied
    说明

    执行完清理工作后,在准备端云联调的辅助资源和本地环境形成的阻塞也将退出。

断点调试

如何使用VSCode实现断点调试

下文以Python 3为例,介绍在端云联调的过程中实现断点调试:

  1. 在项目目录中执行以下命令,准备端云联调需要的辅助资源和本地环境。

    s proxied setup --config vscode --debug-port 3000 

    输出示例:

    [2021-10-08T15:55:16.653] [INFO ] [S-CLI] - Start ...
      Session created, session id: S-d847ae28-767f-4532-a2f2-307ee2b99c5f.
    [2021-10-08T15:55:18.395] [INFO ] [FC-PROXIED-INVOKE] - Deploying helper function...
    [2021-10-08T15:55:18.397] [INFO ] [FC-PROXIED-INVOKE] - Creating cleaner service...
    [2021-10-08T15:55:22.021] [INFO ] [FC-DEPLOY] - Using region: cn-hangzhou
    [2021-10-08T15:55:22.022] [INFO ] [FC-DEPLOY] - Using access alias: default
    [2021-10-08T15:55:22.022] [INFO ] [FC-DEPLOY] - Using accessKeyID: LTAI4G4cwJkK4Rza6xd9****
    [2021-10-08T15:55:22.022] [INFO ] [FC-DEPLOY] - Using accessKeySecret: eCc0GxSpzfq1DVspnqqd6nmYNN****
     Using fc deploy type: sdk, If you want to deploy with pulumi, you can [s cli fc-default set deploy-type pulumi] to switch.
    [2021-10-08T15:55:22.971] [INFO ] [FC-DEPLOY] - Checking Service SESSION-S-d847a exists
    [2021-10-08T15:55:23.293] [INFO ] [FC-DEPLOY] - Setting role: AliyunFCDefaultRole
    [2021-10-08T15:55:23.886] [INFO ] [RAM] - Checking Role AliyunFCDefaultRole exists
    [2021-10-08T15:55:24.099] [INFO ] [RAM] - Updating role: AliyunFCDefaultRole
    [2021-10-08T15:55:24.191] [INFO ] [RAM] - Checking Plicy AliyunFCDefaultRolePolicy exists
    [2021-10-08T15:55:24.300] [INFO ] [FC-DEPLOY] - Checking Function http-trigger-py36 exists
    [2021-10-08T15:55:24.577] [WARN ] [FC-DEPLOY] - Image registry.cn-hangzhou.aliyuncs.com/aliyunfc/ts-remote:v0.1.1 dose not exist locally.
    Maybe you need to run 's build' first if it dose not exist remotely.
    [2021-10-08T15:55:24.580] [INFO ] [FC-DEPLOY] - Checking Trigger httpTrigger exists
    [2021-10-08T15:55:24.807] [INFO ] [FC-DEPLOY] - Checking Trigger httpTrigger exists
      Make service SESSION-S-d847a success.//辅助服务部署成功。
      Make function SESSION-S-d847a/http-trigger-py36 success.//辅助函数部署成功。
      Make trigger SESSION-S-d847a/http-trigger-py36/httpTrigger success.//辅助触发器部署成功。
    [2021-10-08T15:55:26.129] [INFO ] [FC-DEPLOY] - Checking Service SESSION-S-d847a exists
    [2021-10-08T15:55:26.568] [INFO ] [FC-DEPLOY] - Checking Function http-trigger-py36 exists
    [2021-10-08T15:55:26.922] [INFO ] [FC-DEPLOY] - Checking Trigger httpTrigger exists
    [2021-10-08T15:55:27.542] [INFO ] [FC-DEPLOY] - Using customDomain: auto: fc will try to generate related custom domain resources automatically
      End of request
      Deployed.
      End of request
    [2021-10-08T15:55:35.499] [INFO ] [FC-DEPLOY] - Generated auto custom domain: http-trigger-py36.session-s-d847a.188077086902****.cn-hangzhou.fc.devsapp.net
    [2021-10-08T15:55:35.499] [INFO ] [FC-DEPLOY] - Creating custom domain: http-trigger-py36.session-s-d847a.188077086902****.cn-hangzhou.fc.devsapp.net
    [2021-10-08T15:55:35.679] [INFO ] [FC-DOMAIN] - Creating custom domain: http-trigger-py36.session-s-d847a.188077086902****.cn-hangzhou.fc.devsapp.net
    
    There is auto config in the service: SESSION-S-d847a
      Helper function is set to 1 provison and 0 elasticity.//函数计算(C)容器启动完成。
      Proxy container is running.//本地环境(A)中的代码容器启动完成。
      Session established!//Session建立成功。
    [2021-10-08T15:56:13.251] [INFO ] [FC-PROXIED-INVOKE] - Pulling image registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-python3.6:1.9.19, you can also use 'docker pull registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-python3.6:1.9.19' to pull image by yourself.
    1.9.19: Pulling from aliyunfc/runtime-python3.6
    ......
    Digest: sha256:6a4da97962dba5f6cb00c5e8e83024c4758ec358e5bf884efff897b9826d9454
    Status: Downloaded newer image for registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-python3.6:1.9.19
    [2021-10-08T15:56:15.694] [INFO ] [FC-PROXIED-INVOKE] - Checking Server in function container is up. exists
    FunctionCompute python3 runtime inited.
    FC Invoke End RequestId: unknown_request_id, Error: Unhandled function error
    [2021-10-08T15:56:16.831] [INFO ] [FC-PROXIED-INVOKE] - Server in function container is up!//本地环境(A)中函数计算的执行环境启动成功。
    End of method: proxied

    成功执行准备命令后,项目将会被阻塞在此处等待被调用,若直接执行调用命令,实现的将是正常模式的本地调用流程。如果需要断点调试,则需在VSCode编辑器的侧边栏为函数代码增加断点(图示①),然后单击调试图标(图示②)。image.png

  2. 打开一个新的终端,执行以下命令调用函数。

    s proxied invoke

    执行完调试命令后,您可以重新回到VSCode的界面,此时函数已经开始断点调试。

  3. 执行以下命令,清理辅助资源、Session和本地调试容器。

    s proxied clean

如何使用IntelliJ IDEA实现断点调试

下文以Java 8为例,介绍如何使用IntelliJ IDEA实现断点调试:

  1. 在项目目录中,执行以下命令,安装依赖:

    s build --use-docker

    输出示例:

    [2021-10-09T15:28:39.391] [INFO ] [S-CLI] - Start ...
    [2021-10-09T15:28:40.520] [INFO ] [FC-BUILD] - Build artifact start...
    [2021-10-09T15:28:40.548] [INFO ] [FC-BUILD] - Use docker for building.
    [2021-10-09T15:28:40.867] [INFO ] [FC-BUILD] - Build function using image: registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-java8:build-1.9.21
    [2021-10-09T15:28:41.411] [INFO ] [FC-BUILD] - begin pulling image registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-java8:build-1.9.21, you can also use docker pull registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-java8:build-1.9.21 to pull image by yourself.
    build-1.9.21: Pulling from aliyunfc/runtime-java8
    ......
    Digest: sha256:35eb46f6235729dbcb1fa48f4ce4ae7b213b967a0f1b9c625e464dbba58af22d
    Status: Downloaded newer image for registry.cn-beijing.aliyuncs.com/aliyunfc/runtime-java8:build-1.9.21
    [2021-10-09T15:30:52.690] [INFO ] [FC-BUILD] - Build artifact successfully.
    
    Tips for next step
    ======================
    * Invoke Event Function: s local invoke
    * Invoke Http Function: s local start
    * Deploy Resources: s deploy
    End of method: build
  2. 执行以下命令,准备端云联调需要的辅助资源和本地环境。

    s proxied setup --debug-port 3000 

    成功执行准备命令后,项目将会被阻塞在此处等待被调用,若直接执行调用命令,实现的将是正常模式的本地调用流程。如果需要断点调试,则需在首次调试时调试配置IntelliJ IDEA:

    1. 打开IntelliJ IDEA,然后打开对应的项目目录,在IDEA的顶部菜单栏选择Run>Debug Configurations...

    2. 在Run/Debug Configurations对话框中,单击加号图标,选择Remote设置相关参数:

      • Name:自定义调试器名称。

      • Port:设置端口为3000。当您在准备端云联调时--debug-port参数指定为其他端口时,Port也需做相应修改。

    3. 单击ok,完成IDEA配置。

  3. 在IDEA编辑器的侧边栏为函数代码增加断点(图示①),然后单击调试图标(图示②)。image.png

  4. 打开一个新的终端,执行以下命令调用函数:

    s proxied invoke

    执行完调试命令后,您可以重新回到IDEA的界面,此时函数已经开始断点调试。

  5. 执行以下命令,清理辅助资源、Session和本地调试容器。

    s proxied clean