从实例中拷贝文件到OSS_系统运维管理(OOS)-阿里云帮助中心

我们经常遇到从某台线上的机器把日志文件拷贝出来或者是进程卡了，通过jstack输出到某个文件再进行分析。我们使用osscmd把数据导到OSS里，但是手动配置过程实在是过于繁琐。通过如下模板您可很容易地处理此需求。

解决方案

从实例中拷贝文件可以利用云助手的RunCommand去到机器上去调用ossutil做数据拷贝，关键问题是AK怎么传，当然我们可以把AK当参数以命令的方式传入，但是这种太不安全了，相当于AK明文传输。那么有没有别的办法解决AK明文传输的问题呢？此时就想到了InstanceRole的功能，可以通过给instance attach一个ram role，那么在实例内部curl某个固定地址就能拿到以这个role身份产生的sts-token。详细参见概述。
```
curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}	
```

以上AK的问题解决了，另外的问题ossutil在实例中访问不了公网的情况怎么下载呢？咨询了OSS的支持同事，他们并不提供内网下载的地址。只能自己搭建了，我们自己创建了个bucket为oos-public的bucket提供全局只读的权限可以方便内网下载。解析JSON我们需要个方便的工具jq，我们同时把这个文件传到公共bucket上经过调试最终的脚本如下。

test -e ossutil64 || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/ossutil64 && chmod 755 ossutil64
test -e jq || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/jq && chmod 755 jq
stsToken=`curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}`
accessKeyId=`echo $stsToken |./jq .AccessKeyId | awk -F'\"' '{print $2}'` 
accessKeySecret=`echo $stsToken | ./jq .AccessKeySecret | awk -F'\"' '{print $2}' 
securityToken=`echo $stsToken | ./jq .SecurityToken | awk -F'\"' '{print $2}' 
endpoint=https://oss-{{ACS::RegionId}}.aliyuncs.com;
./ossutil64 -i $accessKeyId -k $accessKeySecret -t $securityToken -e $endpoint cp {{SrcUrl}} {{DestUrl}}

再转换成OOS模板。

YAML

FormatVersion: OOS-2019-06-01
Description: Copy ECS instance file to OSS
Parameters:
  InstanceId:
    Type: String
    Description: The Ecs instance
  SrcUrl:
    Type: String
    Description: Src url
  DestUrl:
    Type: String
    Description: Dest url
  InstanceAssumeRole:
    Type: String
    Description: ''
  OOSAssumeRole:
    Type: String
    Description: oos assume this role to execution task.
    Default: OOSServiceRole
RamRole: '{{OOSAssumeRole}}'
Tasks:
  - Name: checkInstanceReady
    Action: ACS::CheckFor
    Description: describe instances with specified parameters.
    Properties:
      API: DescribeInstances
      Service: ECS
      PropertySelector: Instances.Instance[].Status
      DesiredValues:
        - Running
      Parameters:
        InstanceIds:
          - '{{ InstanceId }}'
  - Name: runCommand
    Action: ACS::ECS::RunCommand
    Description: ''
    Properties:
      commandContent:
        Fn::Join:
          - |+

          - - test -e oos || mkdir oos;
            - cd oos;
            - test -e ossutil64 || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/ossutil64 && chmod 755 ossutil64
            - test -e jq || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/jq && chmod 755 jq
            - stsToken=`curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}`
            - accessKeyId=`echo $stsToken |./jq .AccessKeyId | awk -F'"' '{print $2}'` ;
            - accessKeySecret=`echo $stsToken | ./jq .AccessKeySecret | awk -F'"' '{print $2}'` ;
            - securityToken=`echo $stsToken | ./jq .SecurityToken | awk -F'"' '{print $2}'` ;
            - endpoint=https://oss-{{ACS::RegionId}}-internal.aliyuncs.com;
            - ./ossutil64 -i $accessKeyId -k $accessKeySecret -t $securityToken -e $endpoint cp {{SrcUrl}} {{DestUrl}}
      commandType: RunShellScript
      instanceId: '{{InstanceId}}'
    Outputs:
      CommandOutput:
        Type: String
        ValueSelector: InvocationResult[].Output

JSON

{
  "FormatVersion": "OOS-2019-06-01",
  "Description": "Copy ECS instance file to OSS",
  "Parameters": {
    "InstanceId": {
      "Type": "String",
      "Description": "The Ecs instance"
    },
    "SrcUrl": {
      "Type": "String",
      "Description": "Src url"
    },
    "DestUrl": {
      "Type": "String",
      "Description": "Dest url"
    },
    "InstanceAssumeRole": {
      "Type": "String",
      "Description": ""
    },
    "OOSAssumeRole": {
      "Type": "String",
      "Description": "oos assume this role to execution task.",
      "Default": "OOSServiceRole"
    }
  },
  "RamRole": "{{OOSAssumeRole}}",
  "Tasks": [
    {
      "Name": "checkInstanceReady",
      "Action": "ACS::CheckFor",
      "Description": "describe instances with specified parameters.",
      "Properties": {
        "API": "DescribeInstances",
        "Service": "ECS",
        "PropertySelector": "Instances.Instance[].Status",
        "DesiredValues": [
          "Running"
        ],
        "Parameters": {
          "InstanceIds": [
            "{{ InstanceId }}"
          ]
        }
      }
    },
    {
      "Name": "runCommand",
      "Action": "ACS::ECS::RunCommand",
      "Description": "",
      "Properties": {
        "commandContent": {
          "Fn::Join": [
            "\n",
            [
              "test -e oos || mkdir oos;",
              "cd oos;",
              "test -e ossutil64 || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/ossutil64 && chmod 755 ossutil64",
              "test -e jq || wget https://oos-public.oss-{{ACS::RegionId}}-internal.aliyuncs.com/x64/jq && chmod 755 jq",
              "stsToken=`curl http://100.100.100.200/latest/meta-data/Ram/security-credentials/{{InstanceAssumeRole}}`",
              "accessKeyId=`echo $stsToken |./jq .AccessKeyId | awk -F'\"' '{print $2}'` ;",
              "accessKeySecret=`echo $stsToken | ./jq .AccessKeySecret | awk -F'\"' '{print $2}'` ;",
              "securityToken=`echo $stsToken | ./jq .SecurityToken | awk -F'\"' '{print $2}'` ;",
              "endpoint=https://oss-{{ACS::RegionId}}-internal.aliyuncs.com;",
              "./ossutil64 -i $accessKeyId -k $accessKeySecret -t $securityToken -e $endpoint cp {{SrcUrl}} {{DestUrl}}"
            ]
          ]
        },
        "commandType": "RunShellScript",
        "instanceId": "{{InstanceId}}"
      },
      "Outputs": {
        "CommandOutput": {
          "Type": "String",
          "ValueSelector": "InvocationResult[].Output"
        }
      }
    }
  ]
}

传入参数实例ID，拷贝的文件，拷贝路径，传好之后执行。
执行成功后的结果。
根据Execution的执行日志可以看到执行过程base64 decode出来后可以看到Shell执行的具体信息，从结果上看是成功了，再到OSS上看，文件在不在。

总结

以上我们介绍了如果使用系统运维管理方便的从实例中拷贝文件到OSS上，结合拷贝文件的场景与执行命令的组合，我们可以方便的做出到某台机器jstack输出文件并拷贝出来等相应场景。