本文档介绍通过Terraform管理阿里云ES的方法,包括创建、更新、查看、删除实例等操作。

背景信息

通过Terraform,您可以使用代码配置实现物理机等资源的分配。也就是说通过Terraform,写一个配置文件,就可以帮助您购买一台云机器,或者申请到阿里云Elasticsearch(简称ES)、OSS等云资源。

安装并配置Terraform环境

说明 您可以使用以下两种方式安装并配置Terraform环境:
  1. 前往Terraform官网下载适用于您的操作系统的程序包。
    本文以Linux系统为例。如果您还没有Linux环境,可购买阿里云ECS实例,详情请参见创建ECS实例
  2. 将程序包解压到/usr/local/bin
    如果将可执行文件解压到其他目录,需要按照以下方法为其定义全局路径:
  3. 运行terraform验证路径配置。
    terraform
    Usage: terraform [-version] [-help] <command> [args]
    运行terraform
  4. 为提高权限管理的灵活性和安全性,建议您创建RAM用户,并为其授权。
    1. 登录RAM控制台
    2. 创建名为Terraform的RAM用户,并为该用户创建AccessKey。
      具体操作方法请参见创建RAM用户
      注意 请不要使用主账号的AccessKey配置Terraform工具。
    3. 为RAM用户授权。
      在本示例中,给用户Terraform授予AliyunElasticsearchFullAccessAliyunVPCFullAccess权限,具体操作方法请参见为RAM用户授权
  5. 创建测试目录。
    因为每个Terraform项目都需要创建一个独立的执行目录,所以需要先创建一个测试目录。以下创建一个名为terraform-test的测试目录。
    mkdir terraform-test
  6. 进入terraform-test目录。
    cd terraform-test
  7. 创建配置文件,并配置身份认证信息。
    Terraform在运行时,会读取该目录空间下所有*.tf和*.tfvars文件。因此,您可以按照实际用途,将配置信息写入到不同的文件中。以下列出几个常用的配置文件:
    配置文件 说明
    provider.tf provider配置。
    terraform.tfvars 配置provider要用到的变量。
    varable.tf 通用变量。
    resource.tf 资源定义。
    data.tf 包文件定义。
    output.tf 输出。

    例如创建provider.tf文件时,可以按照以下格式配置您的身份认证信息:

    vim provider.tf
    provider "alicloud" {
        region      = "cn-hangzhou"
        access_key  = "LTA**********NO2"
        secret_key   = "MOk8x0*********************wwff"
        }

    更多配置信息参见alicloud_elasticsearch_instance

  8. 在当前目录下创建plugh目录,下载provider插件并解压到plugh目录下。
    请单击下载provider插件
  9. 初始化工作目录,使用-plugin-dir指定provider所在的路径,完成配置。
    terraform init -plugin-dir=./plugh/

    输出Terraform has been successfully initialized表示初始化成功。

    注意 每个Terraform项目在新建Terraform工作目录并创建配置文件后,都需要初始化工作目录。

创建阿里云ES实例

  1. 在测试目录下,创建一个elastic.tf配置文件。
  2. 参考以下脚本配置elastic.tf文件,创建一个跨可用区的6.7版本的阿里云ES实例。
    resource "alicloud_elasticsearch_instance" "instance" {
      instance_charge_type = "PostPaid"
      data_node_amount     = "2"
      data_node_spec       = "elasticsearch.sn2ne.large"
      data_node_disk_size  = "20"
      data_node_disk_type  = "cloud_ssd"
      vswitch_id           = "vsw-bp1f7r0ma00pf9h2l****"
      password             = "es_password"
      version              = "6.7_with_X-Pack"
      master_node_spec     = "elasticsearch.sn2ne.large"
      description          = "zl-terraform"
      zone_count           = "2"
    }

    支持的所有参数说明如下。

    参数 是否必选 描述
    description 实例名称,长度为0~30个字符,以大小字母,数字或中文开头,可包含“_”或“-” 。
    instance_charge_type 计费模式。可选值:PrePaidPostPaid(默认)。
    period 购买时长(单位:月),当instance_charge_typePrePaid时有效。可选值:[1~9]、12、24、36,默认是1月。
    data_node_amount ES集群的数据节点的个数。可选值:2~50之间。
    data_node_spec 数据节点实例规格。
    data_node_disk_size 指定磁盘空间。不同类型的磁盘,支持的最大存储空间大小不同:
    • cloud_ssd:SSD盘,支持最大存储2048GiB(2T)。
    • cloud_efficiency:高效云盘,支持最大5T的存储空间,提供较为低廉的存储能力,适合大规模数据量的日志及分析场景。高效云盘超过2048Gib时,只能取:2560、3072、3584、4096、4608、5120。
    data_node_disk_type 存储类型。可选值:cloud_ssdcloud_efficiency
    vswitch_id VSwitch的ID。
    password 实例密码,支持大小写、数字、特殊字符,长度为8~32位字符,特殊字符:!@#$%^&*()_+-=
    kms_encrypted_password kms加密密码。如果配置了password,该字段将被忽略。passwordkms_encrypted_password必须配置一个。
    kms_encryption_context kms加密上下文。用于对使用kms_encrypted_password加密创建或更新的实例进行解密。可参见encryption context,设置kms_encrypted_password有效。
    version ES版本。可选值:5.5.3_with_X-Pack6.3_with_X-Pack6.7_with_X-Pack
    private_whitelist 设置实例VPC网络白名单。
    kibana_whitelist 设置Kibana访问白名单。
    master_node_spec Master节点规格。
    zone_count 可用区数量。取值为1~3之间,data_node_amount必须是该值的整数倍。

    更多参数详情请参见alicloud_elasticsearch_instance

    注意 kms_encrypted_passwordkms_encryption_context参数要求provider插件版本在1.57.1及以上;zone_count参数要求provider插件版本在1.44.0及以上。
  3. 执行terraform plan命令,查看将会执行的操作。
    执行后,返回结果如下:
    # terraform plan
    Refreshing Terraform state in-memory prior to plan...
    The refreshed state will be used to calculate this plan, but will not be
    persisted to local or remote state storage.
    ------------------------------------------------------------------------
    An execution plan has been generated and is shown below.
    Resource actions are indicated with the following symbols:
      + create
    
    Terraform will perform the following actions:
      # alicloud_elasticsearch_instance.instance will be created
      + resource "alicloud_elasticsearch_instance" "instance" {
          + data_node_amount     = 2
          + data_node_disk_size  = 20
          + data_node_disk_type  = "cloud_ssd"
          + data_node_spec       = "elasticsearch.sn2ne.large"
          + description          = "zl-terraform"
          + domain               = (known after apply)
          + id                   = (known after apply)
          + instance_charge_type = "PostPaid"
          + kibana_domain        = (known after apply)
          + kibana_port          = (known after apply)
          + kibana_whitelist     = (known after apply)
          + master_node_spec     = "elasticsearch.sn2ne.large"
          + password             = (sensitive value)
          + port                 = (known after apply)
          + private_whitelist    = (known after apply)
          + public_whitelist     = (known after apply)
          + status               = (known after apply)
          + version              = "6.7_with_X-Pack"
          + vswitch_id           = "vsw-bp1f7r0ma00pf9h2l****"
          + zone_count           = 2
        }
    
    Plan: 1 to add, 0 to change, 0 to destroy.
    ------------------------------------------------------------------------
    Note: You didn't specify an "-out" parameter to save this plan, so Terraform
    can't guarantee that exactly these actions will be performed if
    "terraform apply" is subsequently run.
  4. 执行terraform apply命令,运行工作目录中的配置文件,输入yes
    执行后,返回结果如下:
    # terraform apply
    Plan: 1 to add, 0 to change, 0 to destroy.
    Do you want to perform these actions?
      Terraform will perform the actions described above.
      Only 'yes' will be accepted to approve.
      Enter a value: yes
     alicloud_elasticsearch_instance.instance: Creating...
     alicloud_elasticsearch_instance.instance: Still creating... [10s elapsed]
    alicloud_elasticsearch_instance.instance: Still creating... [20s elapsed]
     ...............
     Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
  5. 登录阿里云ES控制台,查看创建成功的ES集群。
    创建成功的ES集群

更新资源

  1. 进入测试目录,修改elastic.tf配置文件。
    例如修改data_node_disk_size规格为50G
    resource "alicloud_elasticsearch_instance" "instance" {
      instance_charge_type = "PostPaid"
      data_node_amount     = "2"
      data_node_spec       = "elasticsearch.sn2ne.large"
      data_node_disk_size  = "50"
      data_node_disk_type  = "cloud_ssd"
      vswitch_id           = "vsw-bp1f7r0ma00pf9h2l****"
      password             = "es_password"
      version              = "6.7_with_X-Pack"
      master_node_spec     = "elasticsearch.sn2ne.large"
      description          = "zl-terraform"
      zone_count           = "2"
    }
    注意
    • 创建实例后,version无法修改。
    • 每次请求,只支持一项操作修改。例如同时修改data_node_specdata_node_disk_size,系统将会出现错误响应。
  2. 执行terraform plan查看资源信息。
  3. 执行terraform apply等待资源升配结束。

导入阿里云ES资源

如果阿里云ES实例不是通过Terraform创建,可通过命令,将阿里云ES导入到Terraform的state目录下。

  1. 在测试目录下,创建一个main.tf文件。
    # vim main.tf
  2. 进行资源声明,指定所要导入的资源在state中的存放路径。
    resource "alicloud_elasticsearch_instance" "test" {}
  3. 开始资源导入操作。
    # terraform import alicloud_elasticsearch_instance.test  es-cn-0pp1f1y5g000h****
    alicloud_elasticsearch_instance.test: Importing from ID "es-cn-0pp1f1y5g000h****"...
    alicloud_elasticsearch_instance.test: Import prepared!
      Prepared alicloud_elasticsearch_instance for import
    alicloud_elasticsearch_instance.test: Refreshing state... [id=es-cn-0pp1f1y5g000h****]
    
    Import successful!
    
    The resources that were imported are shown above. These resources are now in
    your Terraform state and will henceforth be managed by Terraform.
    说明 有关import如何实现存量资源的管理,请参见一文揭秘存量云资源的管理难题

查看所有被管理的资源

使用terraform show命令,查看当前state中所有被管理的资源及其所有属性值。

# terraform show
# alicloud_elasticsearch_instance.instance:
resource "alicloud_elasticsearch_instance" "instance" {
    data_node_amount     = 2
    data_node_disk_size  = 20
    data_node_disk_type  = "cloud_ssd"
    data_node_spec       = "elasticsearch.sn2ne.large"
    description          = "zl-terraform"
    domain               = "es-cn-dssf9op81lz4q****.elasticsearch.aliyuncs.com"
    id                   = "es-cn-dssf9op81lz4q****"
    instance_charge_type = "PostPaid"
    kibana_domain        = "es-cn-dssf9op81lz4q****.kibana.elasticsearch.aliyuncs.com"
    kibana_port          = 5601
    kibana_whitelist     = []
    master_node_spec     = "elasticsearch.sn2ne.large"
    password             = (sensitive value)
    port                 = 9200
    private_whitelist    = []
    public_whitelist     = []
    status               = "active"
    version              = "6.7.0_with_X-Pack"
    vswitch_id           = "vsw-bp1f7r0ma00pf9h2l****"
    zone_count           = 2
}

# alicloud_elasticsearch_instance.test:
resource "alicloud_elasticsearch_instance" "test" {
    data_node_amount     = 3
    data_node_disk_size  = 51
    data_node_disk_type  = "cloud_ssd"
    data_node_spec       = "elasticsearch.r5.large"
    description          = "zl-es-cn"
    domain               = "es-cn-0pp1f1y5g000h****.elasticsearch.aliyuncs.com"
    id                   = "es-cn-0pp1f1y5g000h****"
    instance_charge_type = "PostPaid"
    kibana_domain        = "es-cn-0pp1f1y5g000h****.kibana.elasticsearch.aliyuncs.com"
    kibana_port          = 5601
    kibana_whitelist     = []
    port                 = 9200
    private_whitelist    = []
    public_whitelist     = []
    status               = "active"
    version              = "6.7.0_with_X-Pack"
    vswitch_id           = "vsw-bp1f7r0ma00pf9h2l****"
    zone_count           = 1

    timeouts {}
}

删除实例

进入测试目录,执行terraform destroy命令,输入yes,即可销毁该实例。

# terraform destroy
alicloud_elasticsearch_instance.instance: Refreshing state... [id=es-cn-v3x49h5397fau****]

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # alicloud_elasticsearch_instance.instance will be destroyed
  - resource "alicloud_elasticsearch_instance" "instance" {
      - data_node_amount     = 2 -> null
      - data_node_disk_size  = 20 -> null
      - data_node_disk_type  = "cloud_ssd" -> null
      - data_node_spec       = "elasticsearch.sn2ne.large" -> null
      - description          = "zl-terraform" -> null
      - domain               = "es-cn-v3x49h5397fau****.elasticsearch.aliyuncs.com" -> null
      - id                   = "es-cn-v3x49h5397fau****" -> null
      - instance_charge_type = "PostPaid" -> null
      - kibana_domain        = "es-cn-v3x49h5397fau****.kibana.elasticsearch.aliyuncs.com" -> null
      - kibana_port          = 5601 -> null
      - kibana_whitelist     = [] -> null
      - master_node_spec     = "elasticsearch.sn2ne.large" -> null
      - password             = (sensitive value)
      - port                 = 9200 -> null
      - private_whitelist    = [] -> null
      - public_whitelist     = [] -> null
      - status               = "active" -> null
      - version              = "6.7.0_with_X-Pack" -> null
      - vswitch_id           = "vsw-bp1f7r0ma00pf9h2l****" -> null
      - zone_count           = 2 -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you really want to destroy all resources?
  Terraform will destroy all your managed infrastructure, as shown above.
  There is no undo. Only 'yes' will be accepted to confirm.

  Enter a value: yes

alicloud_elasticsearch_instance.instance: Destroying... [id=es-cn-v3x49h5397fau971l]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 10s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 20s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 30s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 40s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 50s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 1m0s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 1m10s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 1m20s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 1m30s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 1m40s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 1m50s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 2m0s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 2m10s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 2m20s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 2m30s elapsed]
alicloud_elasticsearch_instance.instance: Still destroying... [id=es-cn-v3x49h5397fau****, 2m40s elapsed]
alicloud_elasticsearch_instance.instance: Destruction complete after 10m2s

Destroy complete! Resources: 1 destroyed.