Serverless instances

Limitations

• Serverless instances are available in the following regions and zones:

  Region	Zone
  China (Hangzhou)	Zones J and K
  China (Shanghai)	Zones E and L
  China (Shenzhen)	Zones F and D
  China (Beijing)	Zones I and L
  China (Hong Kong)	Zones B and D
  Singapore	Zones A and C

• Each Alibaba Cloud account can create only one serverless instance in each region.

Features

Serverless instances differ from virtual warehouse and general-purpose instances in the following ways:

• Each serverless instance has a maximum computing resource quota of 512 CUs. You can still set the available resources at the SQL level. For more information, see Work with serverless computing.

• All read and write requests use Serverless Computing resources. You cannot manually set the hg_computing_resource parameter.

• Serverless instances do not support query queues. All requests follow the resource request rules and queuing rules of serverless computing.

• A serverless instance supports a maximum of 256 connections. For more information, see Manage connections.

• A serverless instance has a default shard count of 16. The total shard count for all table groups within the instance cannot exceed 128. Avoid configuring a large shard count. For more information, see Manage table shards.

Serverless instances do not support the following features:

• Scenarios that are not supported by serverless computing, such as Data Lake Formation (DLF) foreign tables, cross-database queries, and multi-row DML mixed transactions. For more information, see Serverless computing limitations.

• row-oriented storage and row-column hybrid storage. Serverless instances support only column-oriented storage. For more information, see Table storage formats: column-oriented, row-oriented, and row-column hybrid.

• Fixed Plan. The Hologres query engine (HQE) automatically routes queries for execution.

• Binlog generation is supported, but binlog consumption is not. For more information, see Hologres Binlog.

• infrequent access (IA) storage (cold storage). For more information, see Data tiering.

• Setting a time-to-live (TTL) period for data with the time_to_live_in_seconds property. For more information, see Set table properties. To clean up data, you must delete it manually at regular intervals.

• Manual compaction. Background auto-compaction is not required. Data compacts during the import process. Therefore, the data import performance of a serverless instance is lower than that of a virtual warehouse or general-purpose instance.

• disaster recovery and backup and restoration.

• vector computation.

• Remote user-defined function (UDF).

• real-time materialized views. They support dynamic tables.

Serverless instances have the following integration considerations:

• Flink: Serverless instances cannot serve as Flink source tables. You can use them as Flink dimension tables and result tables. However, because serverless instances do not support Fixed Plan, the Hologres query engine (HQE) handles execution automatically, which may degrade performance. Perform thorough testing before use.

• DataWorks Data Integration: Because serverless instances do not support Fixed Plan, the HQE handles import tasks automatically, which may degrade performance. Perform thorough testing before use.

• PAI-Rec: Because serverless instances do not support vector computation or Fixed Plan, use a virtual warehouse instance to integrate with the PAI-Rec recommendation system.

Create a serverless instance

1. Go to the purchase page or log on to the Hologres console. In the left-side navigation pane, click Instances. On the Instances page, click Create Instance.

2. On the purchase page, configure the following parameters:

   Parameter	Value
   Product Type	Select Exclusive Instance (Pay-As-You-Go).
   Region	Select a region that supports serverless instances.
   Instance Type	Select Serverless.
   Zone	Select the target zone.
   Virtual Private Cloud (VPC)	Select the VPC and vSwitch that you created. For more information, see VPCs and vSwitches.
   vSwitch
   Instance Name	Enter a custom name.
   Service-linked Role	The status is Created. If you are purchasing a Hologres instance for the first time, click Create Service-linked Role at the bottom of the page.
   Resource Group	Select the Default Resource Group. You can also create and select a different resource group.

3. After you configure the parameters, click Buy Now.

4. On the Confirm Order page, click Activate Now.

   Instance creation takes a few minutes.

Manage a serverless instance

Use a serverless instance

You use a serverless instance the same way as a virtual warehouse or general-purpose instance. Connect to the instance and run your queries.

The following example demonstrates using a serverless instance with GitHub public event data.

1. Create a serverless instance.

2. Create a database.

3. Import a public dataset with one click or manually create an SQL query to create a MaxCompute foreign table and import data with the following statements:

   -- Create a schema for the foreign table.
   CREATE SCHEMA IF NOT EXISTS hologres_foreign_dataset_github_event;
   
   -- Create a schema for the internal table and data import.
   CREATE SCHEMA IF NOT EXISTS hologres_dataset_github_event;
   
   -- Create a foreign table.
   DROP FOREIGN TABLE IF EXISTS hologres_foreign_dataset_github_event.dwd_github_events_odps;
   
   IMPORT FOREIGN SCHEMA "bigdata_public_dataset#github_events" LIMIT TO
   (
       dwd_github_events_odps
   ) 
   FROM SERVER odps_server INTO hologres_foreign_dataset_github_event OPTIONS(if_table_exist 'error',if_unsupported_type 'error');
   
   -- Create an internal table.
   DROP TABLE IF EXISTS hologres_dataset_github_event.hologres_github_event;
   BEGIN;
   CREATE TABLE hologres_dataset_github_event.hologres_github_event (
       id BIGINT,
       actor_id BIGINT,
       actor_login TEXT,
       repo_id BIGINT,
       repo_name TEXT,
       org_id BIGINT,
       org_login TEXT,
       type TEXT,
       created_at TIMESTAMP WITH TIME ZONE NOT NULL,
       action TEXT,
       iss_or_pr_id BIGINT,
       number BIGINT,
       comment_id BIGINT,
       commit_id TEXT,
       member_id BIGINT,
       rev_or_push_or_rel_id BIGINT,
       ref TEXT,
       ref_type TEXT,
       state TEXT,
       author_association TEXT,
       language TEXT,
       merged boolean,
       merged_at TIMESTAMP WITH TIME ZONE,
       additions BIGINT,
       deletions BIGINT,
       changed_files BIGINT,
       push_size BIGINT,
       push_distinct_size BIGINT,
       hr TEXT,
       month TEXT,
       year TEXT,
       ds TEXT
   );
   CALL set_table_property('hologres_dataset_github_event.hologres_github_event', 'distribution_key', 'id');
   CALL set_table_property('hologres_dataset_github_event.hologres_github_event', 'event_time_column', 'created_at');
   CALL set_table_property('hologres_dataset_github_event.hologres_github_event', 'clustering_key', 'created_at');
   
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.id IS 'Event ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.actor_id IS 'Event initiator ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.actor_login IS 'Event initiator logon name';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.repo_id IS 'Repository ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.repo_name IS 'Repository name';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.org_id IS 'Repository organization ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.org_login IS 'Repository organization name';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.type IS 'Event type';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.created_at IS 'Time when the event occurred';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.action IS 'Event action';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.iss_or_pr_id IS 'Issue or pull request ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.number IS 'Issue or pull request number';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.comment_id IS 'Comment ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.commit_id IS 'Commit ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.member_id IS 'Member ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.rev_or_push_or_rel_id IS 'Review, push, or release ID';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.ref IS 'Name of the created or deleted resource';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.ref_type IS 'Type of the created or deleted resource';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.state IS 'State of the issue, pull request, or pull request review';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.author_association IS 'Relationship between the actor and the repository';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.language IS 'Programming language';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.merged IS 'Indicates whether the merge is accepted';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.merged_at IS 'Time when the code was merged';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.additions IS 'Number of added lines of code';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.deletions IS 'Number of deleted lines of code';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.changed_files IS 'Number of files changed in the pull request';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.push_size IS 'Number of pushes';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.push_distinct_size IS 'Number of distinct pushes';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.hr IS 'Hour when the event occurred. For example, if the event occurred at 00:23, the value is 00.';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.month IS 'Month when the event occurred. For example, if the event occurred in October 2015, the value is 2015-10.';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.year IS 'Year when the event occurred. For example, if the event occurred in 2015, the value is 2015.';
   COMMENT ON COLUMN hologres_dataset_github_event.hologres_github_event.ds IS 'Date when the event occurred. The format is YYYY-MM-DD.';
   COMMIT;
   
   -- Import data.
   SET hg_experimental_serverless_computing_required_cores = 192;
   INSERT INTO hologres_dataset_github_event.hologres_github_event
   SELECT
       *
   FROM
       hologres_foreign_dataset_github_event.dwd_github_events_odps
   WHERE
       ds BETWEEN (CURRENT_DATE - interval '365 day')::text AND (CURRENT_DATE - interval '1 day')::text;
   RESET hg_computing_resource;
   
   -- Update table statistics.
   ANALYZE hologres_dataset_github_event.hologres_github_event;

4. Query data.

   For example, to query the most active projects (those with the highest number of events) in the last week, execute the following statement:

   SELECT
       repo_name,
       COUNT(*) AS events
   FROM
       hologres_dataset_github_event.hologres_github_event
   WHERE
       ds >= (CURRENT_DATE - INTERVAL '7 day')::text
   GROUP BY
       repo_name
   ORDER BY
       events DESC
   LIMIT 5;

5. (Optional) Build a real-time dashboard with DataV.

   You can use a DataV dashboard template and configure the serverless instance as a data source to quickly build a real-time dashboard for GitHub event data.

   1. Create a Hologres data source.

      In DataV, add the Hologres instance and database as a new data source. For more information, see DataV.

   2. Create a visualization application.

      1. Log on to the DataV console.

      2. On the Workspace page, click Create PC screen.

         Select the Real-time analysis of GitHub event data with Hologres template.

      3. Modify the data source of the components in the template.

         For example, for the Total number of public events today component in the upper-left corner:

         1. Click the number box for Total number of public events today. In the right-side pane, click Data source and set Data source type to Hologres. In the SQL editor, enter the query select count(*) from hologres_dataset_github_event.hologres_github_event where created_at >= date_trunc('day', now()); to get and display the total number of daily public events in real time.

         2. For Select an existing data source, select the data source that you created.

            If the table name and schema in your Hologres instance are the same as in this tutorial, you do not need to modify the SQL statement.

         3. The dashboard then displays the refreshed, real-time data.

         4. Update the data source and table name for all other components as shown in the following figure.

            

      4. In the upper-right corner, click Publish to complete the dashboard setup.

         You can also click Preview to preview the real-time updates on the dashboard.