Configure the wide table connector for Flink SQL jobs-阿里云帮助中心

You can use Lindorm wide tables as dimension tables or sink tables when processing computing tasks with a stream engine. This topic describes how to configure the wide table connector to submit computing tasks using Flink SQL.

Background information

To submit a Flink SQL job on a stream engine, you need to use the CREATE TABLE statement and configure connector parameters.

Using wide tables in the stream engine

CREATE TABLE lindorm_table(
  c1 VARCHAR,
  c2 DOUBLE,
  c3 BIGINT,
  PRIMARY KEY (c1, c2) NOT ENFORCED --The primary key must be the same as the primary key of the sink table created in LindormTable.
)WITH(
      'connector'='lindorm',
      'seedServer'='ld-bp17pwu1541ia****-proxy-lindorm.lindorm.rds.aliyuncs.com:30020',
      'userName'='yourUser',
      'password'='yourPassword',
      'tableName'='yourTablename',
      'namespace'='yourNamespace'
    );--The WITH clause contains the connector parameters.

Note

For more information about the CREATE TABLE statement, see CREATE TABLE syntax.

Connector parameters

Common parameters

Parameter	Default value	Required	Description
seedServer	None	Yes	The endpoint of LindormTable for the HBase Java API. For more information about how to obtain the endpoint, see View the endpoint of LindormTable.
namespace	None	Yes	The namespace where the wide table resides.
userName	None	Yes	The username.
password	None	Yes	The password.
tableName	None	Yes	The name of the wide table.
bufferSize	5000	No	The number of rows for batch data writes.
flushIntervalMs	2000	No	The interval at which the system performs a `flush` operation during data writes. Unit: milliseconds (ms). If a small volume of data is written, the system performs a `flush` operation at this interval. Note Standards for data write volume vary by service.
columnFamily	f	No	The name of the column family. Note When you use a wide table in the DPI engine, whether you need to configure the columnFamily parameter depends on how the table was created: Created using the HBase Java API: This parameter is required. No configuration is required for Lindorm SQL.

Spatio-temporal parameters

Parameter	Default value	Required	Description
cacheTTLMs	-1	No	The time-to-live (TTL) of the data cache. Unit: milliseconds. The default value is -1, which indicates that no data is cached. After the data in the cache expires, the spatio-temporal index is rebuilt when the next query is executed.
geomHint	None	No	Specifies a spatio-temporal lookup join query. The format is `<columnName>:<queryFunction>`, such as `fence:st_contains`. When you execute a lookup join query, the spatio-temporal equation in the join condition is replaced with the specified spatio-temporal query. For example, `fence=ST_MakePoint(x,y))` is replaced with the specified spatio-temporal query `ST_Contains(fence, ST_MakePoint(x,y))`. The following functions are supported for queryFunction: ST_Contains ST_Within ST_DWithin ST_DWithinSphere ST_Intersects ST_Overlaps ST_Equals
geomIndex	None	No	Specifies the column for which to create a spatio-temporal index. The spatio-temporal lookup join queries for this table are automatically accelerated by the memory index. Currently, you can create an index only for a single column in a single table. The functions that can be accelerated by the memory index are: ST_Contains ST_Within ST_DWithin ST_DWithinSphere ST_Intersects ST_Overlaps ST_Equals Important The geomIndex parameter must be used with the geomHint and cacheTTLMs parameters. The value of cacheTTLMs must be greater than 0.