MTable Expander expands a MTable column into individual columns, making the data ready for downstream processing and model training.
What it does: Each row that contains a MTable value becomes a flat row with separate columns. For example, a MTable column m_table_col storing (f0 string, f1 double) pairs expands into two separate columns — f0 and f1 — that any downstream component can work with directly.
Before and after expansion
Input: m_table_col (MTable format) |
Output: f0 (string) |
Output: f1 (double) |
|---|---|---|
{f0: "11L", f1: 2.2} |
11L |
2.2 |
{f0: "12L", f1: 2.0} |
12L |
2.0 |
Supported computing resources
-
MaxCompute
-
Apache Flink
-
Deep Learning Containers (DLC)
Configure the component in the PAI console
Input ports
| Port (left-to-right) | Data type | Recommended upstream component | Required |
|---|---|---|---|
| data | None | Read Table, Prophet | Yes |
Parameters
Field setting
| Parameter | Description |
|---|---|
selectedCol |
The MTable column to expand. The column must be STRING in the MTABLE format. |
reservedCols |
The columns to pass through to the output unchanged. |
Parameters setting
| Parameter | Description |
|---|---|
Schema |
The names and types of the expanded columns, in the format colname coltype[, colname2 coltype2[, ...]]. For example: f0 string, f1 bigint, f2 double. |
handleInvalidMethod |
How to handle rows where the MTable value cannot be parsed. Valid values: error (default), skip. |
Execution tuning
| Running mode | Description |
|---|---|
| MaxCompute | Uses MaxCompute computing resources. To configure the number of workers and memory, see Appendix: How to estimate resource usage. |
| Flink | Uses Apache Flink computing resources. To configure the number of workers and memory, see Appendix: How to estimate resource usage. |
| DLC | Uses Deep Learning Containers (DLC) computing resources. Configure the specifications based on the prompts. |
Configure the component by coding
Copy the following code into the PyAlink Script component to replicate the behavior of MTable Expander.
Input schema: id string, f0 string, f1 double (12 rows)
The example first groups rows by id using mtable_agg to produce a MTable column, then expands that MTable column back into separate f0 and f1 columns using FlattenMTableBatchOp.
Output schema: id string, f0 string, f1 int
import numpy as np
import pandas as pd
from pyalink.alink import *
df_data = pd.DataFrame([
["a1", "11L", 2.2],
["a1", "12L", 2.0],
["a2", "11L", 2.0],
["a2", "12L", 2.0],
["a3", "12L", 2.0],
["a3", "13L", 2.0],
["a4", "13L", 2.0],
["a4", "14L", 2.0],
["a5", "14L", 2.0],
["a5", "15L", 2.0],
["a6", "15L", 2.0],
["a6", "16L", 2.0]
])
input = BatchOperator.fromDataframe(df_data, schemaStr='id string, f0 string, f1 double')
zip = GroupByBatchOp()\
.setGroupByPredicate("id")\
.setSelectClause("id, mtable_agg(f0, f1) as m_table_col")
flatten = FlattenMTableBatchOp()\
.setReservedCols(["id"])\
.setSelectedCol("m_table_col")\
.setSchemaStr('f0 string, f1 int')
zip.linkFrom(input).link(flatten).print()