Vector Assembler is a machine learning algorithm for dimensionality reduction and feature extraction that simplifies data processing by converting high-dimensional data into low-dimensional vectors. It applies a mathematical transformation to input vectors to produce a fixed-length vector, which facilitates subsequent classification or clustering tasks. Vector Assembler is widely used in areas such as Natural Language Processing (NLP) and recommendation systems to improve the computational efficiency and accuracy of models.
Supported computing engines
This component supports the MaxCompute and Flink computing engines.
Component parameters
Input ports
| Port | Data types | Upstream component | Required |
|---|---|---|---|
| Data | Structured data stored in MaxCompute or OSS | None | Yes |
Parameter configuration
Field settings
| Parameter | Description |
|---|---|
| Selected columns | The columns to merge into the output vector. Accepts numeric or vector columns. Values are concatenated in the order you select the columns. |
| Reserved columns | The columns to keep in the output alongside the new vector column. |
Parameter settings
| Parameter | Description |
|---|---|
| Output column | The name of the new vector column in the output. |
| Method for handling invalid values | How to handle rows that contain NULL or NaN values. Options: ERROR (default) stops the task and throws an exception; SKIP outputs NULL for the affected row. |
| Number of threads | The number of threads for parallel processing. Default: 1. |
Execution tuning
| Parameter | Description |
|---|---|
| Number of workers | The number of workers. Used together with Memory per worker (MB). Valid range: 1–9,999. |
| Memory per worker (MB) | The memory allocated to each worker. Valid range: 1,024–65,536 MB. |
Output ports
| Port | Storage location | Downstream component | Model type |
|---|---|---|---|
| Data | No configuration required | None | None |
Example
The following PyAlink code performs the same operation as this component. Copy it into a PyAlink Script component to run it in a pipeline.
from pyalink.alink import *
def main(sources, sinks, parameter):
data = sources[0]
selectedColNames = ["col2", "col3"]
trainOp = VectorAssemblerBatchOp()\
.setSelectedCols(selectedColNames)\
.setOutputCol("vec")
result = trainOp.linkFrom(data)
result.link(sinks[0])
BatchOperator.execute()This example selects col2 and col3 as input columns and writes the assembled vector to a new column named vec.