Streaming Tunnel是MaxCompute的流式数据通道,您可以通过Streaming Tunnel向MaxCompute中上传数据,本文为您介绍使用Python版Streaming Tunnel SDK上传数据的简单示例。
注意事项
Streaming Tunnel仅支持上传数据,下文为您介绍一个简单的数据上传的Python SDK示例。
上传示例
from odps.tunnel import TableTunnel
from odps import ODPS
o = ODPS('**your-access-id**', '**your-secret-access-key**', '**your-default-project**', endpoint='**your-end-point**')
table = o.get_table('my_table')
tunnel = TableTunnel(odps)
upload_session = tunnel.create_stream_upload_session(table.name, partition_spec='pt=test')
with upload_session.open_record_writer() as writer:
record = table.new_record()
record[0] = 'test1'
record[1] = 'id1'
writer.write(record)
record = table.new_record(['test2', 'id2'])
writer.write(record)
import pandas as pd
import pyarrow as pa
with upload_session.open_arrow_writer() as writer:
df = pd.DataFrame({"name": ["test1", "test2"], "id": ["id1", "id2"]})
batch = pa.RecordBatch.from_pandas(df)
writer.write(batch)