The DataHub console provides topic-level metrics on the topic details page, including read/write requests, real-time traffic, and historical traffic. Use these metrics to monitor throughput, diagnose failures, and assess topic health.
Metrics
The topic details page provides six metrics:
QPS: Read/write requests per second.
RPS: Read/write records per second.
Throughput: Read/write throughput per second, in KB.
Throughput (Uncompressed): Uncompressed read/write throughput per second, in KB.
Fails: The data volume of failed requests per second, in KB.
Latency: Read/write request latency per request, in microseconds.
QPS metrics
Queries per second (QPS) measures how many read and write requests the topic receives each second. A sudden drop or spike helps you spot ingestion bursts or client connectivity issues.
RPS metrics
Records per second (RPS) measures how many individual records are read from or written to the topic each second. Compare RPS against QPS to understand the average batch size per request.
Throughput
Throughput measures the compressed read and write data throughput in KB per second. This value reflects what is actually transmitted over the network after compression.
Throughput (Uncompressed) metrics
This metric was added in DataHub 2.25.
Throughput (Uncompressed) measures the raw read and write data throughput in KB per second before compression is applied. Compare this value against Throughput to understand your effective compression ratio: a lower Throughput value means data is being compressed on the wire.
If compressed throughput is higher than uncompressed throughput, this usually occurs for one of two reasons:
Compression is not enabled for read/write requests for the topic.
Compression is enabled, but read/write requests are not sent in batches and the write protocol is JSON.
Fails metrics
This metric was added in DataHub 2.25.
If a single request writes to multiple shards, the entire request is considered failed if any write operation fails.
Fails measures the data volume of failed requests per second, in KB. A non-zero value indicates that some requests are being rejected. When this metric rises, investigate shard throttling or network errors.
Latency metrics
Latency measures the end-to-end round-trip time for read and write requests, in microseconds. A rising latency trend — especially for write requests — often indicates resource contention on the shard.
Real-time and historical traffic metrics
The topic details page includes two traffic charts that give you a time-based view of request activity. Click the dots in the upper-right corner of the chart area to switch between views.
The charts for historical traffic and real-time traffic (uncompressed) were added in DataHub 2.25.
Real-time traffic
The real-time traffic chart shows current request activity for the topic. The chart appearance varies depending on whether compression is enabled.
When data is uncompressed, the chart appears as follows:
When data is compressed, the chart appears as follows:
Historical traffic
The historical traffic chart shows request traffic for the topic over the last seven days. Use this view to spot usage trends and recurring traffic patterns.
