This topic describes the observability feature provided by custom Message Queue for Apache Kafka connectors. This topic also describes how to troubleshoot issues.
Observability
Custom Message Queue for Apache Kafka connectors provide the observability feature. You can use this feature to monitor metrics that are related to Message Queue for Apache Kafka instances, workers, and connectors.
Log on to the EventBridge console. In the left-side navigation pane, click Event Streams.
- In the top navigation bar, select a region.
- On the Event Streams page, find the target custom connector for Message Queue for Apache Kafka, and click Details in the Actions column.
- In the left-side navigation pane, click Metric Monitoring.
- Click the ApsaraMQ for Kafka Instance Monitoring tab to view observability data for message production and consumption at the instance level.
- Click the Worker Monitoring tab to view resource consumption at both the Serverless App Engine (SAE) application and instance levels. Because SAE runs the Kafka Connect workers, its resource consumption directly reflects the resource usage of the connector. The page provides a Resource Type drop-down list, where you can select Application or Instance, and a time range filter. The Application Monitoring section includes line charts for CPU Usage (%) and System Average Load (1 min) that let you track metric trends.
- Click the Connector Monitoring tab to check the status of Kafka Connect tasks. These metrics are provided by the Kafka Connect framework. If a task fails, detailed error information is displayed to help you troubleshoot. In the Tasks section, task counts are grouped by Total, RUNNING, FAILED, PAUSED, and UNASSIGNED. The table below shows the status and stack trace for each task in the State, TaskId, and Trace columns.
- Click the ApsaraMQ for Kafka Instance Monitoring tab to view observability data for message production and consumption at the instance level.
Troubleshooting
If a Message Queue for Apache Kafka connector fails to run, you can perform the following steps to troubleshoot the issue:
- On the event stream details page, click Overview in the left-side navigation pane.
- In the Basic Information section, click Running Exception to view the error type and details.
If the error message indicates that a Kafka Connect task failed, you can click the Diagnostics link to navigate to the Connector Monitoring page for details about the failed task.
- If you still cannot resolve the issue after you perform the preceding steps, you can log on to the SAE console for further debugging.
- On the event stream details page, click Overview in the left-side navigation pane.
- In the Worker Information section, click the instance name next to SAE Application to open the application details page in the SAE console.
- On the Basic Information page, click the Instance Deployment Information tab.
- In the Actions column of the instance, click Webshell to log in to the Kafka Connect running environment.
On the application details page, click the Instance Deployment Information tab, and find your instance.You can view the running logs of the connector in the /opt/kafka/logs/connect.log file to troubleshoot the issue.