MetricStore HTTP API return values

Retry policy

The retry policy guides the client's retry behavior when an error occurs. This prevents ineffective retries that can degrade the user experience.

Status codes and recommended actions

Successful request

{
    "status": "success",
    "data": {
        "resultType": "matrix",
        "result": [
            {
                "metric": {},
                "values": [
                    [
                        1673798460,
                        "11111111"
                    ],
                    [
                        1673799060,
                        "22222222"
                    ],
                    [
                        1673799660,
                        "33333333"
                    ]
                ]
            }
        ]
    }
}

Partial data returned

When a query is incomplete, partial data is returned. In this case, the slsStatus field contains a detailed error message and a retry policy.

{
	"status": "success",
	"slsStatus": {
		"retryPolicy": "Once",
		"errorCode": "ShardResourceExceed",
		"errorMessages": ["sls response error 1 task data", "sls response drop 1 task data"]
	},
	"data": {
		"resultType": "vector",
		"result": [{
			"metric": {
				"__name__": "up",
			},
			value: [
				1747807789.37,
				"1"
			]
		}]
	},
	"warnings": ["sls response error 1 task data", "sls response drop 1 task data"]
}

Returned errors

Example 1

{
	"status": "error",
	"slsStatus": {
		"errRetryPolicy": "None",
		"errCode": "EngineResourceExceed",
		"errMessages": ["too many time Series or items"]
	},
	"data": {
	},
	"error": "too many time Series or items"
}

Example 2

{
	"status": "error",
	"slsStatus": {
		"retryPolicy": "None",
		"errorCode": "BadDataError",
		"errorMessages": ["vector cannot contain metrics with the same labelset"]
	},
        "data": {
	},
	"error": "vector cannot contain metrics with the same labelset"
}

Frontend display

MetricStore features built-in metric visualization, supports building Dashboards, and is compatible with Grafana. If a query is abnormal, the built-in visualization solution automatically displays the relevant information on the chart.

• Use Grafana 10.0 or later for better warning messages.

If you build a custom visualization solution based on the MetricStore Prometheus API, handle exceptions based on the following policies:

1. If the HTTP status code is 200, the chart is rendered normally.

   1. If the slsStatus field exists, display a warning message on the chart and provide a solution based on the RetryPolicy.

2. If the HTTP status code is not 200, display the error message directly on the chart and provide a solution based on the RetryPolicy.

Secondary data calculation scenario

MetricStore supports using Scheduled SQL to periodically calculate aggregate data. If you build custom secondary calculation scenarios based on the Prometheus API for MetricStore, we recommend that you handle abnormalities according to the following policies:

1. If the status code is 200:

   1. If the slsStatus field does not exist, store the data and proceed to the next scheduled job.

   2. If the slsStatus field exists, handle the exception based on the RetryPolicy as follows:

      1. None: Retry once. If the policy is still None, mark the task as failed and proceed to the next scheduled job.

      2. Once: Retry once. If the policy is still Once, mark the task as failed and proceed to the next scheduled job.

      3. Continuous: Retry using an annealing strategy. If the retries continue for a certain period (10 to 30 minutes is recommended), mark the task as failed and proceed to the next scheduled job.

2. If the status code is not 200:

   1. If the slsStatus field does not exist, the error is usually caused by network issues. Continuous retries are recommended.

   2. If the slsStatus field exists, handle the exception based on the RetryPolicy as follows:

      1. None: Retry once. If the policy is still None, mark the task as failed and proceed to the next scheduled job.

      2. Once: Retry once. If the policy is still Once, mark the task as failed and proceed to the next scheduled job.

      3. Continuous: Retry using an annealing strategy. If the retries continue for a certain period (10 to 30 minutes is recommended), mark the task as failed and proceed to the next scheduled job.

• We recommend that you write data from failed tasks to a separate LogStore and configure alerts as needed. For more information, see Set alerts for scheduled SQL tasks.

Real-time alerting scenario

SLS lets you create alerts directly on MetricStore. However, if you build a custom real-time alerting system based on the Prometheus API for MetricStore, you can handle abnormalities using the following policies:

1. If the status code is 200:

   1. If the slsStatus field does not exist, monitor the alert conditions normally.

   2. If the slsStatus field exists, handle the exception based on the RetryPolicy as follows:

      1. None: Mark the current alert execution as failed and proceed to the next scheduled execution.

      2. Once: Retry once. If the policy is still Once, mark the current alert execution as failed and proceed to the next scheduled execution.

      3. Continuous: Retry using an annealing strategy. If the retries continue for a certain period (within 1 minute is recommended), mark the current alert execution as failed and proceed to the next scheduled execution.

2. If the status code is not 200:

   1. If the slsStatus field does not exist, the error is usually caused by network issues. Continuous retries are recommended.

   2. If the slsStatus field exists, handle the exception based on the RetryPolicy as follows:

      1. None: Mark the current alert execution as failed and proceed to the next scheduled execution.

      2. Once: Retry once. If the policy is still Once, mark the current alert execution as failed and proceed to the next scheduled execution.

      3. Continuous: Retry using an annealing strategy. If the retries continue for a certain period (within 1 minute is recommended), mark the current alert execution as failed and proceed to the next scheduled execution.

• Recommendations:

  • The total retry time should not exceed the interval between scheduled alerts. For example, if the alert interval is 1 minute and the total retry time exceeds 1 minute, skip the current execution and proceed to the next one. Otherwise, you may miss alerts for new and more important data.

  • We recommend that you write data from failed alert executions to a separate LogStore and configure alerts as needed. For more information, see Custom analysis of alert logs.