Convert and offset datetimes-Simple Log Service(SLS)-阿里云帮助中心

You can handle datetimes to simplify log queries and data visualization. This document describes how to use functions to convert and offset datetimes.

Concepts

Datetime handling in the Simple Log Service (SLS) Domain-Specific Language (DSL) syntax involves three data types: datetime string, datetime object, and UNIX timestamp.

Datetime string
A datetime string is primarily used for display and readability. The SLS DSL syntax supports two types of datetime strings:
- Datetime strings with time zone information, such as 2019-06-02 18:41:26+08:00.
- Datetime strings without time zone information, such as 2019-06-02 10:41:26.
A datetime string with time zone information indicates the time zone by appending the time offset to the datetime:
- 2019-06-02 18:41:26+08:00 represents the time 2019-06-02 18:41:26 in the UTC+8 time zone.
- 2019-06-02 18:41:26-07:00 represents the time 2019-06-02 18:41:26 in the UTC-7 time zone.
Datetime object
A datetime object is an instantiated datetime of the Datetime data type. It provides a readable representation of time and date data.
UNIX timestamp
A UNIX timestamp is the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC) on January 1, 1970. Common scenarios for using UNIX timestamps include the following:
- Representing system time.
  In log events, the __time__ metafield indicates the log generation time, and the __receive_time__ field indicates the log reception time. The values of these fields are UNIX timestamps that represent the system time, as shown in the following example.
```
__source__:  192.0.2.1
__tag__:__receive_time__:  1562741899
__topic__: 
__time__: 1562731122
```
- Performing time-related calculations.
  Because a UNIX timestamp is a numerical value that represents seconds, it simplifies datetime calculations in many scenarios. For example:
  - Raw log
    time1: 1562741899 time2: 1562731122
  - SLS DSL orchestration
    e_set("time_diff", op_sub(v("time1"), v("time2")))
  - Processing result
    time1: 1562741899 time2: 1562731122 time_diff: 10777

Data type conversions and conversion functions

The following figure shows the conversions between datetime strings, datetime objects, and UNIX timestamps, along with their corresponding functions. 转化方式 The following table describes these conversion scenarios and functions in more detail.

Conversion scenario		Conversion function
Conversions between datetime objects and UNIX timestamps	Convert a datetime object to a UNIX timestamp	`dt_parsetimestamp`: A flexible function that converts a datetime object or datetime string to a UNIX timestamp. `dt_totimestamp`: A dedicated function that converts a datetime object to a UNIX timestamp.
Conversions between datetime objects and UNIX timestamps	Convert a UNIX timestamp to a datetime object	`dt_parse`: A flexible function that converts a UNIX timestamp or datetime string to a datetime object. `dt_fromtimestamp`: A dedicated function that converts a UNIX timestamp to a datetime object.
Conversions between datetime objects and datetime strings	Convert a datetime object to a datetime string	`dt_str`: A flexible function that converts a datetime object, UNIX timestamp, or datetime string to a datetime string in a specified format. `dt_strftime`: A dedicated function that converts a datetime object to a datetime string.
Conversions between datetime objects and datetime strings	Convert a datetime string to a datetime object	`dt_parse`: A flexible function that converts a datetime string or UNIX timestamp to a datetime object. `dt_strptime`: A dedicated function that converts a datetime string to a datetime object.
Conversions between datetime strings and UNIX timestamps	Convert a datetime string to a UNIX timestamp	`dt_parsetimestamp`: A flexible function that converts a datetime string or datetime object to a UNIX timestamp.
Conversions between datetime strings and UNIX timestamps	Convert a UNIX timestamp to a datetime string	`dt_str`: A flexible function that converts a UNIX timestamp, datetime object, or datetime string to a datetime string in a specified format. `dt_strftimestamp`: A dedicated function that converts a UNIX timestamp to a datetime string.

The figure and table show six conversions among the three data types. These conversions use two types of functions: flexible conversion functions and dedicated functions.

Intelligent transform function
Flexible conversion functions, such as dt_parse, can accept various parameter types, including UNIX timestamps, datetime objects, and datetime strings.
Dedicated functions
Dedicated functions are required for specific scenarios that flexible functions cannot handle. For example, the dt_parse function cannot automatically parse logs with custom date formats. In this scenario, you must use a dedicated function, such as dt_strptime, to parse the specified format.

Note

For more information, see Date and time functions and Conversion functions.

Conversions between datetime objects and UNIX timestamps

Processing functions
- The dt_parsetimestamp flexible conversion function converts a datetime object or datetime string to a UNIX timestamp.
- The `tz` parameter in the e_set function can convert a datetime object without a time zone to one with a time zone, or change the time zone of an existing datetime object.

Convert a UNIX timestamp to a datetime object with a time zone.

Raw log
```
time: 1562741899
```

SLS DSL orchestration

e_set("new_time", dt_parse(v("time"), tz="Asia/Shanghai"))

Processing result

time: 1562741899
new_time: 2019-07-10 06:58:19+08:00

Conversions between datetime strings and UNIX timestamps

Processing functions
- The dt_str flexible conversion function can convert a UNIX timestamp, datetime object, or datetime string to a datetime string in a specified format.
- The dt_strftimestamp function only supports converting a UNIX timestamp to a datetime string.
- The dt_parsetimestamp flexible conversion function can convert a datetime string or datetime object to a UNIX timestamp.
Scenario 1: Convert a datetime string without time zone information to a UNIX timestamp.
To convert a datetime string without time zone information, such as 2019-06-02 18:41:26, to a UNIX timestamp, you must specify a time zone. The resulting UNIX timestamp value depends on the specified time zone.
- Raw log
```
time: 2019-06-02 18:41:26
```
- SLS DSL orchestration
```
e_set("Shanghai_timestamp", dt_parsetimestamp(v("time"), tz="Asia/Shanghai"))
e_set("Los_Angeles_timestamp", dt_parsetimestamp(v("time"), tz="America/Los_Angeles"))
e_set("UTC_timestamp", dt_parsetimestamp(v("time")))
```
- Processing result
```
Shanghai_timestamp: 1559472086
Los_Angeles_timestamp: 1559526086
UTC_timestamp: 1559500886
time: 2019-06-02 18:41:26
```
Note
- tz="Asia/Shanghai" indicates that the time represented by the time field is in the Shanghai time zone.
- If you do not specify a time zone, the UTC time zone is used by default.
- For a list of all available time zone strings for the tz=time_zone_string parameter, see Time zones.

Scenario 2: Convert a datetime string with time zone information to a UNIX timestamp.

You do not need to specify the time zone parameter for a datetime string that already includes time zone information, such as 2019-06-02 18:41:26+08:00.

Raw log

China_time : 2019-06-02 18:41:26+08:00
America_time: 2019-06-02 3:41:26-07:00
UTC_time : 2019-06-02 10:41:26+00:00

SLS DSL orchestration

e_set("timestamp1", dt_parsetimestamp(v("China_time")))
e_set("timestamp2", dt_parsetimestamp(v("America_time")))
e_set("timestamp3", dt_parsetimestamp(v("UTC_time")))

Processing result

America_time:2019-06-02 3:41:26-07:00
China_time:2019-06-02 18:41:26+08:00
UTC_time:2019-06-02 10:41:26+00:00
timestamp1: 1559472086
timestamp2: 1559472086
timestamp3: 1559472086

Scenario 3: Convert a datetime string with a custom format and without a time zone to a UNIX timestamp.

Raw log

time1: 2019-07-10 06:58:19
time2: 2019/07/10 06-58-19

SLS DSL orchestration

e_set("time3", dt_parsetimestamp(v("time1")))   
e_set("time4", dt_parsetimestamp(dt_strptime(v("time2"), "%Y/%m/%d %H-%M-%S")))

Processing result

time1: 2019-07-10 06:58:19
time2: 2019/07/10 06-58-19
time3: 1562741899
time4: 1562741899

Conversions between datetime objects and datetime strings

Processing functions
- The dt_parse flexible conversion function can convert a datetime string or UNIX timestamp to a datetime object.
- The dt_astimezone function converts a datetime object to a different time zone.
Scenario 1: Convert a datetime string without time zone information to a datetime object in a specified time zone.
For a datetime string without time zone information, such as 2019-06-02 18:41:26, you can convert it to a datetime object in a different time zone. The following example shows how to interpret a datetime string as being in the Los Angeles time zone and then convert it to the Shanghai time zone.
- Raw log
```
# The time value in the time field is in the Los Angeles time zone.
time : 2019-06-04 2:41:26
```
- SLS DSL orchestration
```
e_set("timestamp", dt_parsetimestamp(v("time"), tz="America/Los_Angeles"))
e_set("Shanghai_time", dt_parse(v("timestamp"), tz="Asia/Shanghai"))
```
- Processed results
```
Shanghai_time : 2019-06-04 17:41:26+08:00
time : 2019-06-04 2:41:26
timestamp:1559641286
```

Scenario 2: Convert a datetime string without a time zone to a datetime object with a time zone.

Raw log
```
time : 2019-07-10 06:58:19
```

SLS DSL orchestration

e_set("new_time", dt_parse(v("time"), tz="Asia/Shanghai"))

Processing result

time: 2019-07-10 06:58:19
new_time: 2019-07-10 06:58:19+08:00

Scenario 3: Convert a datetime string with a time zone to a datetime object in a target time zone.

Raw log
```
time : 2019-06-04 2:41:26+08:00
```

SLS DSL orchestration

e_set("new_time", dt_astimezone(v("time"), tz="America/Los_Angeles"))

Processing result

new_time : 2019-06-03 11:41:26-07:00
time : 2019-06-04 2:41:26+08:00

Offset datetimes

Processing functions
- The parameters for the dt_add function are as follows:
```
dt_add(field_name, dt1=None, dt2=None, year(s)=None, month(s)=None, day(s)=None, hour(s)=None, minute(s)=None, second(s)=None, microsecond(s)=None, weeks(s)=None, weekday=None)
```
  The `(s)` at the end of parameters such as year(s), month(s), and day(s) indicates that these parameters can be in two forms, namely year and years, or month and months. For example, if you pass the year and years parameters: if you pass the year parameter, the value at the year granularity is overwritten with the value of the year parameter. If you pass the years parameter, the value at the year granularity is increased by the value of the years parameter. The dt_add function, which is used with these parameters, supports modifying (increasing, decreasing, or overwriting) a datetime value at a specific time granularity.
- In the dt_add function, the weekday parameter is typically used with parameters such as dt_MO and dt_TU to offset the date to a specific day of the week, as shown in the following example. For more information, see dt_MO.

Scenario 1: Offset a datetime by year and month.

The following example demonstrates how to offset a date by year and month.

Raw log
```
time1 : 2019-06-04 2:41:26
```

SLS DSL orchestration 1

e_set("time2", dt_add(v("time1"), year=2018))

Processing result 1

time1 : 2019-06-04 2:41:26
time2 : 2018-06-04 02:41:26

SLS DSL orchestration 2

e_set("time2", dt_add(v("time1"), years=2018))

Processing result 2

time1 : 2019-06-04 2:41:26
time2 : 4037-06-04 02:41:26

Scenario 2: Offset a datetime by week.

The following example shows a weekly date offset:

Raw log

# June 4, 2019 is a Tuesday.
 time1 : 2019-06-04 2:41:26

SLS DSL orchestration

# The date of the next Monday after time1.
e_set("nex_Monday", dt_add(v("time1"), weekday=dt_MO(1)))

# The date of the previous Tuesday before time1.
e_set("previous_Tuesday", dt_add(v("time1"), weekday=dt_TU(op_neg(1))))

# The date of the second Saturday after time1.
e_set("nex_next_Saturday", dt_add(v("time1"), weekday=dt_SA(2)))

# The date of the second to last Sunday before time1.
e_set("previous_previous_Sunday", dt_add(v("time1"), weekday=dt_SU(op_neg(2))))

Processing result

next_Monday : 2019-06-10 02:41:26
next_next_Saturday : 2019-06-15 02:41:26
previous_Tuesday : 2019-06-04 2:41:26
previous_previous_Sunday : 2019-05-26 02:41:26
time1 : 2019-06-04 2:41:26

Note

If the date in time1 is a Tuesday, offsetting to the previous Tuesday (for example, using an offset of -1) returns the date of time1 itself.