Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Issues

Go to all issues
Select view

Select search mode

 
50 of 634

LP #1714630: pt-query-digest parses timestamps incorrectly from general log

Description

**Reported in Launchpad by Nickolay Ihalainen last update 09-01-2018 17:38:40

General log contains timestamps like: 2017-08-31T15:58:05.255051Z
my $genlog_line_1= qr{
\A
(?\d{6}\s+\d{1,2}:\d\d:\d\d|\d{4}\d{1,2}\d{1,2}T\d\d:\d\d:\d\d\.\d+Z))? # Timestamp
\s+
(?:\s*(\d+)) # Thread ID
\s
(\w+) # Command
\s+
(.*) # Argument
\Z
}xs;

It's not considered as proper TS or as mysql TS in parse_timestamp function and QPS always calculated as 0:
$mysql_ts = qr/(\d\d)(\d\d)(\d\d) (\d)\d+)\d+)(\.\d+)?/;
$proper_ts = qr/(\d\d\d\d)(\d\d)(\d\d)[T ](\d\d)\d\d)\d\d)(\.\d+)?/;

Suggested fix remove Z from genlog_line_1:
(?\d{6}\s+\d{1,2}:\d\d:\d\d|\d{4}\d{1,2}\d{1,2}T\d\d:\d\d:\d\d\.\d+))?Z? # Timestamp

/opt/mysql/product/mysql/bin/mysqld, Version: 5.7.18-15-29.20-log (Percona XtraDB Cluster binary (GPL) 5.7.18-29.20, Revision 7693d6e, wsrep_29.20). started with:
Tcp port: 3306 Unix socket: /opt/mysql/product/mysql/mysql3306.sock
Time Id Command Argument
2017-08-31T15:58:05.255051Z 2166396 Query SELECT 1
2017-08-31T15:59:05.255051Z 2166396 Query SELECT 1

Environment

None

Smart Checklist

Details

Assignee

Reporter

Priority

Fix versions

Labels

Smart Checklist

Created January 24, 2018 at 9:41 PM
Updated December 20, 2023 at 3:44 AM

Activity

Show:

lpjirasyncJanuary 24, 2018 at 9:42 PM

**Comment from Launchpad by: Carlos Salguero on: 09-01-2018 17:38:39

Was this fixed in the related issue?

lpjirasyncJanuary 24, 2018 at 9:42 PM

**Comment from Launchpad by: Nickolay Ihalainen on: 14-12-2017 04:23:42

related issue:
https://bugs.launchpad.net/percona-toolkit/+bug/1738128

lpjirasyncJanuary 24, 2018 at 9:42 PM

**Comment from Launchpad by: Nickolay Ihalainen on: 14-12-2017 03:42:31

Another example for general log timestamp in 5.7:
/usr/local/mysql/bin/mysqld, Version: 5.7.19-17-log (Percona Server (GPL), Release 17, Revision e19a6b7b73f). started with:
Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock
Time Id Command Argument
2017-12-08T14:56:57.252473-07:00 1 Query BEGIN

Works with:
my $genlog_line_1= qr{
\A
(?\d{6}\s+\d{1,2}:\d\d:\d\d|\d{4}\d{1,2}\d{1,2}T\d\d:\d\d:\d\d\.\d+(?:Z|-?\d\d:\d\d)))? # Timestamp
\s+
(?:\s*(\d+)) # Thread ID
\s
(\w+) # Command
\s+
(.*) # Argument
\Z
}xs;