Concurrent INSERT ... ON DUPLICATE KEY UPDATE statements could cause a failure with a unique index violation.

Description

##how to reproduce:

test table definition:

```sql
CREATE TABLE `extra` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`a` varchar(36) NOT NULL ,
`b` varchar(128) NOT NULL ,
`c` varchar(10240) NOT NULL ,
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_idx` (`a`,`b`),
) ENGINE=ROCKSDB
```

run several threads concurrently, each run the following logic loops for several times:

```sql
A: insert into extra(a, b, c) values (?,'b2','c2'),(?,'b3','c3'),(?,'b4','tc4'),(?,'b5','c5') on duplicate key update c=values(c);

B: insert into extra(a, b, c) values (?, 'b1', 'c1'), (?,'b2','c2'),(?,'b3','c3'),(?,'b4','tc4'),(?,'b5','c5') on duplicate key update c=values(c);
```

The value of field a is generated in random among different threads, but same in sql A and sql B in one loop.

Unexpected result:
in some rows, the value of field c is not c1 when b = b1

  1.  

    1. Dive into the code
      when sql B is running after sql A:(sql B has 5 rows to be inserted)

  • row 1 will be success

  • row 2,3,4,5 will be duplicated

since rocksdb reserve only one increment value each call(see ha_rocksdb::get_auto_increment), row 1 maybe get auto incr value 100, row 2 maybe get auto inc value 110(values between 100 and 110 fetched by other threads).

when row 2 is duplicated in sql_insert.cc:write_record,
table->file->restore_auto_increment(prev_insert_id); will be called, and prev_insert_id = 100;

then when row 3 fetch auto incr (handler::update_auto_increment), next_insert_id is 101, and auto_inc_interval_for_cur_row.maximum() is 110(since row 2 called), so the logic will be incorrenctly, 101 will be used by row 3.

since 101 is used by other thread, row 3 using this auto inc val will be duplicated on the auto inc key incorrectly, and row 3 will update the row whose auto inc val is 101.

this will lead to data updated incorrectly.

  1.  

    1. innodb
      this case will not happen with innodb, because innodb will reserve the auto incr values needed, Since sql B has 5 rows, 100 ~ 104 will be reserved by this thread.

Environment

None

Attachments

2
  • 20 Aug 2019, 10:29 AM
  • 20 Aug 2019, 10:29 AM

Smart Checklist

Activity

Show:

George Lorch January 20, 2020 at 8:45 PM

Fixed upstream:

Manuel November 22, 2019 at 4:54 AM

The rocksdb test case is very easy. Just replace engine=innodb with engine=rocksdb.

In any case, the FB fix for this issue is here:
https://github.com/facebook/mysql-5.6/commit/9a595a5048d243cc9ebe28042f6d049fd1bf9e8b
https://github.com/facebook/mysql-5.6/commit/03919677cb5bf34997e13c384b44dc3cccd41be9
https://github.com/facebook/mysql-5.6/commit/03919677cb5bf34997e13c384b44dc3cccd41be9

Note that this isn't an engine bug, which is why the fix is in sql/sql_insert.cc.

George Lorch November 21, 2019 at 10:52 PM

Incomplete and no activity for > 90 days, please request to re-open if you obtain more information of believe this is in error.

Lalit Choudhary August 20, 2019 at 10:33 AM

The described issue is reproduciable for Innodb storage engine and .7.26

 

 

 

Lalit Choudhary August 20, 2019 at 10:24 AM

Hello

Thank you for the update. I'm able to reproduce the described issue with mysql#96578  testcase for PS 5.7 and for 8.0 it's crashing.

Also if when I'm using the same test with storage engine rocksdb for tables, it crashes for both 5.7 and 8.0 version. 

 

 

 

 

Could please provide us a working test case for rocksdb storage engine.

 

 

Done

Details

Assignee

Reporter

Labels

Components

Priority

Smart Checklist

Created June 4, 2019 at 9:38 AM
Updated March 6, 2024 at 12:06 PM
Resolved January 20, 2020 at 8:45 PM