multi-shard transactions research

Description

PBM v1.0 does no detection of the MongoDB 4.2. transactions that are multi-shard, which need to be buffered and not applied until a "commitTransaction" is reached.

We need to document that v1.0 does not support them.

It does support transactions of the 4.0 style. This means the way they are in non-sharded replicasets (v4.2 too), and ones in 4.2 clusters that are limited to a single shard are the same oplog format, so those are OK.

Environment

None

Smart Checklist

Activity

Show:

Akira Kurogane October 28, 2019 at 12:46 PM

The test we'd like to do here is:

  1. Prepare 2+ shard cluster

  2. Create collection that is spread over 2+ shards. E.g.: use a shard key of the "_id" field; Insert doc with "_id": 99; and another with "_id": 101; sh.splitAt({"_id": 100}); sh.moveChunk("test.foo", {"_id": 101}, "shard2") (whatever the other shard is).

  3. Start a transaction, make it a multi-shard transaction by deleting both docs "_id": 99 and 101. (It could be inserts or updates, but I think deletes is simpler to check). But don't commit the transaction

  4. Without committing the transaction, stop and immediately make the backup. Important to finish the backup before the 60-second transaction timeout finishes.

  5. Let the 60 second expire.

  6. Run pbm restore.

  7. Allow 60 seconds to pass.

  8. Confirm the transaction writes didn't happen. E.g. if there were deletes confirm that docs with id 99 and 101 exist.

Akira Kurogane October 7, 2019 at 3:10 AM

N.b. We're not 100% sure that this doesn't already work. That is, we're not sure if (or if not) a v4.2 primary node that is having these "prepare": true oplog docs applied via the applyOps command will buffer rather than immediately apply them.

Two ways forward: read the code, or do the test.

Done

Details

Assignee

Reporter

Needs QA

Yes

Time tracking

30m logged

Fix versions

Priority

Smart Checklist

Created September 23, 2019 at 3:13 PM
Updated March 5, 2024 at 7:22 PM
Resolved January 15, 2020 at 4:47 PM

Flag notifications