Investigate the effects of having incomplete transactions in backup oplog slices

Description

From https://docs.mongodb.com/master/release-notes/4.2/#backups:

Backups
Starting in MongoDB 4.2, mongodump and mongorestore cannot be part of a backup strategy for sharded clusters. These manual tools do not maintain the atomicity guarantees of transactions across shards.

To maintain the atomicity guarantees of transactions across shards, use the coordinated backup and restore services provided by:
( list of the three MongoDB commercial cloud products )

So it appears that MongoDB upstream is not going to modify mongorestore to handle transactions oplog ops, or maybe it's just vaguely pointing out mongorestore is going to restore one replica set but isn't aimed at synchronizing that in the multiple replicasets across a cluster.

The task of this ticket is to make a test case with some slow-running transactions that make writes on multiple shards, and make PBM backups that finish with a final oplog slice time in the middle of those transactions.

Work on the 4.2 case first. The 4.0 situation used a different format in the oplog which we can now know is transitional design in that version only. (Compatibility notes link regarding the oplog format.)

Environment

None

Smart Checklist

Activity

Akira Kurogane October 28, 2019 at 12:59 PM

Closing this ticket as duplicate

Akira Kurogane October 7, 2019 at 6:36 AM

N.b. We're not 100% sure that this doesn't already work. That is, we're not sure if (or if not) a v4.2 primary node that is having these "prepare": true oplog docs applied via the applyOps command will buffer rather than immediately apply them.

Two ways forward: read the code, or do the test.

Duplicate

Details

Assignee

Reporter

Priority

Smart Checklist

Created August 15, 2019 at 9:39 PM
Updated March 5, 2024 at 7:23 PM
Resolved October 28, 2019 at 1:01 PM

Flag notifications