Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Resync does not always update bucket resulting in faulty list and backups to wrong buckets

Description

I started with a `pbm` config of:

```

pitr:
  enabled: false
  oplogSpanMin: 0
  compression: s2
storage:
  type: s3
  s3:
    region: us-west-2
    bucket: pbm-test-bucket-1
    prefix: data/pbm/backup
    credentials:
      access-key-id: '***'
      secret-access-key: '***'
    insecureSkipTLSVerify: false

```

 

After changing the bucket and access-key and secret key of pbm with `pbm config --set` and re-syncing with `pbm config --force-resync` and waiting for `pbm status` to report that it is done syncing, `pbm status` reports:

```

Cluster:

========

mongodb:

  - mongodb/10.195.41.141:27017 [P]: pbm-agent v2.0.2 OK

 

 

PITR incremental backup:

========================

Status [OFF]

 

Currently running:

==================

(none)

 

Backups:

========

S3 us-west-2 s3://pbm-test-bucket-3/data/pbm/backup

  Snapshots:

    2023-01-27T14:02:27Z 35.11KB <logical> [restore_to_time: 2023-01-27T14:02:33Z]

    2023-01-27T13:57:44Z 27.99KB <logical> [restore_to_time: 2023-01-27T13:57:49Z]

    2023-01-27T13:55:56Z 26.86KB <logical> [restore_to_time: 2023-01-27T13:56:01Z]

    2023-01-26T16:11:44Z 243.17KB <logical> [restore_to_time: 2023-01-26T16:11:49Z]

    2023-01-26T15:01:57Z 20.42KB <logical> [restore_to_time: 2023-01-26T15:02:02Z]

```

The snapshots listed here are incorrect and are instead the contents of my `pbm-test-bucket-1`, which `pbm` was previously configured to (verified by viewing s3 storage in AWS for both  `pbm-test-bucket-1` and `pbm-test-bucket-3`.) Even when I do `pbm backup`  the backup goes to `pbm-test-bucket-1` and *not* `pbm-test-bucket-3` 

 

Only when I `pbm config --force-resync` a *second* time does `pbm status`  correctly resync and report the correct snapshots:

```

Cluster:

========

mongodb:

  - mongodb/10.195.41.141:27017 [P]: pbm-agent v2.0.2 OK

 

 

PITR incremental backup:

========================

Status [OFF]

 

Currently running:

==================

(none)

 

Backups:

========

S3 us-west-2 s3://pbm-test-bucket-3/data/pbm/backup

  Snapshots:

    2023-01-26T16:17:11Z 232.79KB <logical> [restore_to_time: 2023-01-26T16:17:17Z]

```

Environment

Ubuntu Focal

Activity

Mia Altieri 
February 2, 2023 at 9:46 AM

I did some digging in the logs and saw:
```
2023-02-02T09:08:36Z D [mongodb/10.195.41.8:27017] [resync/63db7d93a41d22a0d5d2d9a1] get lock: another operation is running: Snapshot backup '63db7d8a5fe581c00bfe0135'
2023-02-02T09:08:36Z D [mongodb/10.195.41.8:27017] [resync/63db7d93a41d22a0d5d2d9a1] lock not acquired
```
My guess is that the re-sync didn’t occur because the backup didn’t occur but `pbm` updated the name of the bucket anyway which is why `pbm status` shows the new bucket name with the old bucket contents. I haven’t looked at the code and admittedly my go is quite rusty . But I think there are three decent ways to fix this:
1. Don’t update the bucket in `pbm status` if a resync was unsuccessful
2. Retry on failed resins

For now I will check that `pbm status` isn’t running any operations before doing `pbm config --force-resync`

Here is the output of the log messages I got earlier today

```
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] pbm-agent:
Version: 2.0.2
Platform: linux/amd64
GitCommit:
GitBranch:
BuildTime:
GoVersion: go1.18.8
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] starting PITR routine
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] node: mongodb/10.195.41.8:27017
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] listening for the commands
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328650>
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] got epoch {1675328649 14}
2023-02-02T09:04:10Z I [mongodb/10.195.41.8:27017] [resync/63db7c8abe466738cf4730ba] started
2023-02-02T09:04:11Z E [mongodb/10.195.41.8:27017] [resync/63db7c8abe466738cf4730ba] init storage: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: W3S1TC2591P0PACB, host id: g13gUyPWXXmJSUf/tLCJzdK/JFeRJq+vkPvOWPKSSilvVdNhmbfqDHfQeFr8gJv/Bc/YNiidsxg=
2023-02-02T09:04:15Z W [mongodb/10.195.41.8:27017] [agentCheckup/] get current storage status: query mongo: mongo: no documents in result
2023-02-02T09:04:16Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: 281VZRHQ4TE3AEQN, host id: W7SExU1ZmFsYsq9eI5jioPLrQXxkdvoEzYUcxOKioj6fYyjtpC+YADdwymn12oeU8VGt1MmqHSY=
2023-02-02T09:04:21Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: V6150X1HAQ4M559V, host id: MVItN3cetyh1YxrkMkrwEkltz0DDXcw6jPB5PgFB14J1wdnEu22jifcUi83HTwlBwd/CG4TASoo=
2023-02-02T09:04:25Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: unable to get storage: storage undefined
2023-02-02T09:04:25Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328665>
2023-02-02T09:04:25Z I [mongodb/10.195.41.8:27017] got epoch {1675328661 2}
2023-02-02T09:04:25Z I [mongodb/10.195.41.8:27017] [resync/63db7c996efd669f109dbd24] started
2023-02-02T09:04:25Z E [mongodb/10.195.41.8:27017] [resync/63db7c996efd669f109dbd24] unable to get backup store: storage undefined
2023-02-02T09:04:26Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328666>
2023-02-02T09:04:26Z I [mongodb/10.195.41.8:27017] got epoch {1675328661 2}
2023-02-02T09:04:26Z I [mongodb/10.195.41.8:27017] [resync/63db7c9a08721a53e5d65c78] started
2023-02-02T09:04:27Z E [mongodb/10.195.41.8:27017] [resync/63db7c9a08721a53e5d65c78] init storage: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: N3G73DJYV5KX757C, host id: cAmdQtrqMJZwVOKXC/0p/bQIswgMVkCx7G8DbIOzdYdNFFfYBtsVRMSge4XL+clizipBJZ+/Hfw=
2023-02-02T09:04:31Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: QMSKP8XET3N1VT2K, host id: 47FvfaNS8k622EiyrxTD7kpIipUWzXXB/A339lbKBQqFFFj/F6sGo3knqP5iodCgUj2g+MlNqsw=
2023-02-02T09:04:36Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: J39GH86F7X0CBZAM, host id: OhDj58bGVSdrl4HuF8JedNRbmdHdtJ9O4283TBjTHjBaegPsgCnI6KnsCEFsqf8ptpbwaWNuvR0=
2023-02-02T09:04:41Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: 045AM21P1HST6BCK, host id: KgL8Wdbs1SAda1aoEigKkC1Khfm5X95KmaohQilL3UOmEip2N5GGpYTbSRhmM9CANVHzXqffgr8=
2023-02-02T09:04:46Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: PY6V2HSAMTNY0N5H, host id: YnK3qTUmcFyjqcMA+RB0IU9mTN8ocr1cJXyO2aM8bRrJQ02UWo2QP/GFY2aL7BRGzc+J5y3pJ94=
2023-02-02T09:04:51Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: 5QME5R3VCH9VHYMD, host id: k15J6IvTrtUV+Nc23CWZ+QQ7fd6OBkBY0dJYneHhJLGuEhT9IgLFkOJ1JOpDdFgLwO02x0PMqt0=
2023-02-02T09:04:56Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: VPBGDRG8DDCZ5MQP, host id: 4GmSNKxXXm/gU+5X5dpD0DUgbFcgxWIfikq7sZdN6BiioeYh5E2IrsGyAd6EBQYTR+K/rKS3xNo=
2023-02-02T09:05:01Z E [mongodb/10.195.41.8:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
status code: 404, request id: GV0P82S48K8JHY6D, host id: ybKy1lvRTTQQyoe+aHXcnmNt+40cycKl6HOn4DofcUW+lXR75/KATfmEkYH18fctop8yhY1yGbg=
2023-02-02T09:05:04Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328704>
2023-02-02T09:05:04Z I [mongodb/10.195.41.8:27017] got epoch {1675328701 2}
2023-02-02T09:05:04Z I [mongodb/10.195.41.8:27017] [resync/63db7cc0f9f0079f2d410a91] started
2023-02-02T09:05:04Z E [mongodb/10.195.41.8:27017] [resync/63db7cc0f9f0079f2d410a91] unable to get backup store: storage undefined
2023-02-02T09:05:05Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328705>
2023-02-02T09:05:05Z I [mongodb/10.195.41.8:27017] got epoch {1675328701 2}
2023-02-02T09:05:05Z I [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] started
2023-02-02T09:05:06Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] got physical restores list: 0
2023-02-02T09:05:07Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] got backups list: 19
2023-02-02T09:05:07Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-02-02T09:05:08Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-02-02T09:05:09Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-27T13:55:56Z.pbm.json
2023-02-02T09:05:10Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-27T13:57:44Z.pbm.json
2023-02-02T09:05:11Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-27T14:02:27Z.pbm.json
2023-02-02T09:05:12Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-31T09:31:13Z.pbm.json
2023-02-02T09:05:13Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-31T09:38:11Z.pbm.json
2023-02-02T09:05:15Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-01-31T17:04:57Z.pbm.json
2023-02-02T09:05:16Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T09:19:23Z.pbm.json
2023-02-02T09:05:17Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T09:23:36Z.pbm.json
2023-02-02T09:05:18Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T10:28:42Z.pbm.json
2023-02-02T09:05:19Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T11:04:59Z.pbm.json
2023-02-02T09:05:20Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T11:06:23Z.pbm.json
2023-02-02T09:05:21Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T14:50:15Z.pbm.json
2023-02-02T09:05:22Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T14:51:15Z.pbm.json
2023-02-02T09:05:23Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T16:45:53Z.pbm.json
2023-02-02T09:05:24Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T16:48:20Z.pbm.json
2023-02-02T09:05:26Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T16:48:33Z.pbm.json
2023-02-02T09:05:27Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] bcp: 2023-02-01T16:56:33Z.pbm.json
2023-02-02T09:05:28Z I [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] succeed
2023-02-02T09:05:28Z D [mongodb/10.195.41.8:27017] [resync/63db7cc1eacbb95288684f33] epoch set to {1675328728 20}
2023-02-02T09:05:33Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328732>
2023-02-02T09:05:33Z I [mongodb/10.195.41.8:27017] got epoch {1675328728 20}
2023-02-02T09:05:33Z I [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] started
2023-02-02T09:05:34Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] got physical restores list: 0
2023-02-02T09:05:34Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] got backups list: 19
2023-02-02T09:05:34Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-02-02T09:05:35Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-02-02T09:05:36Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-27T13:55:56Z.pbm.json
2023-02-02T09:05:37Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-27T13:57:44Z.pbm.json
2023-02-02T09:05:38Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-27T14:02:27Z.pbm.json
2023-02-02T09:05:40Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-31T09:31:13Z.pbm.json
2023-02-02T09:05:41Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-31T09:38:11Z.pbm.json
2023-02-02T09:05:42Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-01-31T17:04:57Z.pbm.json
2023-02-02T09:05:43Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T09:19:23Z.pbm.json
2023-02-02T09:05:44Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T09:23:36Z.pbm.json
2023-02-02T09:05:45Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T10:28:42Z.pbm.json
2023-02-02T09:05:46Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T11:04:59Z.pbm.json
2023-02-02T09:05:47Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T11:06:23Z.pbm.json
2023-02-02T09:05:48Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T14:50:15Z.pbm.json
2023-02-02T09:05:49Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T14:51:15Z.pbm.json
2023-02-02T09:05:50Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T16:45:53Z.pbm.json
2023-02-02T09:05:51Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T16:48:20Z.pbm.json
2023-02-02T09:05:52Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T16:48:33Z.pbm.json
2023-02-02T09:05:53Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] bcp: 2023-02-01T16:56:33Z.pbm.json
2023-02-02T09:05:55Z I [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] succeed
2023-02-02T09:05:55Z D [mongodb/10.195.41.8:27017] [resync/63db7cdc52c2af5f4b346870] epoch set to {1675328755 20}
2023-02-02T09:06:22Z I [mongodb/10.195.41.8:27017] got command backup [name: 2023-02-02T09:06:22Z, compression: s2 (level: default)] <ts: 1675328782>
2023-02-02T09:06:22Z I [mongodb/10.195.41.8:27017] got epoch {1675328755 20}
2023-02-02T09:06:22Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] init backup meta
2023-02-02T09:06:22Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] nomination list for mongodb: [[10.195.41.8:27017]]
2023-02-02T09:06:22Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] nomination mongodb, set candidates [10.195.41.8:27017]
2023-02-02T09:06:23Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] backup started
2023-02-02T09:06:26Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] wait for tmp users {1675328786 5}
2023-02-02T09:06:27Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] mongodump finished, waiting for the oplog
2023-02-02T09:06:30Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] set oplog span to {1675328783 5} / {1675328787 2}
2023-02-02T09:06:30Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] dropping tmp collections
2023-02-02T09:06:30Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] epoch set to {1675328790 7}
2023-02-02T09:06:32Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] backup finished
2023-02-02T09:06:32Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:06:22Z/63db7d0e7f521dce1294ba66] releasing lock
2023-02-02T09:07:06Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328826>
2023-02-02T09:07:06Z I [mongodb/10.195.41.8:27017] got epoch {1675328825 1}
2023-02-02T09:07:06Z I [mongodb/10.195.41.8:27017] [resync/63db7d3a3790a2fe2867a980] started
2023-02-02T09:07:06Z E [mongodb/10.195.41.8:27017] [resync/63db7d3a3790a2fe2867a980] init storage: get S3 object header: InvalidParameter: 1 validation error(s) found.

  • minimum field size of 1, HeadObjectInput.Bucket.

2023-02-02T09:07:07Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328827>
2023-02-02T09:07:07Z I [mongodb/10.195.41.8:27017] got epoch {1675328825 1}
2023-02-02T09:07:07Z I [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] started
2023-02-02T09:07:08Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] got physical restores list: 0
2023-02-02T09:07:09Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] got backups list: 8
2023-02-02T09:07:09Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-01-26T16:17:11Z.pbm.json
2023-02-02T09:07:10Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-01-27T14:21:24Z.pbm.json
2023-02-02T09:07:11Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-01-30T14:56:49Z.pbm.json
2023-02-02T09:07:12Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-02-01T10:31:03Z.pbm.json
2023-02-02T09:07:14Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-02-01T11:06:48Z.pbm.json
2023-02-02T09:07:15Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-02-01T16:47:40Z.pbm.json
2023-02-02T09:07:16Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-02-01T16:48:07Z.pbm.json
2023-02-02T09:07:17Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] bcp: 2023-02-01T16:55:52Z.pbm.json
2023-02-02T09:07:18Z I [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] succeed
2023-02-02T09:07:18Z D [mongodb/10.195.41.8:27017] [resync/63db7d3b14d4361bdd4d53f8] epoch set to {1675328838 9}
2023-02-02T09:07:22Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328842>
2023-02-02T09:07:22Z I [mongodb/10.195.41.8:27017] got epoch {1675328838 9}
2023-02-02T09:07:22Z I [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] started
2023-02-02T09:07:23Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] got physical restores list: 0
2023-02-02T09:07:23Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] got backups list: 8
2023-02-02T09:07:23Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-01-26T16:17:11Z.pbm.json
2023-02-02T09:07:24Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-01-27T14:21:24Z.pbm.json
2023-02-02T09:07:25Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-01-30T14:56:49Z.pbm.json
2023-02-02T09:07:26Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-02-01T10:31:03Z.pbm.json
2023-02-02T09:07:27Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-02-01T11:06:48Z.pbm.json
2023-02-02T09:07:29Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-02-01T16:47:40Z.pbm.json
2023-02-02T09:07:30Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-02-01T16:48:07Z.pbm.json
2023-02-02T09:07:31Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] bcp: 2023-02-01T16:55:52Z.pbm.json
2023-02-02T09:07:32Z I [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] succeed
2023-02-02T09:07:32Z D [mongodb/10.195.41.8:27017] [resync/63db7d4acab80305481dc1d5] epoch set to {1675328852 9}
2023-02-02T09:07:59Z I [mongodb/10.195.41.8:27017] got command backup [name: 2023-02-02T09:07:59Z, compression: s2 (level: default)] <ts: 1675328879>
2023-02-02T09:07:59Z I [mongodb/10.195.41.8:27017] got epoch {1675328852 9}
2023-02-02T09:07:59Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] init backup meta
2023-02-02T09:07:59Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] nomination list for mongodb: [[10.195.41.8:27017]]
2023-02-02T09:08:00Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] nomination mongodb, set candidates [10.195.41.8:27017]
2023-02-02T09:08:00Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] backup started
2023-02-02T09:08:03Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] wait for tmp users {1675328883 5}
2023-02-02T09:08:04Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] mongodump finished, waiting for the oplog
2023-02-02T09:08:07Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328886>
2023-02-02T09:08:07Z I [mongodb/10.195.41.8:27017] got epoch {1675328885 5}
2023-02-02T09:08:07Z D [mongodb/10.195.41.8:27017] [resync/63db7d769e3223744558fa79] get lock: another operation is running: Snapshot backup '63db7d6fbdb858347aad3a95'
2023-02-02T09:08:07Z D [mongodb/10.195.41.8:27017] [resync/63db7d769e3223744558fa79] lock not acquired
2023-02-02T09:08:07Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] set oplog span to {1675328880 8} / {1675328884 2}
2023-02-02T09:08:07Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] dropping tmp collections
2023-02-02T09:08:07Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] epoch set to {1675328887 12}
2023-02-02T09:08:08Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328887>
2023-02-02T09:08:08Z I [mongodb/10.195.41.8:27017] got epoch {1675328887 12}
2023-02-02T09:08:08Z D [mongodb/10.195.41.8:27017] [resync/63db7d77ced1c7e4d404560f] get lock: another operation is running: Snapshot backup '63db7d6fbdb858347aad3a95'
2023-02-02T09:08:08Z D [mongodb/10.195.41.8:27017] [resync/63db7d77ced1c7e4d404560f] lock not acquired
2023-02-02T09:08:10Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] backup finished
2023-02-02T09:08:10Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:07:59Z/63db7d6fbdb858347aad3a95] releasing lock
2023-02-02T09:08:11Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328890>
2023-02-02T09:08:11Z I [mongodb/10.195.41.8:27017] got epoch {1675328887 12}
2023-02-02T09:08:11Z I [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] started
2023-02-02T09:08:11Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] got physical restores list: 0
2023-02-02T09:08:11Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] got backups list: 9
2023-02-02T09:08:11Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-01-26T16:17:11Z.pbm.json
2023-02-02T09:08:12Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-01-27T14:21:24Z.pbm.json
2023-02-02T09:08:13Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-01-30T14:56:49Z.pbm.json
2023-02-02T09:08:14Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-02-01T10:31:03Z.pbm.json
2023-02-02T09:08:15Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-02-01T11:06:48Z.pbm.json
2023-02-02T09:08:17Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-02-01T16:47:40Z.pbm.json
2023-02-02T09:08:18Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-02-01T16:48:07Z.pbm.json
2023-02-02T09:08:19Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-02-01T16:55:52Z.pbm.json
2023-02-02T09:08:20Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] bcp: 2023-02-02T09:07:59Z.pbm.json
2023-02-02T09:08:21Z I [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] succeed
2023-02-02T09:08:21Z D [mongodb/10.195.41.8:27017] [resync/63db7d7a70722281083702b6] epoch set to {1675328901 11}
2023-02-02T09:08:27Z I [mongodb/10.195.41.8:27017] got command backup [name: 2023-02-02T09:08:26Z, compression: s2 (level: default)] <ts: 1675328906>
2023-02-02T09:08:27Z I [mongodb/10.195.41.8:27017] got epoch {1675328901 11}
2023-02-02T09:08:27Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] init backup meta
2023-02-02T09:08:27Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] nomination list for mongodb: [[10.195.41.8:27017]]
2023-02-02T09:08:27Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] nomination mongodb, set candidates [10.195.41.8:27017]
2023-02-02T09:08:27Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] backup started
2023-02-02T09:08:30Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] wait for tmp users {1675328910 6}
2023-02-02T09:08:31Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] mongodump finished, waiting for the oplog
2023-02-02T09:08:32Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328911>
2023-02-02T09:08:32Z I [mongodb/10.195.41.8:27017] got epoch {1675328910 7}
2023-02-02T09:08:32Z D [mongodb/10.195.41.8:27017] [resync/63db7d8fea2709b95c000ee7] get lock: another operation is running: Snapshot backup '63db7d8a5fe581c00bfe0135'
2023-02-02T09:08:32Z D [mongodb/10.195.41.8:27017] [resync/63db7d8fea2709b95c000ee7] lock not acquired
2023-02-02T09:08:33Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328912>
2023-02-02T09:08:33Z I [mongodb/10.195.41.8:27017] got epoch {1675328910 7}
2023-02-02T09:08:33Z D [mongodb/10.195.41.8:27017] [resync/63db7d90698dd858fed6afd0] get lock: another operation is running: Snapshot backup '63db7d8a5fe581c00bfe0135'
2023-02-02T09:08:33Z D [mongodb/10.195.41.8:27017] [resync/63db7d90698dd858fed6afd0] lock not acquired
2023-02-02T09:08:34Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] set oplog span to {1675328907 14} / {1675328911 12}
2023-02-02T09:08:35Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] dropping tmp collections
2023-02-02T09:08:35Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] epoch set to {1675328915 4}
2023-02-02T09:08:36Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675328915>
2023-02-02T09:08:36Z I [mongodb/10.195.41.8:27017] got epoch {1675328915 4}
2023-02-02T09:08:36Z D [mongodb/10.195.41.8:27017] [resync/63db7d93a41d22a0d5d2d9a1] get lock: another operation is running: Snapshot backup '63db7d8a5fe581c00bfe0135'
2023-02-02T09:08:36Z D [mongodb/10.195.41.8:27017] [resync/63db7d93a41d22a0d5d2d9a1] lock not acquired
2023-02-02T09:08:37Z I [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] backup finished
2023-02-02T09:08:37Z D [mongodb/10.195.41.8:27017] [backup/2023-02-02T09:08:26Z/63db7d8a5fe581c00bfe0135] releasing lock
2023-02-02T09:17:01Z I [mongodb/10.195.41.8:27017] got command resync <ts: 1675329421>
2023-02-02T09:17:01Z I [mongodb/10.195.41.8:27017] got epoch {1675328915 4}
2023-02-02T09:17:01Z I [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] started
2023-02-02T09:17:02Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] got physical restores list: 0
2023-02-02T09:17:03Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] got backups list: 20
2023-02-02T09:17:03Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-02-02T09:17:04Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-02-02T09:17:05Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-27T13:55:56Z.pbm.json
2023-02-02T09:17:06Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-27T13:57:44Z.pbm.json
2023-02-02T09:17:07Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-27T14:02:27Z.pbm.json
2023-02-02T09:17:08Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-31T09:31:13Z.pbm.json
2023-02-02T09:17:10Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-31T09:38:11Z.pbm.json
2023-02-02T09:17:11Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-01-31T17:04:57Z.pbm.json
2023-02-02T09:17:12Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T09:19:23Z.pbm.json
2023-02-02T09:17:13Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T09:23:36Z.pbm.json
2023-02-02T09:17:14Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T10:28:42Z.pbm.json
2023-02-02T09:17:15Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T11:04:59Z.pbm.json
2023-02-02T09:17:16Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T11:06:23Z.pbm.json
2023-02-02T09:17:17Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T14:50:15Z.pbm.json
2023-02-02T09:17:18Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T14:51:15Z.pbm.json
2023-02-02T09:17:19Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T16:45:53Z.pbm.json
2023-02-02T09:17:20Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T16:48:20Z.pbm.json
2023-02-02T09:17:21Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T16:48:33Z.pbm.json
2023-02-02T09:17:22Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-01T16:56:33Z.pbm.json
2023-02-02T09:17:23Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] bcp: 2023-02-02T09:06:22Z.pbm.json
2023-02-02T09:17:25Z I [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] succeed
2023-02-02T09:17:25Z D [mongodb/10.195.41.8:27017] [resync/63db7f8debf8395d9c4ad5e5] epoch set to {1675329445 21}
```

Mia Altieri 
February 2, 2023 at 9:45 AM

Update - let me split up my comment it might be too much text

Hi Andrew I encountered this problem again today when changing my bucket from `pbm-bucket-3` to `pbm-bucket-1`. Like before `pbm status` showed the bucket name `pbm-bucket-1`, but listed the backups in `pbm-bucket-3` ie:
```
Cluster:
========
mongodb:

  • mongodb/10.195.41.8:27017 [P]: pbm-agent v2.0.2 OK

PITR incremental backup:
========================
Status [OFF]

Currently running:
==================
(none)

Backups:
========
S3 us-west-2 s3://pbm-test-bucket-1/data/pbm/backup
Snapshots:
<backups in pbm bucket 3>
```

Mia Altieri 
February 2, 2023 at 9:44 AM

Hi Andrew I have some insight that I would like to share but for some reason when I try to add my comment it does not appear on this thread.

Mia Altieri 
January 27, 2023 at 3:33 PM

Hi Andrew, yes here are the logs. Thanks for the help on this

```

ubuntu@juju-109b00-1:~$ percona-backup-mongodb logs -sD -t0 -x
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] pbm-agent:
Version:   2.0.2
Platform:  linux/amd64
GitCommit:
GitBranch:
BuildTime:
GoVersion: go1.18.8
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] starting PITR routine
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] node: mongodb/10.195.41.141:27017
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] listening for the commands
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827187>
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] got epoch {1674827186 14}
2023-01-27T13:46:27Z I [mongodb/10.195.41.141:27017] [resync/63d3d5b32ebe652bdc78fd05] started
2023-01-27T13:46:28Z E [mongodb/10.195.41.141:27017] [resync/63d3d5b32ebe652bdc78fd05] init storage: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: Q9TRMR1SWJ9YHV5G, host id: +6+QZOddrKNYIJdoYdgJkxHvyvLveuR4WYm8eiefR68r3kgBePjDDl9+Mx4eLWlvbZvRWTP5H00=
2023-01-27T13:46:32Z W [mongodb/10.195.41.141:27017] [agentCheckup/] get current storage status: query mongo: mongo: no documents in result
2023-01-27T13:46:33Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: RFFC0YNP8D7P0F47, host id: uNepttOVrxQXevfWnTVXTwkZLWnju+T3JQHTbBA6vlhgx62Q4DDaeAAdFSntjvJV9CmdteUIsDg=
2023-01-27T13:46:38Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 68V53KBMZ8EWBJDE, host id: JZhmOPki6H9SJe4jW1G8RsSwZ/tF0NkmHDJUMNSbNZcRJirzcev123ANrgCZP6pMFh8pC6oCX88=
2023-01-27T13:46:43Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 7MVX5YQ7301YAX9W, host id: qDAldMb4fS8+bumqBEbPWsggv6h3UWAFEvNhNHrhQeTQsavXbiFu27ywAvVZ1G/12DGRg5xu0AI=
2023-01-27T13:46:47Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: A22568RYYRCQ845K, host id: mii4pDFSxnlX1e8nLIDHjVOWU+LPp8JG39mlVpQVGSUNpsYtWKOpwDQSN9dSEG9ZQW0Ekqiychs=
2023-01-27T13:46:52Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: QETE1NKSMCNSFD5S, host id: hOJywq8GFqcUAu+EENZyXof41V5hIFZmX8bz8QufV+L4BN2fdx+vXs++6XBH/Xz2ct1OgLSD9r4=
2023-01-27T13:46:57Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: ZPFKM267JZWRKK45, host id: 9hAcnYVAq/DISn5vHprmBwoqhxdR2/+toLulb4KyHYyofaG9VoFrrygJA0uK5jc3Q+Zs1UL/+ZE=
2023-01-27T13:47:03Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 252TARX44X1Y446E, host id: DJSAQ1sZeCiGKYQn+VzeqvMXSdROBhValnBpkhbJ63KPxcWp5JwKcrVJyxlZiNvCZplpsAkk1ns=
2023-01-27T13:47:07Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: C7FFXNGN156K6CJK, host id: 3nid6l3SXzJy06vnZWzfD6Isb4aKplsCAnOIB6Txf/St/Oi+suT721emXsFv5kVW9RagWiaTCP01IJt9asrbyg==
2023-01-27T13:47:08Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827227>
2023-01-27T13:47:08Z I [mongodb/10.195.41.141:27017] got epoch {1674827223 2}
2023-01-27T13:47:08Z I [mongodb/10.195.41.141:27017] [resync/63d3d5db55954e9aa443017b] started
2023-01-27T13:47:11Z E [mongodb/10.195.41.141:27017] [resync/63d3d5db55954e9aa443017b] init storage: get S3 object header: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2023-01-27T13:47:11Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827228>
2023-01-27T13:47:11Z I [mongodb/10.195.41.141:27017] got epoch {1674827223 2}
2023-01-27T13:47:11Z I [mongodb/10.195.41.141:27017] [resync/63d3d5dca60acce366be0725] started
2023-01-27T13:47:12Z E [mongodb/10.195.41.141:27017] [resync/63d3d5dca60acce366be0725] init storage: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 4ETYRNQS0J45XQ0K, host id: 4mQ3CCdy2GUUENoxAvqVpxgfxFtpkDm/Ktz/GjM/g0fKvUQ8xYhu8LTs4pO6PMPujAwDVDUe0KY=
2023-01-27T13:47:13Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 5E9XBEKT7W9JAWF7, host id: 4gm0gkS2MRHyGU0jrDORTyV+09IEuseNMSEPZLIZGYFjV0DloH6PkV9Svv9rxg/6zH1vobugqho=
2023-01-27T13:47:18Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: ZZZ911GCHV9QTTHE, host id: hww3LKtOQqCTcSFIIhh3+17kd32U8z3wIYf4cAa/xg1tQDEBMqOfY/h1JzgZwTABZ4Vk5VoHy98=
2023-01-27T13:47:23Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: WKBM5TQSF9332M9E, host id: b7uu4WmVvLNF/xAqjBE0FCGYl6Oeu1mr0sN/LZtCTs6zNpDBSXYfb3WW0dgM2CL2IPVqzybUMSY=
2023-01-27T13:47:28Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: T7NB6DXS0SNGY5VB, host id: AM0iCdpbVWz63GQl6zhM7f9aDg2X+twPQ3eTSLl2vr5X8U9rtLkIrNz5dhwOs4ioZvTABkj1j+M=
2023-01-27T13:47:33Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: M74V0DQWQFEFG69D, host id: 6aVNbTYtUJL6QT1st4XyN9uKst17y8gwc31ZNqoeSDuWaWCZ4KLBcXkqw9M8nlrRk+ssMcAi58s=
2023-01-27T13:47:38Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: V5JRJZJ3VPWCWCPB, host id: F+kBJDSVrAbx5zHPmqoTtxt+Kjuhhq/pieY6+BJTaI/5omIJO0qTqwoukCjHwEbkeikiAODM5PE=
2023-01-27T13:47:43Z E [mongodb/10.195.41.141:27017] [agentCheckup/] check storage connection: storage: no init file, attempt to create failed: upload to S3: NoSuchBucket: The specified bucket does not exist
    status code: 404, request id: 83MHWBY9T7S32FGJ, host id: side78yN0oyQaKfJ8m4nZDbhit46iDJcbcQog9w/AlHBxpnuMiIT0ofRqbDcGnYi/ahw8H6Lef8=
2023-01-27T13:47:43Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827263>
2023-01-27T13:47:43Z I [mongodb/10.195.41.141:27017] got epoch {1674827258 2}
2023-01-27T13:47:43Z I [mongodb/10.195.41.141:27017] [resync/63d3d5ff0a7d0c6d05b88c32] started
2023-01-27T13:47:46Z E [mongodb/10.195.41.141:27017] [resync/63d3d5ff0a7d0c6d05b88c32] init storage: get S3 object header: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2023-01-27T13:47:46Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827264>
2023-01-27T13:47:46Z I [mongodb/10.195.41.141:27017] got epoch {1674827258 2}
2023-01-27T13:47:46Z I [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] started
2023-01-27T13:47:47Z D [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] got physical restores list: 0
2023-01-27T13:47:48Z D [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] got backups list: 2
2023-01-27T13:47:48Z D [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-01-27T13:47:49Z D [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-01-27T13:47:50Z I [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] succeed
2023-01-27T13:47:50Z D [mongodb/10.195.41.141:27017] [resync/63d3d60051ed7456a3331d6f] epoch set to {1674827270 3}
2023-01-27T13:48:35Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827314>
2023-01-27T13:48:35Z I [mongodb/10.195.41.141:27017] got epoch {1674827312 1}
2023-01-27T13:48:35Z I [mongodb/10.195.41.141:27017] [resync/63d3d6328545eae3cbe0734b] started
2023-01-27T13:52:35Z E [mongodb/10.195.41.141:27017] [resync/63d3d6328545eae3cbe0734b] init storage: get S3 object header: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2023-01-27T13:52:35Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827315>
2023-01-27T13:52:35Z I [mongodb/10.195.41.141:27017] got epoch {1674827312 1}
2023-01-27T13:52:35Z I [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] started
2023-01-27T13:52:36Z D [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] got physical restores list: 0
2023-01-27T13:52:36Z D [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] got backups list: 2
2023-01-27T13:52:36Z D [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-01-27T13:52:37Z D [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-01-27T13:52:39Z I [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] succeed
2023-01-27T13:52:39Z D [mongodb/10.195.41.141:27017] [resync/63d3d633506616fbeb0ef122] epoch set to {1674827559 1}
2023-01-27T13:54:39Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827679>
2023-01-27T13:54:39Z I [mongodb/10.195.41.141:27017] got epoch {1674827677 1}
2023-01-27T13:54:39Z I [mongodb/10.195.41.141:27017] [resync/63d3d79f58020021854b755b] started
2023-01-27T13:54:39Z E [mongodb/10.195.41.141:27017] [resync/63d3d79f58020021854b755b] unable to get backup store: storage undefined
2023-01-27T13:54:40Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827680>
2023-01-27T13:54:40Z I [mongodb/10.195.41.141:27017] got epoch {1674827677 1}
2023-01-27T13:54:40Z I [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] started
2023-01-27T13:54:41Z D [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] got physical restores list: 0
2023-01-27T13:54:41Z D [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] got backups list: 2
2023-01-27T13:54:41Z D [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-01-27T13:54:42Z D [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-01-27T13:54:44Z I [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] succeed
2023-01-27T13:54:44Z D [mongodb/10.195.41.141:27017] [resync/63d3d7a0485149dfa245baf0] epoch set to {1674827684 3}
2023-01-27T13:55:56Z I [mongodb/10.195.41.141:27017] got command backup [name: 2023-01-27T13:55:56Z, compression: s2 (level: default)] <ts: 1674827756>
2023-01-27T13:55:56Z I [mongodb/10.195.41.141:27017] got epoch {1674827684 3}
2023-01-27T13:55:56Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] init backup meta
2023-01-27T13:55:56Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] nomination list for mongodb: [[10.195.41.141:27017]]
2023-01-27T13:55:56Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] nomination mongodb, set candidates [10.195.41.141:27017]
2023-01-27T13:55:57Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] backup started
2023-01-27T13:56:00Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] wait for tmp users {1674827760 5}
2023-01-27T13:56:01Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] mongodump finished, waiting for the oplog
2023-01-27T13:56:04Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] set oplog span to {1674827757 6} / {1674827761 2}
2023-01-27T13:56:04Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] dropping tmp collections
2023-01-27T13:56:04Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] epoch set to {1674827764 6}
2023-01-27T13:56:06Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] backup finished
2023-01-27T13:56:06Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:55:56Z/63d3d7ec8325a541be39abcd] releasing lock
2023-01-27T13:57:44Z I [mongodb/10.195.41.141:27017] got command backup [name: 2023-01-27T13:57:44Z, compression: s2 (level: default)] <ts: 1674827864>
2023-01-27T13:57:44Z I [mongodb/10.195.41.141:27017] got epoch {1674827764 6}
2023-01-27T13:57:44Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] init backup meta
2023-01-27T13:57:44Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] nomination list for mongodb: [[10.195.41.141:27017]]
2023-01-27T13:57:44Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] nomination mongodb, set candidates [10.195.41.141:27017]
2023-01-27T13:57:45Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] backup started
2023-01-27T13:57:48Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] wait for tmp users {1674827868 5}
2023-01-27T13:57:49Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] mongodump finished, waiting for the oplog
2023-01-27T13:57:52Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] set oplog span to {1674827865 5} / {1674827869 2}
2023-01-27T13:57:52Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] dropping tmp collections
2023-01-27T13:57:52Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] epoch set to {1674827872 7}
2023-01-27T13:57:54Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] backup finished
2023-01-27T13:57:54Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T13:57:44Z/63d3d8589c5cc51f472ff127] releasing lock
2023-01-27T13:58:14Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827894>
2023-01-27T13:58:14Z I [mongodb/10.195.41.141:27017] got epoch {1674827892 1}
2023-01-27T13:58:14Z I [mongodb/10.195.41.141:27017] [resync/63d3d87664fc1fca3f6c98ae] started
2023-01-27T14:02:15Z E [mongodb/10.195.41.141:27017] [resync/63d3d87664fc1fca3f6c98ae] init storage: get S3 object header: NoCredentialProviders: no valid providers in chain. Deprecated.
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2023-01-27T14:02:15Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674827895>
2023-01-27T14:02:15Z I [mongodb/10.195.41.141:27017] got epoch {1674827892 1}
2023-01-27T14:02:15Z I [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] started
2023-01-27T14:02:16Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] got physical restores list: 0
2023-01-27T14:02:16Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] got backups list: 4
2023-01-27T14:02:16Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] bcp: 2023-01-26T15:01:57Z.pbm.json
2023-01-27T14:02:17Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] bcp: 2023-01-26T16:11:44Z.pbm.json
2023-01-27T14:02:18Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] bcp: 2023-01-27T13:55:56Z.pbm.json
2023-01-27T14:02:19Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] bcp: 2023-01-27T13:57:44Z.pbm.json
2023-01-27T14:02:21Z I [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] succeed
2023-01-27T14:02:21Z D [mongodb/10.195.41.141:27017] [resync/63d3d877bfedcdae295c75f3] epoch set to {1674828141 1}
2023-01-27T14:02:28Z I [mongodb/10.195.41.141:27017] got command backup [name: 2023-01-27T14:02:27Z, compression: s2 (level: default)] <ts: 1674828147>
2023-01-27T14:02:28Z I [mongodb/10.195.41.141:27017] got epoch {1674828141 1}
2023-01-27T14:02:28Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] init backup meta
2023-01-27T14:02:28Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] nomination list for mongodb: [[10.195.41.141:27017]]
2023-01-27T14:02:28Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] nomination mongodb, set candidates [10.195.41.141:27017]
2023-01-27T14:02:28Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] backup started
2023-01-27T14:02:32Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] wait for tmp users {1674828152 4}
2023-01-27T14:02:32Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674828152>
2023-01-27T14:02:32Z I [mongodb/10.195.41.141:27017] got epoch {1674828152 5}
2023-01-27T14:02:32Z D [mongodb/10.195.41.141:27017] [resync/63d3d978c209db650243ece1] get lock: another operation is running: Snapshot backup '63d3d97310a5d6d911b5f44f'
2023-01-27T14:02:32Z D [mongodb/10.195.41.141:27017] [resync/63d3d978c209db650243ece1] lock not acquired
2023-01-27T14:02:33Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] mongodump finished, waiting for the oplog
2023-01-27T14:02:33Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674828153>
2023-01-27T14:02:33Z I [mongodb/10.195.41.141:27017] got epoch {1674828152 5}
2023-01-27T14:02:33Z D [mongodb/10.195.41.141:27017] [resync/63d3d979b1353bcc713bb3d0] get lock: another operation is running: Snapshot backup '63d3d97310a5d6d911b5f44f'
2023-01-27T14:02:33Z D [mongodb/10.195.41.141:27017] [resync/63d3d979b1353bcc713bb3d0] lock not acquired
2023-01-27T14:02:36Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] set oplog span to {1674828148 14} / {1674828153 2}
2023-01-27T14:02:36Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] dropping tmp collections
2023-01-27T14:02:36Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] epoch set to {1674828156 6}
2023-01-27T14:02:38Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] backup finished
2023-01-27T14:02:38Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:02:27Z/63d3d97310a5d6d911b5f44f] releasing lock
2023-01-27T14:07:54Z I [mongodb/10.195.41.141:27017] got command resync <ts: 1674828474>
2023-01-27T14:07:54Z I [mongodb/10.195.41.141:27017] got epoch {1674828156 6}
2023-01-27T14:07:54Z I [mongodb/10.195.41.141:27017] [resync/63d3daba10f60584ca1cc190] started
2023-01-27T14:07:55Z D [mongodb/10.195.41.141:27017] [resync/63d3daba10f60584ca1cc190] got physical restores list: 0
2023-01-27T14:07:55Z D [mongodb/10.195.41.141:27017] [resync/63d3daba10f60584ca1cc190] got backups list: 1
2023-01-27T14:07:55Z D [mongodb/10.195.41.141:27017] [resync/63d3daba10f60584ca1cc190] bcp: 2023-01-26T16:17:11Z.pbm.json
2023-01-27T14:07:57Z I [mongodb/10.195.41.141:27017] [resync/63d3daba10f60584ca1cc190] succeed
2023-01-27T14:07:57Z D [mongodb/10.195.41.141:27017] [resync/63d3daba10f60584ca1cc190] epoch set to {1674828477 2}
2023-01-27T14:21:25Z I [mongodb/10.195.41.141:27017] got command backup [name: 2023-01-27T14:21:24Z, compression: s2 (level: default)] <ts: 1674829284>
2023-01-27T14:21:25Z I [mongodb/10.195.41.141:27017] got epoch {1674828477 2}
2023-01-27T14:21:25Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] init backup meta
2023-01-27T14:21:25Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] nomination list for mongodb: [[10.195.41.141:27017]]
2023-01-27T14:21:25Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] nomination mongodb, set candidates [10.195.41.141:27017]
2023-01-27T14:21:25Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] backup started
2023-01-27T14:21:28Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] wait for tmp users {1674829288 6}
2023-01-27T14:21:29Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] mongodump finished, waiting for the oplog
2023-01-27T14:21:32Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] set oplog span to {1674829285 14} / {1674829289 2}
2023-01-27T14:21:32Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] dropping tmp collections
2023-01-27T14:21:33Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] epoch set to {1674829293 3}
2023-01-27T14:21:35Z I [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] backup finished
2023-01-27T14:21:35Z D [mongodb/10.195.41.141:27017] [backup/2023-01-27T14:21:24Z/63d3dde495e6c514f878c3c8] releasing lock

```

andrew.pogrebnoi 
January 27, 2023 at 3:26 PM

Hi  

Thanks for the report. Can you also attach pbm logs here (pbm logs -sD -t0 -x)?

Details

Assignee

Reporter

Labels

Needs QA

Components

Affects versions

Priority

Created January 27, 2023 at 2:30 PM
Updated March 24, 2025 at 11:12 AM