Without the partialFilterExpression - when that field has lots of nulls, the createIndex operation fails due to the duplicates.
This is with mongodb 6.0.19 three node cluster and pbm 2.8.0-1 on ubuntu24.
I’m happy to provide any additional information needed. As near as I can see - the createIndexes() call being made by pbm-agent doesn’t include the partialFilterExpression attribute at all.
Proposed Solution
Add partialFilterExpression and see if we need to add others from the list:
this is regression issue, the changes was introduced in
We need to extend IndexSpec structure at least as it is done in mongo-tools and start tracking the changes for future mongodb releases
Nathan Neulinger January 29, 2025 at 5:52 PM
I downgraded my install to 2.6.0 and it’s now working without issue. I think in another thread there was a coment that it was related to internal change to “your own mongodump implementation” or something along those lines.
Boris Ilijic January 29, 2025 at 5:47 PM
/ Did this work properly in v2.7 or this is regression ?
I marked as Critical because it results in the restore being non-functional, which is core function of the product.
Example index:
{ v: 2, key: { siteId: 1, hubspotVisitorId: 1 }, name: 'siteId_1_hubspotVisitorId_1', unique: true, partialFilterExpression: { hubspotVisitorId: { '$exists': true } } }Without the partialFilterExpression - when that field has lots of nulls, the createIndex operation fails due to the duplicates.
This is with mongodb 6.0.19 three node cluster and pbm 2.8.0-1 on ubuntu24.
I’m happy to provide any additional information needed. As near as I can see - the createIndexes() call being made by pbm-agent doesn’t include the partialFilterExpression attribute at all.
Proposed Solution
Add
partialFilterExpressionand see if we need to add others from the list:# A built index appears with these fields, which must be the same as ListIndexesReplyItem's # fields in list_indexes.idl. NewIndexSpec: description: "A type representing a spec for a new index" strict: true fields: v: description: 'Index spec version' type: safeInt optional: true stability: stable key: description: 'Key to index on' type: object_owned stability: stable name: description: 'Descriptive name for the index' type: string stability: stable ns: # MongoDB 4.2 and older generate this field, see SERVER-41696. type: string ignore: true stability: stable background: type: safeBool optional: true stability: unstable unique: type: safeBool optional: true stability: stable hidden: type: safeBool optional: true stability: stable partialFilterExpression: type: object_owned optional: true stability: stable sparse: type: safeBool optional: true stability: unstable expireAfterSeconds: type: safeInt optional: true stability: stable storageEngine: type: object_owned optional: true stability: unstable weights: type: variant: [string, object_owned] optional: true stability: stable default_language: type: string optional: true stability: stable language_override: type: string optional: true stability: stable textIndexVersion: type: safeInt optional: true stability: stable 2dsphereIndexVersion: type: safeInt optional: true stability: stable bits: type: safeInt optional: true stability: stable min: type: safeDouble optional: true stability: stable max: type: safeDouble optional: true stability: stable bucketSize: type: safeDouble optional: true stability: unstable collation: type: object_owned optional: true stability: stable wildcardProjection: type: object_owned optional: true stability: stable columnstoreProjection: type: object_owned optional: true stability: unstable columnstoreCompressor: type: string optional: true stability: unstable coarsestIndexedLevel: type: safeInt optional: true stability: stable finestIndexedLevel: type: safeInt optional: true stability: stable dropDups: type: safeBool optional: true stability: unstable originalSpec: type: object_owned optional: true stability: unstable clustered: type: safeBool optional: true stability: stable prepareUnique: type: safeBool optional: true stability: unstable