Unhandled exception. System.FormatException: Size nnn is larger than MaxDocumentSize 16793600

The following exception is raised at the end of a large (>50K records) full sync cycle when HyperSync hits an internal document size limit:

Unhandled exception. System.FormatException: Size 31062364 is larger than MaxDocumentSize 16793600.
at MongoDB.Bson.IO.BsonBinaryWriter.BackpatchSize()
at MongoDB.Bson.IO.BsonBinaryWriter.WriteEndDocument()

On further investigation of the HyperSync logs, the error is found to occur immediately after the "Threshold met" notice for an export attribute flow (EAF) rule.

Resolution

When processing large synchronization volumes, particularly on initial load, normal BAU HyperSync threshold email notification processing can be problematic.

The default HyperSync threshold email template is loading details from each action in a table in the email. The >50K rows is producing a message which is too big to save into the workflow queue collection.

To successfully process large volumes, this can be overcome temporarily by changing the row template for that rule (or alternatively the default template) to something like [""] and removing the open and close table tags in the header and footer templates.  Obviously, any threshold email notification will no longer include individual approve/reject links, but for large (e.g. >50K) changes this is a moot point.

Having negotiated the large sync volume, the threshold template can then be reverted.

Important: Don't set the row template to blank or the default standard row template will be used.