Timeout on MIM Full Scan

With a large number of objects or an under-specced server it is possible for the full scan query to retrieve object hashes to time out.

By default REST API queries timeout after 20 minutes. As an example, a hashes query with 400,000 objects may take 30 minutes and return a 45 MB result set.

There are two approaches to resolve the issue:

Change the timeout

Edit C:\Program Files\SoftwareIDM\IdentityPanelWeb\web.config.

Change the executionTimeout property in the httpRuntime element from 1200 to 1800 or 2400 (30 minutes or 40 minutes).

Change the requestTimeout in the aspNetCore element to "00:30:00" or "00:40:00".

Save Changes

On Panel Service edit C:\Program Files\SoftwareIDM\PanelTools\config.json

Change the Rest:TimeoutSeconds property to 1800 or 2400 and save.

Change Database Indexing

It is possible to revise the database indices so that the hashes query is fully covered by the index. This will typically speed up a full scan hashes query by 300%. The tradeoff is that delta hash queries will be slightly slower, and data saving will also be slightly slower due to the increased indexing overhead.

Run C:\Program Files\SoftwareIDM\IdentityPanelWeb\MongoDB\bin\mongo.exe and execute the following commands:

> use idp_f98fcc9e-30bd-4b2d-8f83-c6663492457b_db

> db.removeIndex("i_1_d_1")

> db.createIndex({ i:1, d:1, _id:1, h:1, l:1 })

Note: the createIndex command may take a while (up to an hour) to run for a large environment