Preventing DBQueue Flooding During Mass Imports (4383019)

When performing mass identity imports (e.g., tens of thousands of Person objects) while the configuration parameter QER\Structures\DynamicGroupCheck\CalculateImmediatelyPerson is enabled and many dynamic roles have IsCalculateImmediately = 1 , the system generates an excessive number of individual recalculation jobs into the QBMDBQueuePond and DialogDBQueue tables.

Specifically, this creates millions of QER-K-DynamicGroupSingleBasic tasks, causing severe processing bottlenecks.

Symptoms include:

• Massive queue growth (e.g., 200+ million rows).
• High CPU/disk load and processing delays.
• Delayed or missing dynamic role / birthright assignments.
• Web portal issues (categories and recipients not visible, including base roles like “Base roles\Everyone (Change)”).
• Prolonged HR sync runtimes and backlog of other tasks.

Immediate recalculation is designed for small, near-real-time changes with minimal trigger attributes. During bulk HR operations, every changed Person object can trigger recalculation tasks for every relevant dynamic group, overwhelming the queue mechanism.

Workaround and Best Practices

• Use scheduled dynamic role recalculation for large numbers of organizational or derived roles, especially when driven by HR data.
• Limit immediate recalculation to a small number of critical dynamic roles with minimal trigger attributes.
• Monitor queue sizes regularly during normal operations.

Before a Major Import or Bulk Update

1. In One Identity Manager Designer, go to Configuration Parameters.
2. Disable the parameter: QER\Structures\DynamicGroupCheck\CalculateImmediatelyPerson (Set value to 0).
3. Temporarily disable immediate recalculation on large/critical dynamic roles:
   • Set IsCalculateImmediately = 0 on all dynamic groups.
   • Important: Even with the global parameter disabled, individual dynamic groups with IsCalculateImmediately = 1 can still trigger heavy recalculation when manually started or during bulk processing. Disabling the per-group flag provides additional protection.
4. Save and publish the configuration.

Perform the Import / HR Synchronization

Run the bulk load as normal. The queue will remain manageable.

After the Import

1. Re-enable the parameter in Designer: QER\Structures\DynamicGroupCheck\CalculateImmediatelyPerson (Set value to 1). Save and publish.
2. Trigger a bulk recalculation of all immediately-calculated dynamic groups using this SQL script (execute against the One Identity Manager database).

Please use SQL Server Management Studio (SSMS) and execute the following script against the One Identity Manager database. This script will efficiently insert one bulk recalculation task (QER-K-DynamicGroupTest) per qualifying dynamic group, avoiding the creation of millions of individual tasks.

DECLARE @genprocId VARCHAR(38) = LOWER(NEWID());
DECLARE @DBQueueElements QBM_YDBQueueRaw;

INSERT INTO @DBQueueElements (Object, SubObject, GenProcID)
SELECT dg.UID_DynamicGroup, NULL, @genprocId
FROM DynamicGroup dg
WHERE dg.IsCalculateImmediately = 1;

EXEC QBM_PDBQueueInsert_Bulk 'QER-K-DynamicGroupTest', @DBQueueElements;

This script efficiently inserts one bulk recalculation task per qualifying dynamic group instead of millions of individual tasks.

Alternative GUI Method (Object Browser)

• Open Object Browser → Filter on DynamicGroup with condition IsCalculateImmediately=1. 
• Select all results (Ctrl + A).
• Right-click → Open multiple selection → Object → Properties → Methods tab.
• Select Calculate() and click Run.

Verification Steps

• Monitor DBQueue tables for processing progress.
• Confirm dynamic role memberships (including base application roles) are updated.
• Test the web portal for full functionality.

An Enhancement request (706529) has been submitted to automatically optimize this process in future releases. The proposed mechanism would automatically detect mass identity inserts/updates and natively consolidate them into bulk recalculation jobs.