Active Roles performance bottlenecks (4340870)

Addressing Known Issues

1. In environments with a large number of Active Roles Virtual Attributes, the Active Roles Administration Service host and the Microsoft SQL Server host may experience high CPU utilization and poor performance. This could be related to a known issue that is addressed in Active Roles 7.0.1 and later versions, but which is not enabled by default.
2. In environments with a large number of Microsoft Exchange mailboxes, retrieving accounts with Microsoft Exchange attributes may be noticeably slower than retrieving accounts without Microsoft Exchange attributes. This could be related to a known issue that is addressed in Active Roles 7.2.1 and later versions, but which is not enabled by default.
3. The Application Policy Script Module may be incorrectly configured following an upgrade.
4. If the Active Roles Web Interface is the only Active Roles client experiencing performance issues, then there may be a known issue related to SignalR connectivity requirements.
5. If there is a performance issue specifically with listing groups or with viewing the memberOf tab of an Active Directory object, then an expensive Managed Unit may be causing the poor performance.
6. In environments where the "Replicating Directory Changes" extended right has not been granted to the domain management account, a scenario might arise where Active Roles Managed Units have broken rules. These broken rules can negatively impact the performance of all Active Roles clients and should be identified and addressed.

Establish a baseline or benchmark

It can be very useful to gather a performance baseline in order to be able to better provide contrast against any issues which might be encountered. Active Roles has PerfMon counters which can be very useful for this and for troubleshooting any performance issues.

Update Microsoft Dependencies

Occasionally, there may be an issue in a Microsoft dependency that can cause performance issues and/or memory leaks. Before updating any PowerShell components, ensure the version is supported in the Active Roles Release Notes. Failure to use the proper versions may result in loss of functionality.

Resources on the host server

CPU

CPU resources can become a performance bottleneck if there is insufficient memory. At a minimum, a CPU running at 2ghz with 2 cores is required, however it is recommended to run with a minimum of 4 cores, especially in large environments or when using Dynamic Groups, Managed Units, scripts, and other related features.

Memory

As a general rule, Windows operating system hosts require at least 4GB of RAM. 

In addition to the 4gb of memory for the operating system, Active Roles components require:

• Administration Service: 4gb
• Web Interface: 2gb
• Synchronization Service: 2gb
• Console and Management Tools: 1gb
• SPML Provider: 1gb

This means that at a minimum, the Active Roles server would require 14gb.

This may be sufficient for smaller environments; however, real-world implementations and large environments may require more memory. Features such as Dynamic Groups, Managed Units, Workflows and Scheduled Tasks may also require more resources. Larger environments will notice some performance issues in a minimum configuration. Extremely large production environments with Active Directory objects in the tens or hundreds of thousands, may encounter out-of-memory errors or outright crashes. 

To confirm if additional CPU or memory resources are needed, it is best to utilize the Active Roles PerfMon counters to confirm a local resource bottleneck and reconfigure the host as needed.

Quick Connect and Active Roles Synchronization Service Connector Performance

Poor connector performance in the Quick Connect or the Active Roles Synchronization Service can sometimes be addressed by enabling multithreading for that connector.

SQL Server Considerations

Having a dedicated SQL Server instance (one hosting only Active Roles databases) is always recommended. Sharing SQL Server resources with SQL-intensive applications (such as Quest Change Auditor) will negatively impact Active Roles database performance.

Microsoft SQL is a resource-intensive application, and will consume any available memory on its host server by default. To avoid SQL consuming all available memory and negatively impacting the performance of the host server, the maximum memory available to SQL should be capped leaving at least 4GB of the total amount of available RAM as per this Microsoft resource. For example, if the Windows host has 16GB of RAM allocated, then a dedicated SQL instance on a dedicated SQL server host should be capped at 12GB.

When authenticating to Microsoft SQL Server via Windows Authentication, it is extremely important to ensure that the necessary Service Principal Names are configured and delegated. Kerberos Logging can be enabled on the Active Roles Administration Service and Active Roles Web Interface hosts to ensure that there are no errors with Windows Authentication. This Microsoft resource can assist with configuring the necessary SQL Server SPN's. Once these are configured, delegations can be added in Active Directory Users and Computers on the Delegation tab.

Performance issues affecting only Active Roles Virtual Attributes

If there is a marked difference in performance noted in multiple Active Roles clients when working with Active Roles Virtual Attributes compared to a similar operation that only references Active Directory native attributes, it may be necessary to perform maintenance on SQL indices.

Active Roles Administrative Service and SQL co-location

In labs, pre-production environments, or very small (e.g. 1000 managed objects or less) production environments, it may be possible to have a dedicated SQL instance hosted on the same machine which is hosting Active Roles. In larger production environments, separate dedicated hosts for the Administrative Service and SQL server are recommended.  This is especially true for environments with tens of thousands of managed objects.

Other Windows Host Configuration Considerations

The Windows page file settings on the host server can also negatively impact performance if they are set too low. As per this Microsoft resource, the minimum page file size should be equal to the amount of RAM present on the host server, and the maximum size should be three times the amount of RAM present.

Latency to the environment

Active Roles is most sensitive to latency on communications to the SQL Server and to the associated Global Catalog server. Ideally, these resources would be in the same site, with a latency of less than 1ms.

Cached information from before a change in the environment

Active Roles has a few different places where it caches environmental information. If the environment changes significantly (Active Directory schema or functional level updates, Microsoft SQL or Microsoft Exchange major version change, etc.) then this cache may contain obsolete information which negatively impacts performance.

Follow all steps in this reference in order to clear all of these caches.

Obsolete and/or incompatible components

External applications such as Change Auditor, Defender, Recovery Manager, and Authentication Services all deploy Active Roles components. When Active Roles or these external applications are upgraded, older versions of their deployed components may remain in the Active Roles configuration and will need to be disabled and/or removed.

High CPU on Active Directory Domain Controller when running the "Dynamic Group Updater" Scheduled Task

In environments with a large number of Active Roles Dynamic Groups, the nightly run of the Dynamic Group Updater Scheduled Task can cause high CPU utilization on the target Active Directory Domain Controller and/or run for several hours. To help with this issue, add indices in Active Directory on the attributes that are being queried by the Dynamic Group membership rule(s).

Change Workflow filtering conditions

In environments where a large number of Change Workflows have been implemented (for example, to handle Approvals), inefficient filtering conditions could lead to performance issues when performing an operation which is related to a Workflow triggering condition.

For example: having a script in a filtering condition of a Workflow is less efficient than simply filtering on an attribute of the Request object, as Active Roles will need to execute the script and find the resultant value in order to determine if the Workflow execution is valid.
 

"Expensive" Script calls

If PerfMon logs indicate that the Script module average execution time is a potential performance bottleneck, this could also indicate poor performance when connecting to or working with Microsoft Exchange. When working with Microsoft Exchange 2010, it is an optional configuration to use Remote PowerShell when working with Microsoft Exchange - in all later versions of Microsoft Exchange, it is the only option. These Remote PowerShell calls are included in all Active Roles Script Module performance counters.