Attempting to run an Export job with Microsoft Forefront Identity Manager 2010 R2 throws the error: “stopped-extension-dll-exception”

Problem

You’ve noticed that your previously operational Microsoft Forefront Identity Manager 2010 R2 throws the error the following error when you execute an Export job:

stopped-extension-dll-exception

Exchange 2010 contacts in the are either no longer updated or created in the source domain. 

You proceed into the connector’s properties under Management Agents:

Review and confirm that the service account is correct:

Reviewing the event logs show the following errors displayed in the Application logs:

Log Name: Application

Source: FIMSynchronizationService

Event ID: 6803

Level: Error

Task Category: Management Agent Run Profile

The management agent “FIM Connector” failed on run profile “Export” because the server encountered errors.

Log Name: Application

Source: FIMSynchronizationService

Event ID: 0

Level: Error

Task Category: None

The description for Event ID 0 from source FIMSynchronizationService cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.

The following information was included with the event:

There is an error in Exch2010Extension BeginExportToCd() function.Type: System.Management.Automation.Remoting.PSRemotingTransportException

Message: Connecting to remote server failed with the following error message : The WS-Management service cannot process the request. The system load quota of 1000 requests per 2 seconds has been exceeded. Send future requests at a slower rate or raise the system quota. The next request from this user will not be approved for at least 1316187520 milliseconds. For more information, see the about_Remote_Troubleshooting Help topic.

Stack Trace: at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.EndOpen(IAsyncResult asyncResult)

at System.Management.Automation.Runspaces.RunspacePool.Open()

at System.Management.Automation.RemoteRunspace.Open()

at Exch2010Extension.Exch2010ExtensionClass.OpenConnection(String uri, PSCredential credential)

at Exch2010Extension.Exch2010ExtensionClass.BeginExportToCd(String connectTo, String domain, String server, String user, String password)

the message resource is present but the message is not found in the string/message table

You attempt to use the following TechNet article to further troubleshoot by disabling Exchange provisioning confirming that the export now completes and manually executing the included PowerShell cmdlet:

FIM Troubleshooting: stopped-dll-exception: WinRM cannot process the request: Access Denied
https://social.technet.microsoft.com/wiki/contents/articles/15091.fim-troubleshooting-stopped-dll-exception-winrm-cannot-process-the-request-access-denied.aspx

.. but you run into the error:

‘contoso.com/contoso/Employees/TMRUK/GalContacts/Matthew  Evans’ have been modified.
WARNING: The command completed successfully but no settings of
‘contoso.com/contoso/Employees/TMRUK/GalContacts/Buu Truong’ have been modified.
WARNING: The command completed successfully but no settings of
‘contoso.com/contoso/Employees/TMRUK/GalContacts/Gemma Gregson’ have been modified.
“DG_TMRUK_Pricing” is a MailForestContact and can’t be modified.
    + CategoryInfo          : NotSpecified: (contoso…G_TMRUK_Pricing:ADObjectId) [Set-MailContact], TaskInva
   lidOperationException
    + FullyQualifiedErrorId : 69D6CABF,Microsoft.Exchange.Management.RecipientTasks.SetMailContact

“Terence Luk” is a MailForestContact and can’t be modified.
    + CategoryInfo          : NotSpecified: (contoso…cts/Terence Luk:ADObjectId) [Set-MailContact], TaskInva
   lidOperationException
    + FullyQualifiedErrorId : 1DAD038F,Microsoft.Exchange.Management.RecipientTasks.SetMailContact

WARNING: The command completed successfully but no settings of
‘contoso.com/contoso/Employees/TMRUK/GalContacts/Anna Ivanova’ have been modified.
WARNING: The command completed successfully but no settings of
‘contoso.com/contoso/Employees/TMRUK/GalContacts/DG_Operations’ have been modified.
“Taro Murakami” is a MailForestContact and can’t be modified.
    + CategoryInfo          : NotSpecified: (contoso…s/Taro Murakami:ADObjectId) [Set-MailContact], TaskInva
   lidOperationException
    + FullyQualifiedErrorId : 5FD06EB8,Microsoft.Exchange.Management.RecipientTasks.SetMailContact

“Sara Perdichizzi” is a MailForestContact and can’t be modified.
    + CategoryInfo          : NotSpecified: (contoso…ara Perdichizzi:ADObjectId) [Set-MailContact], TaskInva
   lidOperationException
    + FullyQualifiedErrorId : E298C7BF,Microsoft.Exchange.Management.RecipientTasks.SetMailContact

“Giuseppe Ieraci” is a MailForestContact and can’t be modified.
    + CategoryInfo          : NotSpecified: (contoso…Giuseppe Ieraci:ADObjectId) [Set-MailContact], TaskInva
   lidOperationException
    + FullyQualifiedErrorId : 28CFBAA8,Microsoft.Exchange.Management.RecipientTasks.SetMailContact

“Ken Tarbet” is a MailForestContact and can’t be modified.
    + CategoryInfo          : NotSpecified: (contoso…acts/Ken Tarbet:ADObjectId) [Set-MailContact], TaskInva
   lidOperationException
+ FullyQualifiedErrorId : 2EE5477,Microsoft.Exchange.Management.RecipientTasks.SetMailContact

The AD and Exchange contacts also does not get created.

Other TechNet articles such as the following does not correct the issue:

FIM Troubleshooting: stopped-dll-exception troubleshooter document
https://social.technet.microsoft.com/wiki/contents/articles/8759.fim-troubleshooting-stopped-dll-exception-troubleshooter-document.aspx

Solution

After going through numerous TechNet articles and posts without making any progress, I went ahead and tried changing the Exchange 2010 RPS URI to another Exchange 2010 HT/CAS server:

… and the export job immediately worked.  This lead me to change my search query, which was when I found the following blog post that resolved the issue:

http://www.vspbreda.nl/nl/exchange/exchange-2010/exchange-2010-load-quota-1000-requests-exceeded/

What I needed to do was simply perform an iisreset on the problematic server to prevent the export job from erroring out: