8

After upgrading CRM 2013 on-premises to CRM365 an issue with plugins started to come up. Each time I try to use IOrganizationService from plugin for any operation I get an error. This issue only occures on multi-server instalation, it didn't occure on CRM 2013 version. It also doesn't occure on single-server instalation of CRM365.

Unhandled Exception: System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=8.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: System.Web.HttpUnhandledException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #52BFF7F3Detail: 
<OrganizationServiceFault xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">
  <ActivityId>c291f3ee-3bfe-4c0d-a631-9b7e70c19add</ActivityId>
  <ErrorCode>-2147220970</ErrorCode>
  <ErrorDetails xmlns:d2p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
  <Message>System.Web.HttpUnhandledException: Microsoft Dynamics CRM has experienced an error. Reference number for administrators or support: #52BFF7F3</Message>
  <Timestamp>2017-03-13T12:14:25.2768854Z</Timestamp>
  <ExceptionSource i:nil="true" />
  <InnerFault>
    <ActivityId>c291f3ee-3bfe-4c0d-a631-9b7e70c19add</ActivityId>
    <ErrorCode>-2147220891</ErrorCode>
    <ErrorDetails xmlns:d3p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
    <Message>callManagerInfo</Message>
    <Timestamp>2017-03-13T12:14:25.2768854Z</Timestamp>
    <ExceptionSource i:nil="true" />
    <InnerFault i:nil="true" />
    <OriginalException i:nil="true" />
    <TraceText>

[PluginsMerged: Core.Plugins.ReadAuditLogPlugin]
[b381844a-41b5-e611-b605-0050568c2255: Core.Plugins.ReadAuditLogPlugin: RetrieveMultiple of account]

Entered Core.Plugins.ReadAuditLogPlugin.Execute(), Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
Core.Plugins.ReadAuditLogPlugin is firing for Entity: account, Message: RetrieveMultiple, Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
Entered internal execute in ReadAuditLogService., Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
EnsureInitialized., Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
Entered GetMessage., Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
Entered GetUserFromPlugin., Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
user 21391658-381b-e511-80e0-005056931145., Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
Before context.OrganizationService.Retrieve., Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
callManagerInfo, Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145
Exiting Core.Plugins.ReadAuditLogPlugin.Execute(), Correlation Id: 0a738a82-3329-4f60-a6b3-ea8230290d19, Initiating User: 21391658-381b-e511-80e0-005056931145


</TraceText>
  </InnerFault>
  <OriginalException i:nil="true" />
  <TraceText i:nil="true" />
</OrganizationServiceFault>

The code that is throwing this exception is:

private User GetUserFromPlugin(ILocalPluginContext context)
{
    context.Trace("Entered GetUserFromPlugin.");
    var userId = context.PluginExecutionContext.UserId;
    context.Trace(string.Format("user {0}.", userId));

    context.Trace("Before context.OrganizationService.Retrieve.");
    if (context.OrganizationService == null)
    {
        var organizationServiceNullMessage = "OrganizationService is null";
        context.Trace(organizationServiceNullMessage);
        throw new InvalidPluginExecutionException(organizationServiceNullMessage);
    }

    var systemUser = context.OrganizationService.Retrieve(
        EntityNames.CRMSystemUser,
        userId,
        new ColumnSet(FieldNames.CRMSystemUser.DomainName, FieldNames.CRMSystemUser.LastName, FieldNames.CRMSystemUser.FirstName));
    context.Trace("After context.OrganizationService.Retrieve.");

    var user = new User
    {
        Id = userId,
        DomainName = systemUser.ReadAttributeValue<string>(FieldNames.CRMSystemUser.DomainName),
        FirstName = systemUser.ReadAttributeValue<string>(FieldNames.CRMSystemUser.FirstName),
        LastName = systemUser.ReadAttributeValue<string>(FieldNames.CRMSystemUser.LastName),
    };

    return user;
}

---------EDIT----------------

I'm adding information from Plugin Tracing

Unhandled Exception: System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=8.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: callManagerInfoDetail: 
<OrganizationServiceFault xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/xrm/2011/Contracts">
<ActivityId>dbe29fd8-651c-47d2-ad39-ed460552d4e1</ActivityId>
<ErrorCode>-2147220891</ErrorCode>
<ErrorDetails xmlns:d2p1="http://schemas.datacontract.org/2004/07/System.Collections.Generic">
<KeyValuePairOfstringanyType>
<d2p1:key>OperationStatus</d2p1:key>
<d2p1:value xmlns:d4p1="http://www.w3.org/2001/XMLSchema" i:type="d4p1:int">0</d2p1:value>
</KeyValuePairOfstringanyType>
<KeyValuePairOfstringanyType>
<d2p1:key>SubErrorCode</d2p1:key>
<d2p1:value xmlns:d4p1="http://www.w3.org/2001/XMLSchema" i:type="d4p1:int">-2146233088</d2p1:value>
</KeyValuePairOfstringanyType>
</ErrorDetails>
<Message>callManagerInfo</Message>
<Timestamp>2017-03-14T06:09:08.1648164Z</Timestamp>
<ExceptionSource>PluginExecution</ExceptionSource>
<InnerFault i:nil="true" />
<OriginalException>Microsoft.Xrm.Sdk.InvalidPluginExecutionException
at Core.Plugins.BusinessLogic.Service.ReadAuditLogService.Execute(ILocalPluginContext context)
at Core.Plugins.ReadAuditLogPlugin.InternalExecute(LocalPluginContext context)
at Plugins.Common.PluginBase.Execute(IServiceProvider serviceProvider)
at Microsoft.Crm.Sandbox.SandboxAppDomainHelper.&lt;&gt;c__DisplayClass5.&lt;Execute&gt;b__0()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.Crm.Sandbox.SandboxAppDomainHelper.Execute(IOrganizationServiceFactory organizationServiceFactory, Dictionary`2 sandboxServices, String pluginTypeName, String pluginConfiguration, String pluginSecureConfig, IPluginExecutionContext requestContext)
at Microsoft.Crm.Sandbox.SandboxAppDomainHelper.Execute(IOrganizationServiceFactory organizationServiceFactory, Dictionary`2 sandboxServices, String pluginTypeName, String pluginConfiguration, String pluginSecureConfig, IPluginExecutionContext requestContext)
at Microsoft.Crm.Sandbox.SandboxWorker.Execute(SandboxCallInfo callInfo, SandboxPluginExecutionContext requestContext, Guid pluginAssemblyId, Int32 sourceHash, String assemblyName, Guid pluginTypeId, String pluginTypeName, String pluginConfiguration, String pluginSecureConfig, SandboxRequestCounter&amp; workerCounter, Boolean returnTraceInfo)</OriginalException>
<TraceText>
Entered Core.Plugins.ReadAuditLogPlugin.Execute(), Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
Core.Plugins.ReadAuditLogPlugin is firing for Entity: account, Message: RetrieveMultiple, Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
Entered internal execute in ReadAuditLogService., Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
EnsureInitialized., Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
Entered GetMessage., Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
Entered GetUserFromPlugin., Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
user 21391658-381b-e511-80e0-005056931145., Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
Before context.OrganizationService.Retrieve., Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
callManagerInfo, Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
Exiting Core.Plugins.ReadAuditLogPlugin.Execute(), Correlation Id: 50b0f06c-fc66-40b3-acbd-69a2463be0f4, Initiating User: 21391658-381b-e511-80e0-005056931145
</TraceText>
</OrganizationServiceFault>

Server stack trace: 
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]: 
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.Crm.Sandbox.ISandboxHost.ExecuteAndReturnTraceInfo(SandboxCallInfo callInfo, SandboxPluginExecutionContext requestContext, Guid pluginAssemblyId, Int32 sourceHash, String assemblyName, Guid pluginTypeId, String pluginTypeName, String pluginConfiguration, String pluginSecureConfig, String assemblyContents, Boolean returnTraceInfo)
at Microsoft.Crm.Sandbox.SandboxPlugin.Execute(SandboxClient client, SandboxCallTracker callTracker, IExecutionContext requestContext, String assemblyContents, Boolean returnTraceInfo)
at Microsoft.Crm.Sandbox.SandboxCodeUnit.Execute(IExecutionContext context)
Maco
  • 429
  • 13
  • 19
  • Is there a second plugin registered on `Retrieve` of `SystemUser`? What is happening inside `callManagerInfo`? Did you try turning on Plugin Tracing from the System Settings / Customizations? Since you are getting a "Unexpected Exception" there seems to be something wrong further down the (execution pipe)line. – Filburt Mar 13 '17 at 13:33
  • Hi. No other plugin works on retrieve, however this exception is thrown for all my plugins regardles off message that is triggering the plugin. callManagerInfo is some M$ class and I don't have access to it. I'll post information from Plugin Tracing. – Maco Mar 14 '17 at 06:17
  • The Sub-Errorcode `-2146233088` (hex 80131500) from the plugin trace according to [the Error Codes reference](https://msdn.microsoft.com/en-us/library/gg328182(v=crm.8).aspx) suggests that *"Certificate used for S2S authentication of Dynamics 365 Onpremise with Exchange Online has expired"* so you may want to investigate in this direction. – Filburt Mar 14 '17 at 08:31
  • Thanks. I'll check it. – Maco Mar 14 '17 at 09:35
  • I've just started encountering a similar issue since upgrading to Dynamics 365 from CRM 2016. We are also multi-server environment. 'SendEmailRequests' from workflows would fail some of the time with the same callManagerInfo.I think this is related to authenticating the user. Changing the plugin from 'sandbox' isolation mode to 'none' fixed the issue, but I am unsure of the exact cause. The same code used to work in sandbox. – Alex Mar 28 '17 at 15:22
  • Unfortunately non-sandbox is not an option in our case. We've registered a case at Microsoft support and are waiting to see if they can help us. – Maco Mar 29 '17 at 11:58
  • in our case this was the error in trace logs: >Crm Exception: Message: callManagerInfo, ErrorCode: -2147220989, InnerException: System.ArgumentNullException: Value cannot be null. – Serge Sep 13 '17 at 02:30

2 Answers2

7

Workaround has been provided to us by Microsoft. To stop error set the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MSCRMSandboxService\SingleBox DWord value to 0. After doing that the issue It helped us with the problem.

Maco
  • 429
  • 13
  • 19
0

We encountered the same issue and fixed by Adding the SPN in both the nodes.

Srikanta Mahapatro
  • 314
  • 4
  • 5