2

I have a BizTalk server and a SQL server which BizTalk sends messages via WCF-SQL to. The BizTalk server has been calling to this server for over a year with no problems. I came in this morning any suddenly it can't (it was working on Friday).

The full error I'm getting when calling the WCF-SQL endpoint is:

A message sent to adapter "WCF-SQL" on send port "MyPort" with URI "mssql://mySQLServer" is suspended.
Error details: System.Transactions.TransactionManagerCommunicationException: Communication with the underlying transaction manager has failed. ---> System.Runtime.InteropServices.COMException:
The MSDTC transaction manager was unable to push the transaction to the destination transaction manager due to communication problems.
Possible causes are: a firewall is present and it doesn't have an exception for the MSDTC process, the two machines cannot find each other by their NetBIOS names, or the support for network transactions is not enabled for one of the two transaction managers. (Exception from HRESULT: 0x8004D02A)
at System.Transactions.Oletx.ITransactionShim.Export(UInt32 whereaboutsSize, Byte[] whereabouts, Int32& cookieIndex, UInt32& cookieSize, CoTaskMemHandle& cookieBuffer)
at System.Transactions.TransactionInterop.GetExportCookie(Transaction transaction, Byte[] whereabouts)

I've followed instructions from the following thread: MSDTC on server 'server is unavailable

I've run msdtc -uninstall then msdtc -install and restarted the service several times.

I've rebooted the server several times.

I can connect to the database using Sql Server Management Studio

DTCPing when trying to connect from the SQL server to the Biztalk server results in (when DTCPing is running on the BizTalk):

Problem:fail to invoke remote RPC method
Error(0x6BA) at dtcping.cpp @303
-->RPC pinging exception
-->1722(The RPC server is unavailable.)
RPC test failed

when going from Biztalk to SQL I get this (even thought DTCPing is running on the other end)

Please refer to following log file for details:
C:\Temp\DTCPing\myserv.log
Invoking RPC method on dbaditest
RPC test is successful
++++++++++++RPC test completed+++++++++++++++
Please start PING from dbaditest to complete the test

neither server is running a firewall at all

I'm all out of things to try.

Edit: I can confirm that other servers/computers can connect to the SQL server. So I have to assume that it's the BizTalk server that is the problem.

Edit 2: I tried connecting from BizTalk Server to another SQL server on the network and got the same error. I'm moments away from throwing my hands up and rebuilding my dev environment -- ugg :(

Edit 3: I can telnet to port 135 from BizTalk to SQL Server, so there's nothing blocking it.

Edit 4: DTCTester results in:

tablename= #dtc24449
Creating Temp Table for Testing: #dtc24449
Warning: No Columns in Result Set From Executing: 'create table #dtc24449    (ival int)'
Initializing DTC
Beginning DTC Transaction
Enlisting Connection in Transaction
Error:
SQLSTATE=25S12,Native error=-2147168242,msg='[Microsoft][ODBC SQL Server Driver]Distributed transaction error'
Error:
SQLSTATE=24000,Native error=0,msg=[Microsoft][ODBC SQL Server Driver]Invalid cursor state
Typical Errors in DTC Output When
a.  Firewall Has Ports Closed
-OR-
b.  Bad WINS/DNS entries
-OR-
c.  Misconfigured network 
-OR-
d.  Misconfigured SQL Server machine that has multiple netcards.
Aborting DTC Transaction
Releasing DTC Interface Pointers
Successfully Released pTransaction Pointer.
Community
  • 1
  • 1
Bensonius
  • 1,501
  • 1
  • 15
  • 39
  • To confirm, you've rebooted both BizTalk and SQL servers? – Gruff Nov 24 '15 at 08:18
  • Did anything changed on BizTalk or SQL server side when this problem started such as reboot after patching or something else. Can you verify MSDTC configuration as suggested in link https://msdn.microsoft.com/en-us/library/aa544733(v=cs.70).aspx – Vikas Bhardwaj Nov 24 '15 at 10:08
  • I added an edit. It's the BizTalk server that is the issue. As far as I know, nothing has changed. This is a development environment and everything was still as I left it (all my windows, Visual Studio, still open). – Bensonius Nov 24 '15 at 16:16
  • @VikasBhardwaj I have done the MSDTC config many times and restarted the service etc. – Bensonius Nov 24 '15 at 16:21
  • Assuming these are VMs check the CID especially if one of them was cloned, these need to be unique and I can vaguely remember this being an issue for me once. – Rodders Nov 25 '15 at 17:33
  • They are unique, it was not cloned. Besides, I believe the doing `msdtc -uninstall` and `msdtc -install` generates new CIDs on the the machine (which I have done) – Bensonius Nov 25 '15 at 18:57

4 Answers4

2

You've already taken some steps here, but carefully go through the MSDN Article on Troubleshooting MSDTC.

I'd be concerned that someone imaged another server off of yours, but uninstalling and reinstalling MSDTC should have fixed that. It might be worth checking on these registry values as well (from the above link):

Windows enhances security by requiring authenticated calls to the RPC interface. This functionality is configurable through the EnableAuthEpResolution and RestrictRemoteClients registry keys. To ensure that remote computers are able to access the RPC interface, follow these steps:

Click Start, click Run, type regedit.exe, and then click OK to start Registry Editor.

Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT

Under the RPC key, create the following DWORD entries with the indicated values. If the RPC key does not exist then it must be created.

DWORD entry             Default value  Recommended value
EnableAuthEpResolution  0 (disabled)   1
RestrictRemoteClients   1 (enabled)    0

Close Registry Editor.

Restart the MSDTC Service.

Dan Field
  • 20,885
  • 5
  • 55
  • 71
  • I did some Windows Updates, then I did this and no change. I'm still getting the same error as posted in the original Question. – Bensonius Nov 24 '15 at 23:53
1

  1. Is your BizTalk/SQL computer name unique? (no conflicts with other machine)

  2. Can you DTC connect to another SQL server from your BizTalk server? I would suggest you to use DTCTester testing the DTC connection instead of DTCPing.

Zee
  • 830
  • 1
  • 9
  • 22
  • yes, names are unique. No I cannot DTC connect to another SQL server from my BizTalk server (see edits in original question) – Bensonius Nov 24 '15 at 17:37
  • I added the DTCTester results to the original question. – Bensonius Nov 24 '15 at 17:48
  • The 1722 error normally indicates it is a network connection issue. So I do not think rebuild your BizTalk server environment will solve your problem. Something you can try: 1. Contact your corp's network team to see if they are blocking some ports in network infrastructure. 2. Use Wireshark to capture all network packages in your biztalk server to see what happened. Good luck. – Zee Nov 24 '15 at 17:50
  • BTW: Port 135 is not the only port used by MSDTC. DTC also use a range of ports to communicate. Refer to this question in SO: http://stackoverflow.com/questions/673806/msdtc-how-many-ports-are-needed – Zee Nov 24 '15 at 18:03
  • I can successfully use DTCTester from another machine to the SQL server. It's only this one machine. I talked to the network team and there are no firewall rules internally. – Bensonius Nov 24 '15 at 18:05
0

Not sure if this will help but thought I'd mention it.

From BOTH servers:

Start -> Admin Tools -> Component Services

Expand Component Services -> Computers -> My Computer -> Distributed Transaction Coordinator and right-click Local DTC. Go to Security tab and check over the settings there.

  • Enable Network DTC Access
  • Allow Remote Clients
  • Allow Inbound/Outbound as required
  • Select correct authentication
  • Enable XA Transactions as required

MSDTC Service should auto restart. These settings could perhaps have changed since Friday? I have had this happen before for reasons unknown

Rodders
  • 2,425
  • 2
  • 20
  • 34
0

Wow, I finally figured it out. As most people said, it MUST be some kind of network issue (and I didn't disagree). The kicker was that my PC was allowed DTC from it to SQL, but the VM running on my PC didn't. What it ended up being was that we were pushed to install Symantec Endpoint Protection just last week (right before I left for the weekend).

I uninstalled it and all it working now.

Bensonius
  • 1,501
  • 1
  • 15
  • 39