Conquering NServiceBus part 5 – Troubleshooting DTC

by Mikael Henriksson 1. April 2010 15:50

This post is part of a series and the source code can be found at http://github.com/MikaelHenrixon/ConqueringNServiceBus

  1. Conquering NServiceBus part 1 – Getting Started
  2. Conquering NServiceBus part 2 – Initial configuration
  3. Conquering NServiceBus part 3 – A simple Saga
  4. Conquering NServiceBus part 4 – Testing
  5. Conquering NServiceBus part 5 – Troubleshooting DTC
  6. Conquering NServiceBus part 6 – Upgrading StructureMap

NServiceBus uses Distributed Transactions Coordinator to manage the messages from when they arrive in the queue to when the Unibus has completed the handling and in the Sagas case moved them to at least a temporary storage of your choosing. This is great because you can specify how many times a message will be retried before moved to the error queue. It can also be a major pain in the ass if the storage of your choosing happens to be on another server. Personally I’d recommend setting up some sort of virtual environment to test on in case you fudged up something and don’t know what to restore. Fortunately it ships some tools to help you out with NServiceBus what you should be looking for is a folder called “msmqutils”. Inside this folder is an executable called "Runner”. This will uninstall DTC and MSMQ if anything is incompatible with NServiceBus and install it so that it works however it does not solve all problems but first things first. If you have a problem with DTC this is where you should start. The next thing I’d like to recommend is to download a copy of DTCPing. This is a great tool for tracking down DTC issues.

So how do you really make things work? Well there is obviously a lot more to it that I can possibly cover in one blog post but let me walk you through how I got DTC to work over the internet.

First thing I did was to run RunMeFirst.bat and it did not look like below image. I had all kinds of incompatible things running on my servers. The below is how you WANT it to look. It could be that you actually need a restart after this so save your work.

output from the msmqutils runner when successful and compatible

Very verbose and clear.

From here on I went down a completely wrong path. I tried about every possible wrong and unrelated thing I could think of until NH Prof saved my ass with a message about failing transactions actually at this time NServiceBus spammed my server with about 1MB of debug messages per second but the message was not one I could make much sense of and the mere size of the log made it impossible to open when I finally found out that something was terribly wrong. Obviously I need to try something new so i fire up DTCPing. The image below from DTCPing was NOT the image I was looking at. At that time there was a message about the RPC Endpoint Mapper not being available so I had to Google it.

this is what you the initial DTCPing window to look like.

Google is great! I found tons of articles about DTC and ping so I started with firing up “d comcnfg” from the cmd prompt. And according to articles I found on the internet there was a few steps to follow. After you start dcomcnfg you should be faced with something like image 1. From here I chose to address the web server first. What is needed is to open a few ports (a thousand ports actually… just to be sure!) by right clicking on “My Computer” and selecting properties  see image 2.

1. this is the component services and dtc configuration 

2. dtc_dcomcnfg_2

 

 

If this does not work you can also modify the registry to achieve the same. One thing to note though is that you really really REALLY need to restart your computer for this to take but wait until you have completed the security settings of the Local DTC. I spent a good few minutes trying to figure out why my new settings did not work even though I was promised it should work. On a side note it’s probably a really bad idea to open up one thousand ports like this so try and keep it to a hundred!

Click ok two times and then right click, select properties on local DTC. Your settings should probably equal mine.

dtc security settings

Time to restart the computer!

DTCPing is still not working, now I do all the other necessary things like open the firewall for necessary ports. I even remove the DTC exceptions in the firewall and add them fresh just to make sure. Eventually I got a message from DTCPing about finding the name but not reaching it! Let me recommend that the first thing you do when trying to debug TCP/IP problems is to do a simple ping you know fire up the command prompt and do “ping computer_name”. Here I discover that my web server has a problem with a DNS entry because it cannot be reached by ping and had I used the computer name instead of the IP address for my remote desktop I would have also discovered this a bit earlier. Come to think of it I remember changing the remote desktop session to use the ip address when we changed IP on the server a few weeks ago thinking that the DNS issue would resolve itself automatically after a few reboots and what not…. Computer problems usually don’t solve themselves all of a sudden and it caused me a couple of hours or more in debugging and tweaking but I am relieved I finally made it work on the web server so I move on to the database server. Repeat the steps above for the db server and restart the computer.  I was still not done though, I forgot to check the ports in use on the server and the database was configured for TCP port 5000 so I switched the db server to port 4000 and finally after a long time made everything run in transactional!

Tags: , ,

NServiceBus | SQL Server | Windows Server

blog comments powered by Disqus

About the author

Life architect specialized in programming

Month List

Widget Twitter not found.

Root element is missing.X