BizTalk 2009 HTTPs connection closed issue

I was trying to send an AS2 payload to a trading partner over HTTPS.  Normally, I rely on AS2’s built-in security but this particular partner requires me to use HTTPS as well.

When I attempt to send a message it fails immediately with an event in the log indicating: “The underlying connection was closed: An unexpected error occurred on a send.”  I had tested HTTPS communication between two BizTalk instances and it worked fine but the client is not using BizTalk.  In fact, their solution was based on webMethods which uses Apache as the web server.

Potential Causes / Solutions:
There are several things that could be the issue in HTTPs communication:

1. Apache does not support TLS … only SSL3.0 
By default, newer Microsoft operating systems attempt to communicate with TLS first then allow the client and server to renegotiate (renego) to a common cipher/protocol.  According to this article and this article I could disable the SCHANNEL client from trying to use TLS and default to SSL3 instead.  However, after making these changes and restarting nothing changed.

2. The target server is expecting something … else.
My next thought was that simply there was something missing in our communication.  It was obviously something to do with HTTPS and not AS2 since the communication was failing immediately on the HTTPS send port vs. an AS2 communication issue.  But I was unsure how to test this.  After a bit of searching I came across a recommendation for Wfetch.

Wfetch is a tool by Microsoft that is used to diagnose HTTP(S) communications against any web server (IIS included, of course).  Still suspecting that it was a cipher/protocol issue I first tried to communicate with plain old HTTPS:

 As you can see there was some sort of error that was returned: “An unknown error occurred while processing the certificate.” Hmmm … maybe something to do with not being able to negotiate a protocol.  So I changed the protocol to use TLS 3 and ran the test again.

Now I get a different, more specific error about not being able to negotiate a common cipher.  So at least that proves that TLS won’t work.  After changing the protocol to SSL3 I got the same error as above … something about a certificate.

Then it occurred to me that the server requires a client certificate for authentication!  Using Wfetch I selected a certificate out of my Personal certificate store (which was a fully trusted certificate from a CA) and I finally succeeded! 


BizTalk Solution:
To solve this in BizTalk all I needed to do is somehow add a client certificate to the outbound HTTPS requests for this partner.  In the HTTP configuration of my send adapter, there is an authentication tab where I just pasted in the thumbprint of my client certificate and everything is working just fine.  The servers are able to renego down to SSL3 so I didn’t even have to disable TLS.

BizTalk AS2 and Chunked Encoding (Disable it!)

In a recent project with a client we were configuring our messaging-only AS2 solution for moving data to and from their customers via AS2 (HTTP).  We had successfully moved a few customers on to the BizTalk 2009 solution already but ran into a strange issue while testing another client.
During testing, we generally used smaller test messages that could be easily detected and thrown away so we did not pick up on this problem initially.  My client stated noticing that when larger AS2 payloads were sent (by larger I mean maybe 1 kbyte and larger files so not big) we were getting HTTP connection errors.  The error message was: "Unable to read data from the transport connection.  The connection was closed.”  It appeared the remote HTTP server was closing the connection sooner than BizTalk expected.  Strangely enough, the payload seemed to go though most of the time anyway.
Originally, while researching this issue we came across many anecdotes about HTTP Keep Alives and how some Apache servers did not jive well with using that feature so we tried turning them off in our AS2 party configuration.  Nothing changed, however.  We investigated everything from firewall logs on both sides to considering sniffing the wire to see what the HTTP communication was doing.
The Fix
In the end, nothing we tried fixed the issue so I went back to the basics, read through all the Microsoft documentation and AS2 walkthroughs when finally I saw something that caught my eye: "Enable Chunked Encoding" was explicitly disabled/unchecked on the HTTP send ports.  There was no rationale as to why this was done so it very quickly fell through the cracks.  As soon as we unchecked Enable Chunked Encoding on the send ports of this customer the problem went away and we’ve been sailing smooth ever since.

BizTalk BAM Portal failure and configuration woes

Well … BizTalk just never fails to surprise me with the "one-thing-after-another" issues which I run into…
Symptom …
I did a fres BizTalk 2009 install on a virtual machine for some PoC deveopment … went to prepare a demo of BAM and the BAM Portal and when I hit the website it just failed with a generic message.  Well … as is standard practice at this point it MUST have been something I did wrong … so I decided I needed to fix the issue by re-configuring the BAM Portal.
The First Try …
For some reason, I had the bright idea that I should delete the BAM Portal web applications and app pool before un-configuring. I did that, restarted IIS and ran the BizTalk configuration tool.  Next, I un-configured the BAM Portal and completed the wizard … no problems, right? Of course not!  When I went back into the BizTalk Configuration Tool to configure the BAM Portal feature the services accounts and website options were disabled!
The Second Try …
It turns out that the BAM database was still configured to point to the original website even though I had unconfigured that feature.  So, according the this blog post I needed to use BM.exe to remove that configuration information.
The Fix…
  1. Open the command prompt and find your way to BM.exe (Program FilesBizTalk …tracking)
  2. Retrieve the current configuration information:
    bm get-config -FileName:"BAM_Config.xml"
  3. Update the config file to remove the reference to the current website:
    <GlobalProperty Name="BAMVRoot">http://blah.blah.remove.this.URL</GlobalProperty&gt;
  4. Update the BAM configuration database:
    bm update-config -FileName:"BAM_Config.xml"
  5. Re-run the BizTalk Configuration Tool and re-configure the BAM Portal feature and you should be good to go


Nothing in BizTalk is as it seems Wink

BizTalk Error: Failed to grant permission to execute. (Exception from HRESULT: 0x80131418)

Here is an issue I ran into for BizTalk 2009 when using the WCF-BasicHttp or WCF-WSHttp adapter. 
I received the following error:
The adapter "WCF-WSHttp" raised an error message. Details "System.Security.Policy.PolicyException: Failed to grant permission to execute.
What was I doing?
What I was trying to do is expose a schema as a web service endpoint so that any calls to that service would dump that message straight to the message box.  Now … not wanting to expose my internal schema I created a schema definition especially for this service and a corresponding map to map it to my canonical/internal schema.  Deployed the solution.
Next, I used the handy BizTalk WCF Service Publishing Wizard to create the .svc and ports, etc.  I then updated the Receive Port to run the map.  When I tested the endpoint I got the above error.  I then removed the map and the test ran succesfully (albeit with my intended map). 
What the heck was happening?
Well … as luck would have it I had quickly ramped up this virtual server environment and put the portal on port 80 which meant WSS was running on port 80.  Upon inspecting the .NET Trust Level for my web application it was set to WSS_Minimal!!! Arggg!  I had spent way too much time on this one!
What was the fix?
Just some things to remember when setting up a web application to run a service into BizTalk:
  • Create a separate App Pool running as a user that has permission to the BizTalk Isolated Host (I just ran it as my BizTalk Iso service account as a shortcut)
  • Make sure this app pool is running as classic mode and not integrated mode
  • CHECK THE TRUST LEVEL to make sure it is appropriate … in my case I just set it to Full Trust to get it through my test
  • Don’t for get to enable the receive location, it is not enabled by the wizard … you couldn’t imagine how many times I forgot to do this while trying to figure out why my darn service isn’t working again!

Well … hope this helps someone else!

Checking in BizTalk file before editing can corrupt the file

We’ve all been there…
When you are working with a team of developers (whether two or ten),
who are collaborating on a project in Visual Studio.Net, there
certainly is the chance that you will need to add a new file to the
solution.  Of course, to do this you have to check out the project
file locking it for yourself.

Play Nice!
The courteous thing to do (karma, right?) is to add your file and
immediately check in the new project file with your newly added file
(as long as it doesn’t break the build 🙂 ).  Try doing this in
BizTalk sometime (if you already haven’t) and watch the quick
destruction of your file.  Once you start editing that file again
and check it back in you get a nice little warning about the encoding
type changing.  Visual Source Safe does not handle this very
gracefully; Visual Source Safe does not handle this at all.  You
will be fooled into thinking your file is safe because you are working
with the version on your local disk but the next time one of your team
members gets the latest version of your project the file will be toast.

What the …?
The issue here is that when you create new files in it likes to
create them as UTF-8 encoded files.  Seems like a good idea; a
nice standard encoding type.  BizTalk, however, has to be
different and save everything as UTF-16.  Once again, another
nice, standard format.  However, visual source safe is completely
oblivious to the change in encoding types an will gladly assume that
your are still managing a UTF-8 encoded file.

The Moral
You can still be a gracious team member and have a quick
turn-around on adding new files to the project but just give those
files a little edit first before you check everything in.

True != true … duh!

Ok … so once again one of the simplest of errors put me into out-thinking-myself mode.  Simply put, when you have a boolean in an XML file that goes into a BizTalk Orchestration, it is case-sensitve.
So if you get an error stating that you have invalid text for your System.Boolean type then now you know: "True" is not true … "true" is true.
Put this on under the "duh" category.

Problem naming a BizTalk operation ReceiveMessage or SendMessage

Here was a strange one that took me a while to nail down.  First, the error messages:
The keyword new is required on ‘Microsoft.Samples.BizTalk.SendMail.ReceivePortType.ReceiveMessage’ because it hides inherited member ‘Microsoft.BizTalk.XLANGs.BTXEngine.BTXPortBase.ReceiveMessage(int, Microsoft.XLANGs.Core.Envelope, Microsoft.XLANGs.BaseTypes.XLANGMessage, Microsoft.XLANGs.Core.Correlation[], Microsoft.XLANGs.Core.Context, Microsoft.XLANGs.Core.Segment)’

‘Microsoft.Samples.BizTalk.SendMail.ReceiveSend.ReceivePort’ denotes a ‘field’ where a ‘class’ was expected

Static member ‘Microsoft.Samples.BizTalk.SendMail.ReceivePortType.ReceiveMessage’ cannot be accessed with an instance reference; qualify it with a type name instead
‘Microsoft.Samples.BizTalk.SendMail.ReceivePortType.ReceiveMessage’ denotes a ‘field’ where a ‘method’ was expected
After searching for a few hours on the Internet for that one and playing around with recent changes I had made I finally had realized what I did:  I was getting a little too creative for my own good (or not creative enough, perhaps). 
I decided that Operation_1 and Operation_2 were not good names for my receive and send ports so I decided to name them something else.  Of course, in a burst of creativness I used ‘ReceiveMessage’ and ‘SendMessage’ for the names.  I had done that a while back but apparently never did a build as I made several other changes since.  Then I compiled and got the errors above on the receive port.
Figuring it had something to do with the ports I decided to just nuke them and recreate them.  I used my usual internal justification that Microsoft’s software had somehow screwed up my code (uh … it never turns out that way).  After doing this everything compiled just fine once again.  Ah Ha! It was Microsoft screwing up my code! Then I immediately went and renamed the ports once again to my ever-so-creative names and boom. 
Moral: Either get very creative with your naming or have a complete lack of creativness altogether.  Don’t sit in the middle.