BizTalk Null Adapter with Installer Script

It comes up a few times a year where I need to make messages in BizTalk simply so away.  For that, I have nearly always turned to the good ol’ BizTalk Null Adapter from Tomas Restrepo.  Well, to be honest it still works fine but has a little dust on it.  Also, the installation process was still manual and wasn’t very polished.

Open source, right?

I decided to update the adapter to ensure it works with BizTalk 2013 R2 as well as to add a simple PowerShell install script to work on both 32- and 64-bit systems.  I also tossed together a little nicer of a Readme and pushed it all up to Github.

Hope someone else finds this useful!


BAM Alerts and cross-database ownership chaining issue

BizTalk offers up many surprises to me even after about 8 years of constant experience with it.  Today’s lesson to me was a permission error we were getting in the BAMAlertsApplication stating that our SQL Notification Service user no longer had permissions to SELECT data from some of the tables.  I had never run into this before … ever … not in 8 years.

After much digging, we determined that starting the BAM notification server was calling a stored procedure in the BAMAlertsNSMain database: NSGetApplicationActivationStatus.  This stored procedure was, in turn, querying off tables in BAMAlertsApplication such as NSProcessActivationStates.  So why was this previously working but no longer?  We looked at permissions granted to this service account and it never had SELECT permissions on these tables, only EXEC rights to the stored proc.  From past experience, I knew there was some sort of “loop-hole” in which a stored procedure could return results from a table the user did not have permissions to.  This is, apparently, called ownership chaining and to make matters worse this was doing a cross-database join.  The setting allows this only if the table, stored procedure and databases are all owned by the same user.

After poking around awhile in the database and server options we found an option called: “Cross-database ownership chaining” which was turned off for the server as well as for these databases. Consulting another BizTalk setup that was working I noticed that this setting was turned on for these two databases but no others.  As luck would have it, the DBA had just detached and re-attached the database files to move them to a different location and a side-effect of this is that this one particular setting as noted here.  Seriously, this is the ONLY side-effect of re-attaching database files.  Crazy.

To fix this you can either grant the service account correct access to all the tables, etc., in the database or restore this database option.  Of course, it would have been better if Microsoft properly setup permissions on these databases in the first place.  Oh … and make sure you don’t change the database owner to an account that does not also own all the objects in these databases.


WCF Web Service Latency on BizTalk 2010

I was recently working with a client to roll-out some WCF web services to BizTalk 2010 which access an Oracle database to facilitate the query (also using the WCF-OracleDb adapter).  As expected, we were getting some pretty serious latency issues (1-2 seconds per request) as BizTalk is tuned for throughput and not low-latency.  This, however, is not really an acceptable solution for web services.

To fix this, we needed to adjust some BizTalk tuning parameters to improve the latency.  Specifically, we looked at the polling interval for messages on a given host.  By default, it is set a 500 ms which means we’ll most likely have a MINIMUM of .5 seconds to process the request and most likely more as the message is sent to the orchestration, to the WCF-OracleDb adapter, and then back to the orchestration.

We decided to create a new BizTalk low latency in-process host to accomodate our web services.  Fortunately for us, BizTalk 2010 makes it even easier to set some of the previously registry-based on inaccessible tuning parameters.  After creating our host and host instances, it was pretty straight-forward:

  1. Open the Admin console, right-click on the BizTalk group and select settings
  2. Under the Hosts node, we selected our low latency host from the drop-down
  3. Now, under the Polling intervals we set both Messaging and Orchestration to 50 ms instead of 500 ms
    1. A better practice would be to have split receive, process, and send hosts and set the polling more specifically.   This was not required in our case.
  4. After selecting the host in the bindings (orchestration and send ports, not needed for the isolated receive port) and restarting we got the average response time down to 300 ms which puts us at the mercy of the Oracle databases response time. 

We also adjusted the internal message queue size for the host to ensure more messages can be kept in memory for even lower latency.

Take a look at this article for more specific details.


Getting off the ground with the WCF-OracleDB adapter in BizTalk

Time and time again I walk into a client which uses Oracle and I need to connect BizTalk to it.  And … time and time again I run into issues getting the correct Oracle client installed, then getting Visual Studio to pick that up and so forth.  Finally, I think I have arrived at a formula for getting things working.

Step 1: Install the WCF LOB Adapter pack

Ensure that you have installed at least the 32-bit version on a development workstation as well as the 64-bit version for any 64-bit environments.  The BizTalk 2010 installation program does a nice job of walking you through installing the SDK and then the LOB packs so I won’t go into any more details.

Step 2: Obtain and install the ODAC client

Technically, the adapter pack is compatible with an older 11g client but this will be taken care of via a slew of Publisher Policy’s that will “redirect” your client to the installed version.  To get the client you can search for ODAC1110720 or go to Oracle’s Windows download page and locate the ODAC client links and then the specific version listed above.  I honestly have not tested with the newer 11g clients but perhaps the same/similar process will work with them.

Step 3: Update the adapter pack’s assmbly binding

Locate the Microsoft.Adapters.OracleDB.Config file in the Adapter Pack’s install directory (c:\Program Files\Microsoft BizTalk Adapter Pack\bin) and add the following XML snippet in the <assemblyBinding…> element:

<assemblyIdentity name="Oracle.DataAccess" publicKeyToken="89b483f429c47342" culture="neutral" />
<bindingRedirect oldVersion="" newVersion=""/>

Now this will redirect all 11g calls to th 11.7.0 client version to the installed version.

At this point, the Add Generated Items->Consume Adapter Service wizard should work to connect to your Oracle server.  Depending on your standards, you may need to get a tnsnames.ora file in the correct location OR skip TNS resolution and use the server’s direct settings in the binding configuration.

Please leave a comment if this does (or does not) work for you!

BizTalk Schema Inheritance Practices / Examples

Frequently I find myself at clients explaining BizTalk best practices and one of the items I always try and push forward is proper schema inheritance practices.  Much like object-oriented design, schema inheritance can help create a nice canonical domain model.  However, there are always questions as to which type of inheritance to use.  While these practices go beyond just BizTalk, I wanted to focus this on implementation within the BizTalk tools.

Microsoft (and others) do put out documentation on this very subject but it doesn’t really go into any good examples of each type so I plan to plug that gap here.  The first few sections go into some background and other practices … if you are just looking for examples and practices for the given types feel free to skip ahead.

Types of Schema Inheritance

To summarize, there are three ways to re-use schemas within your BizTalk solution:

  • XSD Import – Importing types from another schema/namespace
  • XSD Include – Importing types from another schema within the SAME namespace
  • XSD Redefine – Importing types from another schema with the intent on extending/overriding the types

These all sound very similar in definition but there are some key usage scenarios for each in practice.

Organization of Schemas

Before going into the re-use of schemas it is important to think of re-use of your schemas beyond just your solution and extend into the greater collection of BizTalk applications at a given organization.  This becomes even MORE crucial in the realm of BizTalk when dealing with dependency, deployment and maintenance issues.  Without getting into too much detail, here are some quick pointers:

  • When creating canonical schemas to be shared with multiple BizTalk applications they MUST be isolated in their own project/assembly.
  • Treat yourself like a 3rd party vendor when consuming these schemas: add a reference to a well-known, strongly versioned DLL of these schemas
  • Remember that BizTalk only allows a given schema message type to be defined once in the entire BizTalk group so sharing these at the source code level won’t work
  • Track these central schemas as a separate project/lifecycle so that there is a clean separation between your consuming application and these soon-to-be highly shared schemas
  • Enforce schema versioning (especially in the namespace) from the beginning.  When you roll out an update you’ll not only have to version the .NET DLL but also the schema namespace
Creating ComplexTypes

The first step in re-using other schemas is to define the reusable schema itself (obviously) but then to create actual schema types (ComplexType) out of the bits to re-use.  This is the simple part: Simply select the “record” node which you want to re-use/share and set the DataStructureType property to the name you wish to use for the type.  The following screen shot shows the data structure type property.

BizTalk - Schema Data Structure Type

BizTalk - Schema Data Structure Type

It would be a good idea to define a naming standard/practice for these types … I generally use something like the node name plus the word type.  For example if I were creating an Order schema I would create a complex type named simply: OrderType.

The following sections define a typical usage scenario for each type of schema inheritance.

XSD Import – Composition

Frequently, you have the same type of information repeated throughout your projects and solutions.  A classic example is the Address.  When defining a schema like Order, you can have several addresses all containing the same fields but representing different locations.  For example, you may want to have a shipping and billing address in your order and ensure the addresses are consistent. This is very simple to accomplish:

  1. Create a schema which represents your address, and create a ComplexType out of it as in the above example.
  2. Create a schema called Order and ensure it has a different target namespace than the Address schema
  3. Click on the <Schema> node, then open the Imports collection in the properties window
  4. In this windows you’ll now be able to “XSD Import” the address schema into the Order schema

The following screenshots depict the XSD Import of the Address schema:

BizTalk - XSD Import

XSD Import

Next, create a couple of Child Records nodes for holding a Billing and a Shipping address. In the properties window for each record, set the Base Data Type to the complex type created above.

BizTalk - Base Data Type

Base Data Type

Now as the address type is changed, it will be reflected in your Order schema as well.

Another, perhaps more useful, example is adding additional context details to a schema.  Take, for example, the Order Schema created above.  If you wanted to add additional process context items you could wrap the order schema into another schema (call it: MyProcessOrder) where a child record node represents the Order and additional items are added as sibling nodes.  Using this method isolates your Order schema as the “Order entity” without poluting it with non-Order related attributes.

XSD Include – Batch Schemas

XSD Includes are slightly different from Imports in that the target namespaces of the schema and the included schema must match.  The primary use case I have used here is when creating envelope schemas used for debatching.  Continuing the above Order example, convert the Order schema to a ComplexType (using the Data Structure Type node) so it can be included in another schema.

Use the following steps to create an Orders schema containing one or more Order records:

  1. Create a new schema called Orders
  2. Ensure the target namespace of the Order and Orders schema are IDENTICAL
  3. Open the Imports collection (same as above) and now you can “XSD Include” the Order schema
  4. Now, add a child record and set it’s type to the OrderType created above

The following screen shots show including the Order schema into the Orders envelope schema:

XSD Include

XSD Include

And then creating the Order record …

Order Record

Order Record

Since this will be used for debatching, a few more properties need set at:

  • Envelope: Yes
  • Root Reference: Orders
  • On the Orders node, set the Body Xpath property to point to the Order node
  • On the Order node, set the maxOccurs to unbounded
XSD Redefine – Embrace and Extend

The XSD Redefine is probalby the least uses type of schema reuse for me as I genreally find myself using it to “hack” around another schema or trying to over-simplify something … and actually making it more cumbersome in the end.

An easy example of this is to take the AddressType above and create a new type from it to include some phone numbers:

  1. Create a new schema called AddressPhoneType
  2. Ensure the namespace on AddressPhoneType matches that of AddressType above
  3. As above, use the Imports property to “XSD Redefine” the Address schema
  4. Add a new child record and set its Data Structure Type property to AddressType (this is different from earlier where the Base Data Type property was used)
  5. Now, add a new Sequence Group to the new child record and customize!

The following screen shot depicts the AddressPhoneType schema:

XSD Redefine

XSD Redefine

Now you can change the Address schema without affecting other “consumers” of it.  Very useful if you want to embrace the re-usability of the Address type but not break aa bunch of other schemas which inherit it as well.

Now, with the above examples BizTalk developers can embrace re-usability in their BizTalk schemas along with other best practices!

BizTalk 2006 R2 configuration … still gets me

For as many times as I have set up BizTalk you would think I have seen it all.  But the sad truth is that I run into a unique problem each and every time.

This time, I had some initial issues getting even the SSO up-and-running but I had forgotten some of the basics (disabling shared memory, etc.).  Then I had some spelling mistakes in the group names (sloppy on my part) and then I forgot to add myself to one of the groups (again, sloppy).  Finally, I had most of these housekeeping items taken care of and then get the SSO up.

Next, when I tried to set up the Group feature I kept getting a “logon failed” for the BizTalkRuleEngineDB.  But … there was no BizTalkRuleEngineDb … and I was a sysadmin on SQL.  WTF?!?

Well … as it turns out the very first time I had attempted to configure BizTalk the Rules Engine configuration had actually succeeded.  However, I went in and deleted the existing databases when I was trying to get the SSO up-an-running.  BizTalk thought there should have been a rules engine database and was failing trying to “log in” to it.

To fix it, I unconfigured the Rules Engine feature and everything went smoothly going forward.  Would have been nice to get a “database ain’t there” error message!

BizTalk and DFS share permissions

We were transitioning our BizTalk application to a new server which was leveraging DFS for our folders/storage.  Every time we tried to start our receive locations they would disabled immediately with a permissions error in the event log such as: “File transport does not have read/write privileges for receive location.” 

Now, I have experienced this error on plenty of occasions where I simply did not grant the BizTalk host users group the correct access … but for this DFS share we setup Full Control permissions using the DFS admin console.

The Fix:
The problem, as it turned out, is as obvious as it seems … sort of.  As I was told, the “normal” way of gaining access to a DFS share is via the DFS admin console.  Which we verified when we logged into the server as the BizTalk host account … we could create files and delete them, etc.

The real problem was that BizTalk could not verify that it actually had permissions (or that I had not 100% appropriately assigned the permissions).  All I had to do is navigate to the folder and setup the ACLs like I normally would on any other shared file folder.  Everything worked fine after doing that.

BizTalk 2009 HTTPs connection closed issue

I was trying to send an AS2 payload to a trading partner over HTTPS.  Normally, I rely on AS2’s built-in security but this particular partner requires me to use HTTPS as well.

When I attempt to send a message it fails immediately with an event in the log indicating: “The underlying connection was closed: An unexpected error occurred on a send.”  I had tested HTTPS communication between two BizTalk instances and it worked fine but the client is not using BizTalk.  In fact, their solution was based on webMethods which uses Apache as the web server.

Potential Causes / Solutions:
There are several things that could be the issue in HTTPs communication:

1. Apache does not support TLS … only SSL3.0 
By default, newer Microsoft operating systems attempt to communicate with TLS first then allow the client and server to renegotiate (renego) to a common cipher/protocol.  According to this article and this article I could disable the SCHANNEL client from trying to use TLS and default to SSL3 instead.  However, after making these changes and restarting nothing changed.

2. The target server is expecting something … else.
My next thought was that simply there was something missing in our communication.  It was obviously something to do with HTTPS and not AS2 since the communication was failing immediately on the HTTPS send port vs. an AS2 communication issue.  But I was unsure how to test this.  After a bit of searching I came across a recommendation for Wfetch.

Wfetch is a tool by Microsoft that is used to diagnose HTTP(S) communications against any web server (IIS included, of course).  Still suspecting that it was a cipher/protocol issue I first tried to communicate with plain old HTTPS:

 As you can see there was some sort of error that was returned: “An unknown error occurred while processing the certificate.” Hmmm … maybe something to do with not being able to negotiate a protocol.  So I changed the protocol to use TLS 3 and ran the test again.

Now I get a different, more specific error about not being able to negotiate a common cipher.  So at least that proves that TLS won’t work.  After changing the protocol to SSL3 I got the same error as above … something about a certificate.

Then it occurred to me that the server requires a client certificate for authentication!  Using Wfetch I selected a certificate out of my Personal certificate store (which was a fully trusted certificate from a CA) and I finally succeeded! 


BizTalk Solution:
To solve this in BizTalk all I needed to do is somehow add a client certificate to the outbound HTTPS requests for this partner.  In the HTTP configuration of my send adapter, there is an authentication tab where I just pasted in the thumbprint of my client certificate and everything is working just fine.  The servers are able to renego down to SSL3 so I didn’t even have to disable TLS.

BizTalk AS2 and Chunked Encoding (Disable it!)

In a recent project with a client we were configuring our messaging-only AS2 solution for moving data to and from their customers via AS2 (HTTP).  We had successfully moved a few customers on to the BizTalk 2009 solution already but ran into a strange issue while testing another client.
During testing, we generally used smaller test messages that could be easily detected and thrown away so we did not pick up on this problem initially.  My client stated noticing that when larger AS2 payloads were sent (by larger I mean maybe 1 kbyte and larger files so not big) we were getting HTTP connection errors.  The error message was: "Unable to read data from the transport connection.  The connection was closed.”  It appeared the remote HTTP server was closing the connection sooner than BizTalk expected.  Strangely enough, the payload seemed to go though most of the time anyway.
Originally, while researching this issue we came across many anecdotes about HTTP Keep Alives and how some Apache servers did not jive well with using that feature so we tried turning them off in our AS2 party configuration.  Nothing changed, however.  We investigated everything from firewall logs on both sides to considering sniffing the wire to see what the HTTP communication was doing.
The Fix
In the end, nothing we tried fixed the issue so I went back to the basics, read through all the Microsoft documentation and AS2 walkthroughs when finally I saw something that caught my eye: "Enable Chunked Encoding" was explicitly disabled/unchecked on the HTTP send ports.  There was no rationale as to why this was done so it very quickly fell through the cracks.  As soon as we unchecked Enable Chunked Encoding on the send ports of this customer the problem went away and we’ve been sailing smooth ever since.

BizTalk BAM Portal failure and configuration woes

Well … BizTalk just never fails to surprise me with the "one-thing-after-another" issues which I run into…
Symptom …
I did a fres BizTalk 2009 install on a virtual machine for some PoC deveopment … went to prepare a demo of BAM and the BAM Portal and when I hit the website it just failed with a generic message.  Well … as is standard practice at this point it MUST have been something I did wrong … so I decided I needed to fix the issue by re-configuring the BAM Portal.
The First Try …
For some reason, I had the bright idea that I should delete the BAM Portal web applications and app pool before un-configuring. I did that, restarted IIS and ran the BizTalk configuration tool.  Next, I un-configured the BAM Portal and completed the wizard … no problems, right? Of course not!  When I went back into the BizTalk Configuration Tool to configure the BAM Portal feature the services accounts and website options were disabled!
The Second Try …
It turns out that the BAM database was still configured to point to the original website even though I had unconfigured that feature.  So, according the this blog post I needed to use BM.exe to remove that configuration information.
The Fix…
  1. Open the command prompt and find your way to BM.exe (Program FilesBizTalk …tracking)
  2. Retrieve the current configuration information:
    bm get-config -FileName:"BAM_Config.xml"
  3. Update the config file to remove the reference to the current website:
    <GlobalProperty Name="BAMVRoot">http://blah.blah.remove.this.URL</GlobalProperty&gt;
  4. Update the BAM configuration database:
    bm update-config -FileName:"BAM_Config.xml"
  5. Re-run the BizTalk Configuration Tool and re-configure the BAM Portal feature and you should be good to go


Nothing in BizTalk is as it seems Wink