BizTalk Schema Inheritance Practices / Examples

Frequently I find myself at clients explaining BizTalk best practices and one of the items I always try and push forward is proper schema inheritance practices.  Much like object-oriented design, schema inheritance can help create a nice canonical domain model.  However, there are always questions as to which type of inheritance to use.  While these practices go beyond just BizTalk, I wanted to focus this on implementation within the BizTalk tools.

Microsoft (and others) do put out documentation on this very subject but it doesn’t really go into any good examples of each type so I plan to plug that gap here.  The first few sections go into some background and other practices … if you are just looking for examples and practices for the given types feel free to skip ahead.

Types of Schema Inheritance

To summarize, there are three ways to re-use schemas within your BizTalk solution:

  • XSD Import – Importing types from another schema/namespace
  • XSD Include – Importing types from another schema within the SAME namespace
  • XSD Redefine – Importing types from another schema with the intent on extending/overriding the types

These all sound very similar in definition but there are some key usage scenarios for each in practice.

Organization of Schemas

Before going into the re-use of schemas it is important to think of re-use of your schemas beyond just your solution and extend into the greater collection of BizTalk applications at a given organization.  This becomes even MORE crucial in the realm of BizTalk when dealing with dependency, deployment and maintenance issues.  Without getting into too much detail, here are some quick pointers:

  • When creating canonical schemas to be shared with multiple BizTalk applications they MUST be isolated in their own project/assembly.
  • Treat yourself like a 3rd party vendor when consuming these schemas: add a reference to a well-known, strongly versioned DLL of these schemas
  • Remember that BizTalk only allows a given schema message type to be defined once in the entire BizTalk group so sharing these at the source code level won’t work
  • Track these central schemas as a separate project/lifecycle so that there is a clean separation between your consuming application and these soon-to-be highly shared schemas
  • Enforce schema versioning (especially in the namespace) from the beginning.  When you roll out an update you’ll not only have to version the .NET DLL but also the schema namespace
Creating ComplexTypes

The first step in re-using other schemas is to define the reusable schema itself (obviously) but then to create actual schema types (ComplexType) out of the bits to re-use.  This is the simple part: Simply select the “record” node which you want to re-use/share and set the DataStructureType property to the name you wish to use for the type.  The following screen shot shows the data structure type property.

BizTalk - Schema Data Structure Type

BizTalk - Schema Data Structure Type

It would be a good idea to define a naming standard/practice for these types … I generally use something like the node name plus the word type.  For example if I were creating an Order schema I would create a complex type named simply: OrderType.

The following sections define a typical usage scenario for each type of schema inheritance.

XSD Import – Composition

Frequently, you have the same type of information repeated throughout your projects and solutions.  A classic example is the Address.  When defining a schema like Order, you can have several addresses all containing the same fields but representing different locations.  For example, you may want to have a shipping and billing address in your order and ensure the addresses are consistent. This is very simple to accomplish:

  1. Create a schema which represents your address, and create a ComplexType out of it as in the above example.
  2. Create a schema called Order and ensure it has a different target namespace than the Address schema
  3. Click on the <Schema> node, then open the Imports collection in the properties window
  4. In this windows you’ll now be able to “XSD Import” the address schema into the Order schema

The following screenshots depict the XSD Import of the Address schema:

BizTalk - XSD Import

XSD Import

Next, create a couple of Child Records nodes for holding a Billing and a Shipping address. In the properties window for each record, set the Base Data Type to the complex type created above.

BizTalk - Base Data Type

Base Data Type

Now as the address type is changed, it will be reflected in your Order schema as well.

Another, perhaps more useful, example is adding additional context details to a schema.  Take, for example, the Order Schema created above.  If you wanted to add additional process context items you could wrap the order schema into another schema (call it: MyProcessOrder) where a child record node represents the Order and additional items are added as sibling nodes.  Using this method isolates your Order schema as the “Order entity” without poluting it with non-Order related attributes.

XSD Include – Batch Schemas

XSD Includes are slightly different from Imports in that the target namespaces of the schema and the included schema must match.  The primary use case I have used here is when creating envelope schemas used for debatching.  Continuing the above Order example, convert the Order schema to a ComplexType (using the Data Structure Type node) so it can be included in another schema.

Use the following steps to create an Orders schema containing one or more Order records:

  1. Create a new schema called Orders
  2. Ensure the target namespace of the Order and Orders schema are IDENTICAL
  3. Open the Imports collection (same as above) and now you can “XSD Include” the Order schema
  4. Now, add a child record and set it’s type to the OrderType created above

The following screen shots show including the Order schema into the Orders envelope schema:

XSD Include

XSD Include

And then creating the Order record …

Order Record

Order Record

Since this will be used for debatching, a few more properties need set at:

  • Envelope: Yes
  • Root Reference: Orders
  • On the Orders node, set the Body Xpath property to point to the Order node
  • On the Order node, set the maxOccurs to unbounded
XSD Redefine – Embrace and Extend

The XSD Redefine is probalby the least uses type of schema reuse for me as I genreally find myself using it to “hack” around another schema or trying to over-simplify something … and actually making it more cumbersome in the end.

An easy example of this is to take the AddressType above and create a new type from it to include some phone numbers:

  1. Create a new schema called AddressPhoneType
  2. Ensure the namespace on AddressPhoneType matches that of AddressType above
  3. As above, use the Imports property to “XSD Redefine” the Address schema
  4. Add a new child record and set its Data Structure Type property to AddressType (this is different from earlier where the Base Data Type property was used)
  5. Now, add a new Sequence Group to the new child record and customize!

The following screen shot depicts the AddressPhoneType schema:

XSD Redefine

XSD Redefine

Now you can change the Address schema without affecting other “consumers” of it.  Very useful if you want to embrace the re-usability of the Address type but not break aa bunch of other schemas which inherit it as well.

Now, with the above examples BizTalk developers can embrace re-usability in their BizTalk schemas along with other best practices!

Test Flight: Simplified Beta Deployments for iOS Apps

One of the more challenging things in creating iOS applications is the whole provisioning and deployment process.  This becomes especially true during beta testing of your application using Ad-Hoc distribution.

The process is something like this:

  1. Gather all the UDIDs from your test subjects
  2. Enter these into the iOS Provisioning Portal
  3. Create a distribution certificate (if you haven’t already)
  4. Create an Ad-Hoc provisioning profile
  5. Download the profile and certificate
  6. Configure and build your app signing with these credentials

Then, your testers need to get the application and install it.  Generally like this:

  1. Email the testers the provisioning profile and app bundle
  2. The tester need to drag both to iTunes (or Organizer) and then tether and sync

This makes it a little cumbersome for most testers and difficult for non-tech savvy users.  One could post the app bundle to a website with a manifest and then create a launch page (like an internal app store) and then notify the users.  This is where Test Flight (www.testflightapp.com) comes in … managing the web-delivery of your application in a really easy-to-use process.

Remembering the above, now I do this:

  1. (optional) Ask your test users to register their devices
  2. (optional) Create a team out of these testers and download their device IDs
  3. (optional) Upload all of these device IDs at one time to the Provisioning Portal
  4. Follow the same process as above to create a provisioning profile and build
  5. Now upload the IPA bundle to Test Flight and I am done

At this point, the testers can receive a notification of the new build (and subsequent builds) and simply visit Test Flight on their device.  They will be able to install your beta app straight from the browser.  Further, I can report on who has installed the app and who has not.

All this and the website has a nice easy to use design on top of that!

Updated iPad view resizing for keyboard…

There were a couple of glitches in the code I had originally posted in “iPad: resize view to account for keyboard” especially when dealing with a scroll view within a split-view controller, etc.

Originally, I was using the “setContentOffset” method on the UIScrollView:

[scrollView setContentOffset:scrollPoint];

This was problematic depending on the orientation and so forth.  Frankly, I could never get it to behave the way I wanted.  After much debugging I looked at some other methods and found: scrollRectToVisible.  Well … that sounds like just what I need and no longer have to worry about calculating the perfect position for the scroll point.  Here was the updated line:

[scrollView scrollRectToVisible:activeField.frame animated:YES];

There. Much better! See the original post for the rest of the tutorial.

SNLog – Simple logging framework for iOS

[SNLog Project Page Link]

Logging

A simple task that every developer runs into when creating software is proper logging and tracing.  What I have found is the simplest solution for logging is always the best.

Requirements are almost always the same:

  1. Need to log to the console when debugging
  2. Need to write to a file/database for runtime tracing and diagnostics

There are lots of frameworks for just about every platform out there but logging on the iPhone seemed to be lacking so I decided to create my own as an exercise.  Something that would be applicable to my current projects as well as to help out anyone else that might need something like this.

NSLog

NSLog is the de-facto standard for logging during debugging in Xcode but is primarily temporary scaffolding in your code that needs removed prior to deploying the application.  I wanted something as simple to use as NSLog that could be configured to log somewhere more persistent than the console.

SNLog

The plan of attack on this was straightforward:

  1. Minimize the files that need included in the project (just two, the .h and .m)
  2. Make use of it as easy as NSLog
  3. Re-define NSLog to use the framework instead of the built-in function
  4. Use a strategy pattern to allow multiple logging mechanisms (console and file built-in)

In addition, I ran into the following additions while I was working on it:

  1. Don’t let the file logging grow unbounded (especially on a mobile device)
  2. Capturing the log level/importance can allow different mechanisms to capture the level that is required

So that’s it … in two simple files I was able to create a simple logging mechanism that is a drop-in replacement for NSLog.  I can keep all those frivolous NSLog statements in my code and choose where to redirect the log.

Check out the Project Page for more details and the download.

BizTalk 2006 R2 configuration … still gets me

For as many times as I have set up BizTalk you would think I have seen it all.  But the sad truth is that I run into a unique problem each and every time.

This time, I had some initial issues getting even the SSO up-and-running but I had forgotten some of the basics (disabling shared memory, etc.).  Then I had some spelling mistakes in the group names (sloppy on my part) and then I forgot to add myself to one of the groups (again, sloppy).  Finally, I had most of these housekeeping items taken care of and then get the SSO up.

Next, when I tried to set up the Group feature I kept getting a “logon failed” for the BizTalkRuleEngineDB.  But … there was no BizTalkRuleEngineDb … and I was a sysadmin on SQL.  WTF?!?

Well … as it turns out the very first time I had attempted to configure BizTalk the Rules Engine configuration had actually succeeded.  However, I went in and deleted the existing databases when I was trying to get the SSO up-an-running.  BizTalk thought there should have been a rules engine database and was failing trying to “log in” to it.

To fix it, I unconfigured the Rules Engine feature and everything went smoothly going forward.  Would have been nice to get a “database ain’t there” error message!

iPad: Resize view to account for keyboard (updated)

UPDATE (3/9/2011):

There were a couple of glitches in the original code so I found a nicer way to handle the scrolling as updated below!

Classic problem

In an iOS application the keyboard slides up and hides the view/field that the user is editing.  There are a ton of solutions out there but there are problems with just about all of them when it comes to the iPad.  Even more of an issue is the stock “Splitview” applications that we can implement using other UI views like a UIScrollView.

Requirements

First, here is a list of functions that I wanted to implement:

  • Works with a UISplitViewController
  • Works with a UIScrollView
  • Ensures the field being edited is visible
  • Uses smooth animation, not jumpy
  • Looks like a iPad application should behave
The Apple Way

What better way to implement this than to use Apple’s documentation, right? Well, as described in this document Apple has a great approach.  However, there were several problems with this implementation:

  • It was written for an iPhone
  • It does not take into account the rotation of the device
  • It does not take into account the scroll view’s current position
  • It does not take into account the split view design

However, what I do like about is that it takes advantage of simple code with minimal state management and just needed some tweaks.

Let me break it down.

First, when getting the size of the keyboard the following lines of code do NOT take in to account the rotation of the device:

NSDictionary* info = [aNotification userInfo];
CGRect kbRect = [[info objectForKey:UIKeyboardFrameBeginUserInfoKey] CGRectValue];

The good news is there is an easy way to fix this by just translating the CGRect using the current view’s convertRect method:

kbRect = [self.view convertRect:kbRect toView:nil];

Now the rectangle will be oriented correctly no matter which way the device is turned.  Ok … problem one solved.  Next up, the code didn’t take into account the split view controller design pattern. First, when calculating the CGRect which determined what was the “visible” area of the screen it ignored the toolbar as well as fact that the “detail view” was offset by the “master view” when in landscape view.

Accounting for the toolbar was easy by modifying this code:

CGRect aRect = self.view.frame; aRect.size.height -= kbRect.size.height;

… by taking into account the size of the toolbar:

aRect.size.height -= self.toolbar.frame.size.height;

Now, in order to determine if the currently selected field is visible, it’s coordinates must be translated to the same system the detail view is in.  The stock code gets the active field’s coordinates as follows:

CGPoint fieldOrigin = activeField.frame.origin;

… then as before I need to translate to a comparable system:

fieldOrigin = [self.view convertPoint:fieldOrigin toView:self.view.superview];

… and take into account the current scroll position of the UIScrollView (thanks DK_Donuts) and I end up with:

CGPoint fieldOrigin = activeField.frame.origin;
fieldOrigin.y -= scrollView.contentOffset.y;
fieldOrigin = [self.view convertPoint:fieldOrigin toView:self.view.superview];

Now we use this magical method: scrollRectToVisible and the scroll view takes over for us:

[scrollView scrollRectToVisible:activeField.frame animated:YES];

As far as making the positioning smooth and behaving like an iPad app, I need to save the original offset …

originalOffset = scrollView.contentOffset;

…and then reset the position when the keyboard is hidden.

[scrollView setContentOffset:originalOffset animated:YES];

The animation smooths it all out. Finally … something that works the way I expected!

The Code

Below is the listing of the code as modified to work as I needed it.

1. Add some instance variables

CGPoint originalOffset;
UIView *activeField;

2. Create the keyboard notification handlers

- (void)keyboardWasShown:(NSNotification*)aNotification
{
    NSDictionary* info = [aNotification userInfo];
    CGRect kbRect = [[info objectForKey:UIKeyboardFrameBeginUserInfoKey] CGRectValue];
	kbRect = [self.view convertRect:kbRect toView:nil];
    UIEdgeInsets contentInsets = UIEdgeInsetsMake(0.0, 0.0, kbRect.size.height, 0.0);
    scrollView.contentInset = contentInsets;
    scrollView.scrollIndicatorInsets = contentInsets;

    CGRect aRect = self.view.frame;
    aRect.size.height -= kbRect.size.height;
    aRect.size.height -= self.toolbar.frame.size.height;
    CGPoint fieldOrigin = activeField.frame.origin;
    fieldOrigin.y -= scrollView.contentOffset.y;
    fieldOrigin = [self.view convertPoint:fieldOrigin toView:self.view.superview];
    originalOffset = scrollView.contentOffset;
    if (!CGRectContainsPoint(aRect, fieldOrigin) ) {
        [scrollView scrollRectToVisible:activeField.frame animated:YES];
    }
}

… and …

- (void)keyboardWillBeHidden:(NSNotification*)aNotification {
     UIEdgeInsets contentInsets = UIEdgeInsetsZero;
     scrollView.contentInset = contentInsets;
     scrollView.scrollIndicatorInsets = contentInsets;
     [scrollView setContentOffset:originalOffset animated:YES];
 }

3. Wire up the notifications in viewDidLoad

[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(keyboardWasShown:)
    name:UIKeyboardDidShowNotification object:nil];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(keyboardWillBeHidden:)
    name:UIKeyboardWillHideNotification object:nil];

4. It is up to you to wire up the activeField ivar depending on your implementation.

UITextField.Text property = EXC_BAD_ACCESS

Ok … so while I have a couple iPhone/iPad apps under by belt I am still “learning” a view things.  Memory management on these devices seems to be primarily an “art” vs. pure science.  Here is an example of that.

My typical rule is if I created the object then I release it … not 100% fool-proof but it at least gets me thinking in the correct direction.  In this case it messed me up!

I have several UITextField views on my view and I was creating a method to load the values into these fields from the database.  Then the strange behavior stared.  If I had a “space” in the value it would crash upon loading the data into the field with an EXC_BAD_ACCESS in obj_msgsend().  If it did NOT have a space in the value it would only crash when I tried to reference the field’s text value when trying to save subsequently updated data.

The short story was that I was creating an NSString, loading it with data, assigning it to UITextField.text and then releasing the NSString.  This was the issue … this was also causing the text property to get released and was eventually causing the EXC_BAD_ACCESS.

UPDATE:  After going back and re-reviewing the code what I was doing was using a convenience method on a NSString instead of an alloc which meant it was put into the autorelease pool and thus it was getting release two times in that method … once when I release it and once when the pool was drained (and, of course, again when the UITextField was released).

Awesome.

BizTalk and DFS share permissions

Problem:
We were transitioning our BizTalk application to a new server which was leveraging DFS for our folders/storage.  Every time we tried to start our receive locations they would disabled immediately with a permissions error in the event log such as: “File transport does not have read/write privileges for receive location.” 

Now, I have experienced this error on plenty of occasions where I simply did not grant the BizTalk host users group the correct access … but for this DFS share we setup Full Control permissions using the DFS admin console.

The Fix:
The problem, as it turned out, is as obvious as it seems … sort of.  As I was told, the “normal” way of gaining access to a DFS share is via the DFS admin console.  Which we verified when we logged into the server as the BizTalk host account … we could create files and delete them, etc.

The real problem was that BizTalk could not verify that it actually had permissions (or that I had not 100% appropriately assigned the permissions).  All I had to do is navigate to the folder and setup the ACLs like I normally would on any other shared file folder.  Everything worked fine after doing that.

BizTalk 2009 HTTPs connection closed issue

Problem:
I was trying to send an AS2 payload to a trading partner over HTTPS.  Normally, I rely on AS2’s built-in security but this particular partner requires me to use HTTPS as well.

When I attempt to send a message it fails immediately with an event in the log indicating: “The underlying connection was closed: An unexpected error occurred on a send.”  I had tested HTTPS communication between two BizTalk instances and it worked fine but the client is not using BizTalk.  In fact, their solution was based on webMethods which uses Apache as the web server.

Potential Causes / Solutions:
There are several things that could be the issue in HTTPs communication:

1. Apache does not support TLS … only SSL3.0 
By default, newer Microsoft operating systems attempt to communicate with TLS first then allow the client and server to renegotiate (renego) to a common cipher/protocol.  According to this article and this article I could disable the SCHANNEL client from trying to use TLS and default to SSL3 instead.  However, after making these changes and restarting nothing changed.

2. The target server is expecting something … else.
My next thought was that simply there was something missing in our communication.  It was obviously something to do with HTTPS and not AS2 since the communication was failing immediately on the HTTPS send port vs. an AS2 communication issue.  But I was unsure how to test this.  After a bit of searching I came across a recommendation for Wfetch.

Wfetch is a tool by Microsoft that is used to diagnose HTTP(S) communications against any web server (IIS included, of course).  Still suspecting that it was a cipher/protocol issue I first tried to communicate with plain old HTTPS:

 As you can see there was some sort of error that was returned: “An unknown error occurred while processing the certificate.” Hmmm … maybe something to do with not being able to negotiate a protocol.  So I changed the protocol to use TLS 3 and ran the test again.

Now I get a different, more specific error about not being able to negotiate a common cipher.  So at least that proves that TLS won’t work.  After changing the protocol to SSL3 I got the same error as above … something about a certificate.

Then it occurred to me that the server requires a client certificate for authentication!  Using Wfetch I selected a certificate out of my Personal certificate store (which was a fully trusted certificate from a CA) and I finally succeeded! 

 

BizTalk Solution:
To solve this in BizTalk all I needed to do is somehow add a client certificate to the outbound HTTPS requests for this partner.  In the HTTP configuration of my send adapter, there is an authentication tab where I just pasted in the thumbprint of my client certificate and everything is working just fine.  The servers are able to renego down to SSL3 so I didn’t even have to disable TLS.