Meta-Meta-Programming … and I like it

I have a client that needs to rapidly create websites that are similar in functionality but needs to make changes to the data model (sometimes significant changes) and have the UI get updated with it, turning around a change within the day.

For this, I decided to base a solution around ASP.NET Dynamic Data for two main reasons: 1. It’s scaffolding concept and 2. The “generated code” has been tested already, shortening the testing cycle as rapid changes are released.

Of course, the out of the box user experience with Dynamic Data leaves much to be desired and is a little too simple for our needs so it requires some customization.  Once the customizations are done, they can be re-used over and over again but ONLY if they are written against a meta model so that they can apply to any entity on our model, not just coded against known entities, etc.

So what’s “Programming”?

In the normal case, programming is simply putting down the instructions required to make our applications work.  This still holds true in Dynamic Data and other “frameworks” … we are just walled into a set of constraints and conventions.

So what’s “Meta-Programming”?

In Dynamic Data, when creating customizations we have to interrogate the meta-data about the model and code against that instead of directly against the known model. We already have the Type object in .NET to discover much of what we need about the model (data type, field names, etc.) and we can enrich this model with attributes for information like descriptive names, validation rules and so forth.

This makes typical “programming” tasks more challenging starting with simple tasks like presenting the data on the screen to querying the data source without actually knowing the model ahead of time.  LINQ queries change from a simple query syntax to expression trees.  Once you get in the right mindset … it’s kinda of fun.

So what’s “Meta-Meta-Programming”?

This is where the real fun starts.  In Dynamic Data, we are using Model-First Entity Framework to essentially drive the entire application.  This generates the model, which we mark up with the metadata so we can “meta-program” around it.  But what if we can also generate the metadata?

By default, the Entity Object code generator creates decent classes which match the model, play nice in OOP and who’s persistence is managed though the generated ObjectContext. But there are lots of fields on the Entity Model which are largely untouched. What if we could “program” the entire application through the Model and with maybe a little side of metadata on the attributes?

To begin to accomplish this, we ditched the default code generation and downloaded (Nuget rocks) the EF5 Entity Object T4 template.  We then began to generate our metadata through addition fields on the model (DefaultValue, Is Nullable, Documentation).  So the T4 template with these additional fields become our meta-meta-programming tasks, we meta-program the UI to react to the metadata, and we toss in some good old fashioned programming when we get sick of expression trees and reflections.

I may write about the T4 template we created but just contact me if you want a copy in the meantime.

Upgrading Windows 7 Ultimate to Windows 8 Enterprise

After having such a great time at Build I felt it was time to upgrade my work laptop to Windows 8.  Previously, however, I had Windows 7 Ultimate and when I tried to upgrade to Windows 8 Enterprise the installer told me I could not do so: “Windows 7 Ultimate cannot be upgraded to Windows 8 Enterprise.”  The natural path is to upgrade to Windows 8 Pro but I specifically wanted Enterprise since that was what we have at work and for the ability to side-load Windows Store apps (though maybe Pro can do this as well).

Anyway, just by luck I happened upon a post about a very similar problem going from Vista to Windows 7 Enterprise … same issue.  The fix?  Totally stupid … really.

  1. Close the Windows 8 installer if you have it open and
  2. Launch RegEdit to edit the registry
  3. Find the following branch: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current Version
  4. Edit the following keys:
    1. EditionId – Change from “Ultimate” to “Enterprise”
    2. ProductName – Change from “Windows 7 Ultimate” to “Windows 7 Enterprise”

Now the in-place upgrade will run just fine.

BAM Alerts and cross-database ownership chaining issue

BizTalk offers up many surprises to me even after about 8 years of constant experience with it.  Today’s lesson to me was a permission error we were getting in the BAMAlertsApplication stating that our SQL Notification Service user no longer had permissions to SELECT data from some of the tables.  I had never run into this before … ever … not in 8 years.

After much digging, we determined that starting the BAM notification server was calling a stored procedure in the BAMAlertsNSMain database: NSGetApplicationActivationStatus.  This stored procedure was, in turn, querying off tables in BAMAlertsApplication such as NSProcessActivationStates.  So why was this previously working but no longer?  We looked at permissions granted to this service account and it never had SELECT permissions on these tables, only EXEC rights to the stored proc.  From past experience, I knew there was some sort of “loop-hole” in which a stored procedure could return results from a table the user did not have permissions to.  This is, apparently, called ownership chaining and to make matters worse this was doing a cross-database join.  The setting allows this only if the table, stored procedure and databases are all owned by the same user.

After poking around awhile in the database and server options we found an option called: “Cross-database ownership chaining” which was turned off for the server as well as for these databases. Consulting another BizTalk setup that was working I noticed that this setting was turned on for these two databases but no others.  As luck would have it, the DBA had just detached and re-attached the database files to move them to a different location and a side-effect of this is that this one particular setting as noted here.  Seriously, this is the ONLY side-effect of re-attaching database files.  Crazy.

To fix this you can either grant the service account correct access to all the tables, etc., in the database or restore this database option.  Of course, it would have been better if Microsoft properly setup permissions on these databases in the first place.  Oh … and make sure you don’t change the database owner to an account that does not also own all the objects in these databases.

Fun.

Pittsburgh Tech Fest: iOS Best Practices slides & code

Pittsburgh Tech Fest was great this year! It was a perfect opportunity to learn about some different technologies and techniques.  I’d like to give a special thanks to Dave and Eric for doing an awesome job organizing the event and the speakers.

For those that are interested, below are the links to the talk I did on iOS Best Practices: Avoid the Bloat and feel free to comment or ask any questions on this post!

Code before refactoring: https://github.com/JAgostoni/iOS-Best-Practices/tree/master/UglyApp
Code after refactoring: https://github.com/JAgostoni/iOS-Best-Practices/tree/master/NotSoUglyApp
Code as presented at Pittsburgh Tech Fest: https://github.com/JAgostoni/iOS-Best-Practices/tree/master/PghTechFest

PowerPoint Slides: iOS Best Practices – Pittsburgh Tech Fest
PDF Slides: iOS Best Practices (PDF) – Pittsburgh Tech Fest

Thanks again to those that attended my talk!

Creating the Sports Schedules App for the iPhone

I finally released my first real iPhone application in the app store that I did not create specifically for a client. The app is called Sports Schedules (great name, eh?) and is available via the Apple App Store here: bit.ly/SportsAppStore.

First, a quick background

I really do enjoy watching any Pittsburgh sports and it is pretty easy to remember to watch a Steelers football game.  But when it came to college hoops or hockey I frequently forgot about the games and missed them. Being pretty busy with work as well as (the more full time) entertaining my 5 year old I really didn’t leave much mental capacity for remembering there was a game that night.

Well … I have this nice smartphone so I perhaps I’ll put it to work for me, right? So I launched the App Store app and searched for Sports Schedules. I was faced with the very standard problem of finding a useful app in a sea of apps that didn’t do what I wanted. To top I off, there wasn’t an app simply called Sports Schedules so I figured I would just go ahead and create it.

Other apps

There are certainly other apps out there which have schedules for sports in them. First, each sport and even many teams have apps specific to them, but I really didn’t want a bunch of one off apps. Next, there were apps like ESPN, etc. While they have schedules in them … I really wasn’t impressed with the way the information was organized. Lastly, there are apps which simply put events on your calendars and I wasn’t really pleased with that scenario either.

Design

The first thing I was concerned with was creating some a nice, simple design for the application.  I started with some wireframe mockups using Balsamiq (great tool) so that I could initially focus on the information flow within the application.  Since it isn’t (initially) a large app this didn’t take much time: I wanted a dashboard showing the week at-a-glance, and the ability to add and remove teams

Next, I wanted to spend some time skinning this wireframe as I wanted to focus on creating a visually appealing app.  This reinforced the fact that I am a TERRIBLE designer.  I spent hours creating what I thought was an acceptable design and showed it to  my wife.  Trying to be nice she said: “It looks like you did it your self … can’t you buy a design on the Internet or something?”  Thankfully … you can … and I did!

Data

Now that I had proven my lack of design skills, I decided it wasn’t worth putting too much time into the app until I was able to locate a source of the data for the schedules.  Initially, I had thought about just entering it myself or paying someone a few bucks to enter the data into a database manually and keep it updated.  The problem is, that does not scale and is in no way reliable.  Instead, I started the search for a source of the data as an online service.  After searching for well over a week and contacting many many sales people (still gettin’ those emails) I finally found a source for the data … data that is not free and not cheap, as it turns out.  Before fronting the cash, I decided I better actually create the app first.

Developing the app

Once committed to creating the app, the problem became finding time to actually sit down and develop it.  I set a self-imposed deadline of “May” to try to keep motivated. With a full time job and a family this was indeed the most challenging part.  However, I managed to push through it and get the code written so I was able to move on to the next phase, the services.

Services

Next I needed to decide how to get the data updated on the app.  I didn’t want to spend a lot of money on servers but also I wanted to be able to scale up easily.  Elastic cloud computing seemed to make sense but last I had checked cloud services were a little pricey.  Lucky for me, Amazon started their deal where new customers get a pretty good share of EC2 instances and EBS storage free for a year so my decision was pretty easily made.

Next, I needed to decide whether or not to use a Windows instance or a Linux instance.  In a previous life, I did a fair amount of work on Linux but in recent years I was far more productive in Windows so that was my original choice.  The thought was that if I wanted to expand with some web applications I would have a nice .NET and SQL instance setup.  However, I quickly realized that Windows instances in Amazon are not nearly as economical as Linux instances so I scrapped that idea and setup an Amazon AMI Linux instance.  Frankly, the fact that the Windows instance uses a minimum of 30 GB of EBS storage and the Linux instance only 6 GB solidified that plan.

For better scalability and performance I decided to generate static XML data files for the iPhone app to download from.  This would mean fewer resources used on the Linux instance and would enable me to leverage Amazon’s CDN if necessary to better distribute the content.  As a result, I actually have no real web services serving up the content.

Integration

With the plan for services underway, I now needed to figure out how to get the data from my provider into a format suitable for the app.  As an Integration Architect on many projects I learned a lot of patterns and techniques as well as best practices for this sort of problem.  The primary take away is to decouple the data source from the apps in the event I need to replace it or even aggregate the data from multiple sources.  Also, having worked extensively with BizTalk I saw great value in leveraging an integration platform but BizTalk is far too pricey for my problem.  I decided to investigate free/open source integration solutions.

There are several FOSS integration platforms out there: WSO2, Mule, JitterBit, etc.  JitterBit caught my attention for a few reasons: it was simple, it had just the features I needed, and the installation was relatively lightweight.  It also came with a decent development environment.  JitterBit allowed me to very quickly integrate the XML data source with my database (PostgreSQL) and then to output the static XML files.

More app development

Now that I had a steady stream of data coming in from the provider, I needed to finish the coding in the application.  Hooking up to the XML content was a pretty standard academic exercise as was adding in some small features and fixing bugs.

App Store

The part I dreaded most was submitting to the App Store.  One can spend a lot of time and a lot of money just to have Apple reject the app for some completely unforeseen reason.  In nearly all occasions, when I submit to the App Store I get rejected at least once.

I submitted the app fully expecting to get rejected … what I didn’t plan on was that I would be the one doing the rejecting.  In two occasions while waiting for approval I found bugs and needed to reject my own app.  After re-submitting it … twice … I was very pleasantly surprised when my app was approved after nearly EXACTLY one week.

After all the effort, my app was finally in the App Store and working well … maybe a few bugs that need squashed … but overall a pretty smooth launch.

TFS Xcode Build – v1.0 Released

Xcode, TFS and the ALM …

Many organizations have been faced with centralizing all of their ALM tools in order to enable better integration across all the tools for each role in your app lifecycle.  Team Foundation Server (TFS) provides an excelllent integration environment for Microsoft .NET projects and even application developed in Eclipse (Java, Android, etc.).  There have been many recent advances into the mobile space especially in iOS applications and my work is certainly no exclusion to this.  Since CEI has adopted TFS as our ALM platform I have been keeping all of my Xcode projects in TFS via the Subversion bridge.

But what about builds?

While storing Xcode projects in TFS works quite well (including the ability to associate with Work Items) one of the primary features of TFS (and any integrated ALM platform) is Build Automation.  Since Xcode projects can ONLY be built on Mac OS X there simply was no way to trigger a build using Team Build in TFS.  Alternatives exists, for sure … there are other CI platforms that can be triggered via SVN (svnbridge in TFS) but that requires more investment in software.

What I wanted was a solution leveraging Team Build as much as possible and a Mac only where it was needed to compile the Xcode project.  The thought of implemeting a Team Build Agent on the Mac was …. frightening 😉  So, instead, I decided to automate copying the source code from the Team Build server to the Mac (using SCP), remotely triggering xcodebuild (via SSH), and finally retrieving the results (again, via SCP).  It turns out this was pretty straight-forward and reliable.  To share this, I created a Codeplex project to host the source code and binaries.

TFS Xcode Build v1.0

Check out the project hosted on Codeplex here: http://tfsxcodebuild.codeplex.com/.  There you can find the latest source code, binary release and documentation.

iOS Best Practices – Singletons

Problem

Many examples found online utilize the AppDelegate instance for global storage/variables.  While this is a quick way of sharing data and methods between views and classes it can (and usually does) lead to several problems:

No control over global variables/storage

Each referencing class assumes direct control over this variable and won’t necessarily respect how another class is expecting to use it.  With a singleton, the data has been fully encapsulated and controlled in one place.

Repeated business logc

If there is any business logic on how this global storage is to be used it has to be repeated throughout the application.  While some may “encapsulate” this by using accessor methods the logic is in the wrong place.

Big Ball Of Mud

Very quickly, the AppDelegate class will become a big ball of mud and VERY difficult to maintain.  This is compounded over time as the app is revisioned and different developers add more and more code to the ball of mud.

Fixing the problem: Singleton

One way of fixing the “I need to put all my global variables in the AppDelegate” is to use Singletons.  A singleton is a design pattern (and implementation) ensuring that a given class exists with one and only one instance.  The developer can now store like variables and implementations together with the confidence that the same data will be retained throughout the application.  In fact, the AppDelegate is held in a singleton of your application ([UIApplication sharedApplication]).

The developer must also ensure to not repeat the same “big ball of mud” anti-pattern by simply moving all the code from the AppDelegate into one Singleton class.  The concept of single-purpose classes will be covered in a future post.

Implementation

The implementation is pretty straight-forward based on Apple’s Fundamentals and is made even simpler using ARC in iOS 5.  The trick is ensuring all code that references this class is using the exact same instance.

Steps/tips for a Singleton in Objective-C:

1. Implement a “shared manager” static method to dynamically create and retrieve the same instance each time.

static SingletonSample *sharedObject;
+ (SingletonSample*)sharedInstance
{
if (sharedObject == nil) {
sharedObject = [[super allocWithZone:NULL] init];
}
return sharedObject;
}

2. Leverage public shared methods as a convenience factor to encourage use of the singleton.

+(NSString *) getSomeData {
    // Ensure we are using the shared instance
    SingletonSample *shared = [SingletonSample sharedInstance];
    return shared.someData;
}

3. Create and use instance variables and methods as you normally would

@interface SingletonSample : NSObject {
    // Instance variables:
    //   - Declare as usual.  The alloc/sharedIntance.
    NSString *someData;
}

// Properties as usual
@property (nonatomic, retain) NSString *someData;

4. Use the class via the shared methods and/or instance

- (IBAction)singletonTouched:(id)sender {
    // Using the convenience method simplifies the code even more
    self.singletonLabel.text = [SingletonSample getSomeData];
}

The full source code with sample application is available here: https://github.com/JAgostoni/iOS-Best-Practices/tree/master/BigBallOfMud