Release 1.0.1

+ New feature
* Bug Fix
- Removed feature

1.0.1 - 30-Aug-2003
+ Read flows from xml config file specified in app.config
+ Asynchronous File Inputer
+ File Outputer
+ XSLT Processor
+ Generic Blocks (either NOPs or straight conversion)
  + Parser
  + Processor
  + Binary Processor
  + Renderer
  + Output
+ log4net
+ NUnit test cases for
  + BinaryProcess
  + Flow
  + Parse
  + Process

You are here

Now that I've ranted about what I don't want HookUp to be, maybe I should elaborate on what I do want it to be and how I intend to get it there.

Language.

I've chosed .NET, specifically C#. I come from a C++ background mostly under windows, so this seemed to be a good next step (and a good resume builder.) I have used java quite a bit in the past, but mostly just the core language stuff. In fact I've written entire client/server communication frameworks in java but I felt that there's entirely too much to learn about java technologies to build a truly useful system. I'm so far behind the curve on J2EE -- I didn't even get through the tutorial.

So, C# it is. That means a windows based system. I'm not too concerned about this for a couple of reasons:
- It's possible that .NET will be ported to Unix by the time this is ready. Shared Source CLI (a.k.a Rotor) is already targeting FreeBSD and Mac OS.
- Microsoft is a juggernaut. It's probably a safe bet that most companies have at least 1 windows box in them.

Database

I intend on being database agnostic: OleDb all the way. That should open up just about everything as long as I can deal with the different stored procedure versions. First off I will target SQL server (being the easiest to get my hands on,) followed by Oracle and maybe MySQL.

To help me with the data access portion there are a couple of nice tools I'm looking at:
- Microsofts Data Access Application Block seems to be a useful bit of code. You can pretty much search and replace Sql with OleDb and you're set (you lose a few Xml related methods specific to SqlServer but that doesn't seem to be a stopping point.)
- Microsoft ObjectSpaces is an intriguing ORM tool.

Protocols

Internally I'll use Xml all the way. I'll need to provide support for other formats for input and output like raw binary, flat file (delimeted and positional). There are also a bunch of industry specific protocols depending on where the system gets used. There's HL7, HIPAA, X12 for healthcare. FIX, FIXML and STAMP for Fincancial etc, etc.

I'll also provide support (both incoming and outgoing) for web services. That seems to be a fairly hot ticket nowadays, even in non-Microsoft camps. That provides a nice integration point with java systems for instance.

User experience

I'm targeting developers. The first few/several version will have no gui associated with it. Configuration is through Xml and tracking will be through log files.

In future I plan to provide a graphical representation of a given configuration. I played with Lego enough as a kid to be able to visualize a "building-blocks" approach when I see it. Who knows, maybe this GUI will be extended to configure the system and even trace an execution path like a debugger.

Tools

There are a number of potentially interesting tools like the Data Access Application Block at Microsoft Patterns and Practice.
I'm a big fan of NUnit
NDoc looks interesting for documentation. I'll have to check it out further.
I don't really like the built-in logging capabilities of .NET so I've been looking at log4net.

Design goals / Plan of attack

Simple simple simple. The last thing I want to do is create another heavyweight app. I want it simple to extend the basic blocks I will provide. However, I don't want to back the user into doing a bunch of coding just to get things off the ground.

Inputs: .NET Assembly API, Files, MSMQ, Web Service API, Database table
Outputs: Files, MSMQ, Database, e-mail
Configuration: Xml config files a la app.config.
Transformation: Xslt all the way baby.
Database: OleDb compatible but can be used completely without a database.
Pre/Post processing: Encryption, Digital Signatures, Certificates

I plan on having both synchronous and asynchronous execution paths depending on how a given path has been configured (e.g. even a call to an asynchronous method will end up being synchronous if you don't configure some checkpoint.)

I want to make sure the system is very trackable. If something goes wrong you have to be able to find out where.

Since I'm going for an enterprise level system it has to perform, scale and recover.

What's looming

The first releasable package is going to have the following features:
- Inputs: .NET Assembly API, Files, MSMQ
- Outputs: Files, MSMQ
- Pre/Post Processing: Encryption (Crypto API stuff)
- Database: OleDb
- Synchronous and Asynchronous processing
- Proper exception handling and checkpointing
- Not going insane.

Will keep blogging...

A little history

I'm currently working in an application integration department of a medical software company. We're using Microsoft BizTalk to hook up our applications with other apps and external vendors. The "usual" vendor uses HL7 as a protocol for which we use the NeoTool HL7 accelerator.

Our software is all windows based which does simplify the matters a bit. The app I'm involved in most is a standard asp based web app. The asp pages call into my teams framework which takes an Xml document, beefs it up with some extra information from a database and passes it on to BizTalk.

This is all well and good, but I have always found this to be a bit heavy handed. In truth it has been rare that we've actually found some functionality in BizTalk that has justified is large price tag (and learning curve.) I have felt from the beginning that BizTalk is a solution looking for a problem.

Here's my biggest beefs:
- the price. I know that it's not out of my pocket, but I like to think I'm corporate minded.
- the learning curve. You just can't jump into the environment without picking up a fair amount on the various tools.
- the shear number of tools you need to use to get a solution going:
- NeoTool DocSpec generator
- BizTalk Editor
- Mapper
- Messaging Manager
- System Administrator/Receive functions
- the lack of integration with SourceSafe
- the clunkiness of the orchestration module
- the inability to have a port deliver back to a channel (or least I can find a way.)
- the brutal Messaging Manager interface
- It looks like a web based app, but doesn't behave like any I've ever seen. There's no way the Microsoft GUI evangelists have seen that thing.
- If you have to change something on the last property page you have to click next on each page
- You create a docspec/port/envelope/organization from the docspec/port/envelope/organization screen, but you create a channel from the port screen. This is inconsistent plus you can't see the other channels when you are creating a new one. If you have a non-trivial naming convention (which we do) you have to remember to copy an existing channel name aside or run a second instance.
- the mapper
- the inability to re-use code
- can't copy/paste
- the hoops you have to jump through just to do something that would be trivial in a normal XSLT file. (Just try to get 2 of a particular output nodes.)
- the focus on the GUI way of building maps. There's no way a B.A. would ever be able to use this tool. Have you ever made a non-trivial map without some custom functoids or some script?
- just let me stuff in some xslt. I had to write an AIC to do so.
- If can't see an error from the suspended items queue. You only get "see the event viewer for error." So you have to hop over to the event viewer and try to find the entry by the timestamp.
- Document tracking is awful. You have to click so many controls on that page just to see some trivial information.
- There's no easy way to script a configuration. The only easy way they give you is to export an existing configuration. How likely is it that you'll have a duplicate of production somewhere? At some point to have to slog through the implementation using the messaging manager and server administrator.
- You can't parameterize an orchestration except from the document itself.

As you can tell I have my opinions on this tool.

As I thought about the problem that BizTalk solves I began to believe (in my own misdirected sort of way) that it doesn't need to be that complicated.

That's where HookUp came from. That's my working title for a application integration system that focuses on the needs of the developer that will be putting these integrations together:
- No fancy GUIs to edit transformations. Odds are that the developer using this system will have their own favourite: XML Spy, Visual Studio, Notepad or like me, CodeWright.
- Focus on the building block approach. Define some blocks and then string them together abitrarily.
- .Net. Admittedly this is more for my benefit (and my resumes) but if it works well for me then it should be helpful for others too.
- Extensibility. You don't like my implementation of an XSLT transformation or a database checkpoint then write your own.

Lofty? Yes. Doable? I think so. We'll know more soon.

Next time: What hookup is now. What it can do. The tools and technologies.

My first Blog

Typical first Blog [read: no content]