BizTalk ESB Toolkit: Core Components and Examples Published: February 2010 Author: Jon Flanders, MCW Technologies Technical Reviewers: Ofer Ashkenazi, Microsoft Vinay Balasubramaniam, Microsoft Jim Bowyer, Microsoft Peter Kelcey, Microsoft Brian Loesgen, Microsoft Dmitri Ossipov, Microsoft Robert Hogg, Black Marble Applies to: Microsoft® BizTalk® ESB Toolkit 2.0 and Microsoft BizTalk Server 2009 Summary: The BizTalk ESB Toolkit 2.0 is a rich addition to BizTalk Server. Its powerful capabilities on top of BizTalk Server provide technical and business value by laying the groundwork for building the next generation of loosely-coupled mediated services. The BizTalk ESB Toolkit 2.0 provides a stable and powerful platform for services that can change as fast as your business needs them to. The main purpose of an enterprise service bus (ESB) to is to provide a common mediation layer (the “bus”) through which all services connect. By doing so, not only can many of the problems of point-to-point service connectivity be resolved, but a new level of agile service delivery can be achieved. For more information about the Microsoft ESB Toolkit 2.0, see the ESB Toolkit pages on the Microsoft BizTalk Server 1
Transcript
BizTalk ESB Toolkit: Core Components and Examples
Published: February 2010
Author: Jon Flanders, MCW Technologies
Technical Reviewers:
Ofer Ashkenazi, Microsoft Vinay Balasubramaniam, Microsoft Jim Bowyer, Microsoft Peter Kelcey, Microsoft Brian Loesgen, Microsoft Dmitri Ossipov, Microsoft Robert Hogg, Black Marble
Applies to: Microsoft® BizTalk® ESB Toolkit 2.0 and Microsoft BizTalk Server 2009
Summary: The BizTalk ESB Toolkit 2.0 is a rich addition to BizTalk Server. Its powerful capabilities on top of BizTalk Server provide technical and business value by laying the groundwork for building the next generation of loosely-coupled mediated services.
The BizTalk ESB Toolkit 2.0 provides a stable and powerful platform for services that can change as fast as your business needs them to. The main purpose of an enterprise service bus (ESB) to is to provide a common mediation layer (the “bus”) through which all services connect. By doing so, not only can many of the problems of point-to-point service connectivity be resolved, but a new level of agile service delivery can be achieved. For more information about the Microsoft ESB Toolkit 2.0, see the ESB Toolkit pages on the Microsoft BizTalk Server 2009 Web site (http://go.microsoft.com/fwlink/?LinkId=154169) and BizTalk Developer Center (http://go.microsoft.com/fwlink/?LinkId=154069).
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.
This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in, or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property.
Unless otherwise noted, the example companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred.
Microsoft, BizTalk, SQL Server, and Visual Studio are trademarks of the Microsoft group of companies.
All other trademarks are property of their respective owners.
2
Contents
Why an Enterprise Service Bus?..................................................................................................................4
The ESB Pattern.......................................................................................................................................6
Overview of the BizTalk ESB Toolkit............................................................................................................7
ESB as a Service.........................................................................................................................................25
Why an Enterprise Service Bus?The IT landscape in many enterprises has evolved to include many distinct and different line-of-business (LOB) systems and heterogeneous technologies. Companies often buy software for non-business critical l activities (such as finance or human resources), and build systems essential to the company’s day to day activities. It is common that each system is implemented independent of each another and over time, these disconnected silos of functionality and data need to be integrated to provide new levels of business functionality. This is why, according to some analysts, over 70 percent of the code in corporate software systems is in integration solutions that are dedicated to moving data from system to system.
Generally the solution to these integration problems has been to build point-to-point (directly connected) integration between the different systems. Some of these integrations can be built using off-the-shelf solutions, whilst others need to be built using custom coding (or a combination of off-the-shelf and custom code). Although these solutions are generally successful, over time, this ad-hoc point-to-point integration leads to significant levels of complexity as each system can have large numbers of connections and increased cost of managing and maintaining application integration. Changes in either end of the point-to-point solutions will often cause the integrations to break. Upgrades of off-the-shelf solutions often require the point-to-point integration solutions to be upgraded as well. Over time, the ad-hoc point-to-point integration strategy breaks down as the business tries to change to respond to market conditions, but the growing number of interconnections between systems makes the changes impractical. As an organisation matures and the number of systems become increasingly important, the need for structured monitoring and problem handing becomes more important. This lack of agility and monitoring has led vendors and developers to look for a different solution to the point-to-point integration pattern.
One technique often used to solve this problem is Enterprise Application Integration (EAI). EAI allows a business to use an off-the-shelf middleware product to centralize and organize these integrations. Although EAI solutions provide advantages over totally ad-hoc solutions, in the end, the implementations typically lead to the same types of problems. EAI solutions tend to allow endpoints to be hardcoded. Often any change in a service or backend system will cascade, causing part of or even the whole EAI solution to require re-implementation.
Another attempt at solving this problem was the move toward a service-based architecture. There were several driving forces behind the software industry moving from component-based architectures to service-based architectures. One of the primary forces was a move away from single-vendor applications toward applications that could be built on different platforms using different languages and run times. This allowed individual applications to be developed and deployed much more quickly and treated as black-box pieces of functionality (such as services) by other applications. Multiple services could then be combined to build higher-level functionality.
4
There was another motivation to expose current, legacy, and LOB systems to newer applications, which has driven the software in the integration space to also become service-oriented. This allows LOB systems to be treated like any other service, to be composed in new ways into current applications.
In most cases, service-oriented applications also end up delivered as a set of point-to-point solutions. Over time they become just as tightly coupled as a single monolithic application can. If an address changes, or if the message format for one of the endpoints changes, this dictates that the endpoint be modified to match the changes in the service or system. And in such cases the system loses most if not all of the benefits of being service based and in many cases is worse off.
As you can see in Figure 1, a set of autonomous services can quickly devolve into a spaghetti-code-like structure that becomes difficult to maintain. The point-to-point design is hard to scale since each node is connected directly to other nodes. It sometimes leads to redundant functionality when services try to avoid additional outbound calls. It also becomes difficult to get visibility into what the system (or even part of the system) is doing, since everything is directly connected, and importantly there isn’t an easy way to provide monitoring.
Perhaps the most serious limitation of uncontrolled service architecture is that it becomes almost impossible to change without causing devastating side effects. A change to any of the services can have a negative impact on many or all of the other services or applications that rely on that service’s functionality.
Figure 1: Point-to-point spaghetti
5
ESBs are a reaction to some of the problems discovered when using service-oriented applications. The term ESB is a somewhat overloaded term at this point. However, the primary driving force behind the ESB pattern (which has now been embedded in various products and tools by many different vendors) is to create a framework for building service-oriented applications that solve the problems discovered in earlier attempts at integration. This is accomplished by moving the logic away from each individual set of endpoints into a logically centralized layer, with in general one connection per service. Although EAI is based on a centralized layer, what differentiates the ESB pattern from EAI is the emphasis on dynamic execution.
Extensibility is important because as the needs of the business change, software needs to change with (or even before) those needs. The concepts behind the ESB pattern are intended to easily support changes driven by business needs by providing a centralized, loosely-coupled, dynamic layer for managing integrations.
If software is easier to maintain, the business gains value from both lower operating costs and lack of downtime due to fixes.
The ESB PatternThe main purpose of an ESB to is to provide a mediation layer that all services connect through. If all services connect through this layer (the “bus”) many of the problems of point-to-point service connectivity can be solved.
Figure 2: Agility through service composition
6
The ESB can also provide common re-usable services like data transformation, protocol conversion, dynamic routing, and error handling, as well as visibility into what is happening to a particular application or interaction.
Because applications use the ESB in a loosely-coupled way, changing what happens when an application uses a service exposed through the ESB becomes much easier, and leads to greater business agility. If a service changes, the application need not even be aware of that change because the ESB layer can modify that configuration without changing the application. If a message needs to be transformed into another type of message (schema), an additional transformation can be added by the ESB, again without having to change either the service or application.
Overview of the BizTalk ESB ToolkitThe BizTalk ESB Toolkit provides a set of services enhancing the existing BizTalk architecture to enable service consumers and service providers to be part of a loosely-coupled but mediated environment.
Figure 3: Basic architecture of the BizTalk ESB Toolkit 2.0
The framework provided by the BizTalk ESB Toolkit is driven by metadata in the form of an itinerary. An itinerary is an XML document that describes the set of mediated steps that the BizTalk ESB Toolkit should use to process a message. An itinerary might be attached to a message as it arrives in the bus. However, it’s much more likely it will be dynamically determined based on contents of the message itself. For example we could use the customer id inside a message to decide if the message should use a premium or standard itinerary.
7
Once the BizTalk ESB Toolkit determines the itinerary, the set of mediated steps can be executed. These steps can include message transformation, endpoint resolution, or orchestration. Another key feature of the BizTalk ESB Toolkit is that all the mediation services (as well as most other parts of the toolkit) are extensible to provide maximum agility. Rather than going outside of the ESB to get new functionality, that new functionality can be integrated into the ESB, without interrupting existing consumers and providers.
Messages get onto the bus by using a concept known as an OnRamp, which in the BizTalk ESB Toolkit is implemented by a BizTalk Receive Port. Messages leave the bus through OffRamps, which are bound to BizTalk Send Ports.
The BizTalk ESB Toolkit can provide services to messages as they flow through the bus. Most of these services are also exposed outside the bus, so they can be used by other consumers. Services include message transformation, endpoint resolution, and orchestration or pipeline processing.
In a low-level technical manner, the way that the BizTalk ESB Toolkit uses existing BizTalk Server features is a codification of many BizTalk best practices. In a BizTalk application, Receive Ports receive messages and route messages to either Send Ports or Orchestrations. A typical BizTalk application might have these elements bound together statically during design or deployment, making agile modification more difficult.
Figure 4: ESB Toolkit and BizTalk components
The BizTalk ESB Toolkit binds these elements together dynamically using static or dynamic metadata. In the case of messaging-only applications, the ESB Toolkit uses dynamic Send Ports. In the case of processing (orchestration) it uses direct-bound Ports. The BizTalk ESB Toolkit in some ways is a layer of abstraction on top of these very powerful (and often underutilized) BizTalk features. The resulting architecture makes it a valuable way to build BizTalk applications, but also makes BizTalk Server a general-purpose mediation layer for any set of services.
8
BizTalk ESB Toolkit Core Components
ItinerariesAt the heart of the BizTalk ESB Toolkit is the notion of an itinerary. An itinerary is a piece of metadata that describes the path through the bus that a message should use. It can also include a set of services that should be applied to a message as it flows through the bus. It can be thought of as a domain-specific language (DSL) for a lightweight service composition model. The model is then used to determine the flow of messages through the BizTalk ESB Toolkit run time.
The BizTalk ESB Toolkit includes a DSL designer to enable developers to easily build and visualize how service consumers and service providers can be associated.
Figure 5: Simple itinerary in the BizTalk ESB Toolkit Designer
Every message needs an attached itinerary. Messages can come into the bus with an itinerary already attached. (The BizTalk ESB Toolkit supports an itinerary in a SOAP header.) Because hard-coding the itinerary isn’t really a best practice, a much more common scenario is that the itinerary isn’t attached. This allows an ESB component known as a “resolver” to determine the correct itinerary for the message. (The list of out-of-box resolvers will be covered later in this paper.)
Once a message is inside the bus with an itinerary, the BizTalk ESB Toolkit components will use the itinerary to guide the message through its intended pathway. This could include, but isn’t limited to, determining the correct OffRamp, applying services (such as transformation), or routing the message for further processing (through an orchestration).
9
The mechanism that implements itinerary processing is made up of [BizTalk] pipeline components.
These parse resolution instructions from connection strings or from the ESB Itinerary SOAP headers and
they provide endpoint resolution or transformation execution capabilities using the Resolver and
Adapter Provider Frameworks. The itinerary pipeline components can dynamically set the BizTalk Server
endpoint properties or dynamically execute transformation based on resolution instructions in the
itinerary. These components are responsible for managing, updating, and persisting the itinerary across
process and service boundaries.
Since the BizTalk ESB Toolkit is an abstraction layer on top of the existing BizTalk infrastructure, you can think of itinerary routing as an abstraction on top of the BizTalk dynamic Send Port capability. This abstraction simplifies using dynamic ports, and reduces the chance of common dynamic routing errors.
On/Off RampsOnRamps and OffRamps are the endpoints in a BizTalk ESB Toolkit application.
OnRamps are endpoints that bring messages into the bus. An OnRamp is really a named abstraction on top of a BizTalk Receive Port, so each OnRamp is a BizTalk Receive Port configured with BizTalk ESB Toolkit pipeline components. Although messages typically arrive through an OnRamp, it should be noted that since the BizTalk ESB Toolkit works on top of the existing BizTalk infrastructure, there are other ways to get a message into the system. It’s possible for an orchestration or a Receive Port that isn’t configured with the ESB pipeline components (that may or may not be part of an itinerary) to publish a message to the MessageBox with the correct promoted properties to get a message into the bus.
OffRamps are endpoints that send messages out of the bus. OffRamps are built on top of [BizTalk] Send Ports, the outbound endpoint mechanism of BizTalk; each OffRamp is mapped to a BizTalk dynamic Send Port. The fact that OffRamps are dynamic means that a message in the bus can be routed to many different physical endpoints (using different transports and security), without having to change your application.
These abstractions provide a unified view on top of the underlying BizTalk architecture. However, more than that, they provide a conceptual abstraction that enables BizTalk and non-BizTalk developers to think about their services in a more granular manner than through the actual physical architecture.
Adapter Provider FrameworkAdapter Providers are used in the BizTalk ESB Toolkit to set properties on outbound adapters. They provide a mapping between BizTalk ESB Toolkit configuration properties and Send Adapter context properties. The BizTalk ESB Toolkit adapter framework performs this mapping based on the Adapter Provider design-time metadata. The metadata is used to map itinerary configuration (which can be static or dynamically generated) to BizTalk ContextProperty values for a specific Send Adapter.
10
There are a number of Adapter Providers included with the BizTalk ESB Toolkit such as:
The Adapter Provider framework provides a very easy way to extend it to include more BizTalk send adapters that are capable of dynamic configuration. These adapters are defined through the implementation of the IAdapterProvider interface within a .NET Framework assembly. Adapter providers are registered by their BizTalk Server transport type, which set the endpoint information provided by a resolver on the appropriate BizTalk Server adapter.
ESB Resolver FrameworkThe Resolver Framework provides a comprehensive, pluggable architecture for dynamically resolving service metadata. It can be used to provide at runtime the endpoint information of an external service or the parameters and artifacts needed by an ESB service (internal service) such as route, transform or any custom service you add to the ESB Toolkit. For example you can resolve Microsoft BizTalk Server map types for the transform service. The resolver framework lets developers create custom resolvers with specific behavior to support alternative resolution and routing methods.
The Resolver Framework provides out-of-the-box support for various technologies such as: Universal Description, Discovery, and Integration (UDDI) Business Rule Engine (BRE), XML Path Language (XPath)
The Resolver Framework also exposes developer interfaces (IResolveProvider) to allow creation of custom resolvers. These are loaded and cached at run time and support a range of resolution types and connection strings.
Resolving Itineraries One of the main aspects of the BizTalk ESB Toolkit is dynamic itinerary resolution and execution.
A message that is processed by an OnRamp can have an itinerary embedded in the message itself (via a SOAP Header), or the itinerary can be resolved in the OnRamp.
Itinerary resolution can happen via the resolvers shown in Figure 6.
11
Figure 6: Itinerary resolution
In all cases the Resolver then uses the name and retrieves the itinerary from the BizTalk ESB Toolkit database (except of course when the itinerary is attached to the message).
Exception Management FrameworkThe ESB Exception Management Framework provides a unified message-oriented fault generation mechanism for managing all exceptions that may occur within a Microsoft BizTalk Server environment. The ESB Exception Management Framework can receive exception messages published through the Exception Publishing Service, in addition to messages from the BizTalk Server Failed Message Routing mechanism.
The framework provides an API for fault message creation, publication, and management from orchestration processes, that utilizes the BizTalk Server failed message routing feature. In addition, the framework provides facilities for normalizing all exceptions, enriching them, applying Business Activity Monitoring (BAM) tracking, and publishing the final output to the Exception Management database for display and reporting in the ESB Management Portal.
The BizTalk ESB Toolkit includes the pipeline components that support the ESB Exception Management Framework:
Exception Fault processor pipeline component.
BAM Tracking pipeline component.
Routing Examples In most cases, one of the main responsibilities of the BizTalk ESB Toolkit is to route messages associated with an itinerary to the correct service provider (either to an additional processing step or to an OffRamp). Although there may be other services applied along the way (for example, transformation), getting a message from an OnRamp to the correctly configured OffRamp is typically an essential part of processing an itinerary.
In BizTalk terms, this means that the information in the itinerary needs to be translated into BizTalk Message Context properties, which are the basis for subscriptions in the BizTalk pub/sub system. To see how this works, here is a simple example.
12
An UDDI query will retrieve the name of the itineraryUDDI 2.0/3.0
An XPATH statement will be executed against the message to retrieve the itinerary nameXPATH
The itinerary name is set by the resolver connection stringStatic
A BRE Policy will be executed to retrieve the itinerary nameBusiness Rules Engine (BRE)
Example One: Hello BizTalk ESB ToolkitFor this example we will use a common simple BizTalk example: message routing to dynamic URIs. Imagine we receive messages into our service bus, and want to dynamically route them to different URIs based on some condition. (This could be content-based routing if we were looking at the content of the message, or it could be based on some other heuristic.)
The typical BizTalk way to accomplish this would be to create a Receive Port and statically bind it to an Orchestration Receive Shape. Inside of the Orchestration, we use an Expression shape to modify the context of the message to specify the TransportType and the TransportLocation (that is, URI) properties. (This could also be done using a custom pipeline component; either way we’ve got to write and deploy some code.) Then a Send shape in the Orchestration would be bound to a Dynamic Send Port.
To accomplish this with the BizTalk ESB Toolkit, first create an itinerary. (In fact, generally with the BizTalk ESB Toolkit this is always the first step regardless of the problem being solved.) The itinerary in Figure 5 can be used to accomplish this pattern. There are a few ways to implement this pattern with the BizTalk ESB Toolkit, but the itinerary in Figure 5 is a straightforward method.
Here are the three artifacts in this itinerary:
An OnRamp An Itinerary Service An OffRamp
Once a message has been received by an OnRamp and it is determined that this particular itinerary (named SimpleItinerary in this example) should be applied to the message, the BizTalk ESB Toolkit Dispatcher (which is a pipeline component) takes over and starts to process the message.
As mentioned earlier, an itinerary can either be attached to a message, or the OnRamp itself can determine what itinerary to apply. In either case, each OnRamp has to be associated with a particular ReceivePort, in this case a ReceivePort named “ESBON”.
In this example, we are using a special BizTalk ESB Toolkit pipeline in the Receive Location. There are a number of custom pipelines included with the BizTalk ESB Toolkit to help implement different patterns. In this case we use the ItinerarySelectReceivePassthrough, which by its name indicates it will select the correct itinerary, and then simply pass the message to the MessageBox.
The ItinerarySelectReceivePassthrough pipeline is configured with a BizTalk ESB Toolkit–provided pipeline component called the ESB Itinerary Selector. As with pipelines, the BizTalk ESB Toolkit comes with a variety of pipeline components that implement the various Itinerary Services.
If a message doesn’t have an itinerary attached, the BizTalk ESB Toolkit can use a component known as a Resolver to determine the itinerary. The Resolver is a component (a class that implements the IResolverProvider interface) that can be used to dynamically resolve information that the BizTalk ESB Toolkit needs at runtime. In this case, the Resolver is going to determine the itinerary. In other cases, a Resolver might determine endpoint information, which orchestration to use, or what map to apply to a
13
particular message. Resolvers are key components in the BizTalk ESB Toolkit's role in making a BizTalk application more dynamic
The ItinerarySelectReceivePassthrough pipeline has a component named the ESB Itinerary Selector in the first (decode) stage of pipeline processing. The main role of the ESB Itinerary Selector is to configure an Itinerary Resolver. Resolvers in the BizTalk ESB Toolkit are registered through global configuration through monikers (similar to network schemes like http, ftp etc.). In Figure 7, the configured Resolver has a moniker of “ITINERARY:\\name=SimpleItinerary”, which means that the Resolver (in this case the ITINERARY Resolver) will look up the value of the SimpleItinerary from the registered Itinerary store (which is a configured SQL Server database).
Figure 7: ESB Itinerary Selector configuration
This also means that we should have already imported the SimpleItinerary model into the store (which can be done with Visual Studio from the Itinerary designer or through a command line tool). Although hard-coding the OnRamp with this particular itinerary might seem rather inflexible because the itinerary lives in the database we can change it at any time and the BizTalk ESB Toolkit run time will respond. Of course, there are resolvers that can resolve the itinerary dynamically, like the Business Rules Engine (BRE) resolver, or the UDDI3 resolver (which communicates with the UDDI service). However, in this case we have chosen to pre-configure the itinerary.
Note It may seem odd that you have to associate the itinerary with a ReceivePort and then also associate a pipeline with each ReceiveLocation in that ReceivePort with BizTalk ESB Toolkit pipeline components. From a BizTalk architectural point-of-view, this makes sense because the only way to add
14
custom functionality to a ReceivePort is through an adapter or a pipeline component. Rewriting all adapters to be ESB-aware would be difficult at best, so using a pipeline component makes sense.
In summary so far we have built an OnRamp to the BizTalk ESB Toolkit that has a single physical address (for example, ReceiveLocation) where every message that arrives will be associated with the SimpleItinerary itinerary by the ESB-provided pipeline and pipeline component (due to configuration of them on this ReceiveLocation). After being processed by the ReceivePort, the message will go to the MessageBox. Therefore, the next piece we need is a dynamic SendPort that can subscribe to the messages that will be published by this ReceivePort.
In BizTalk ESB Toolkit vocabulary, we need an OffRamp associated with the OnRamp. Associating the OffRamp in the Itinerary Designer with the OnRamp accomplishes this. Like the OnRamp, each OffRamp has to be associated with a physical SendPort. The SendPort will need a filter (for example, subscription) to messages published by the ReceivePort. See Figure 8 for the filter we need for this particular itinerary.
Figure 8: SendPort OffRamp filter
The reason we skipped over the Itinerary Service in the SimpleItinerary design was to better explain how it fits in. The filter on the SendPort has already taken care of associating the OnRamp with the OffRamp, so in this case the Itinerary Service isn’t there to promote those properties into the MessageContext. Instead it is using another Resolver, the BRE Resolver.
15
The BRE Resolver is not being used here to resolve the itinerary (since that’s already been set by the time it executes, after the OnRamp itself), but to resolve the necessary properties to actually send the message somewhere through the OffRamp. It is serving as the resolver of the URI for the endpoint, which in BizTalk is partitioned into two different MessageContext properties: TransportType and TransportLocation.
If you look at Figure 9 (and ignore the conditions that are not a real-world example in this case, unless you only wanted one result), you can see the policy (supplemented with BizTalk ESB Toolkit-specific vocabularies) that resolves these two properties.
Figure 9: BRE policy to resolve TransportType and TransportLocation
Again, the BizTalk ESB Toolkit still relies on the underlying BizTalk infrastructure. In this case the BRE is being used, and can be used in a BizTalk ESB Toolkit application to provide another dynamic (agile) layer. This policy will set cause the SendPort to configure itself using the FILE Adapter (because the two actions in the Rule will cause MessageContext properties to be set), and set the File Location to C:\Labs\InvoiceDrop\%MessageID%.xml. When a new version of a policy is deployed, the Itinerary Service that uses the BRE Resolver will automatically pick up the new version, providing an easy way to change the flow of the itinerary without having to recompile or deploy any new BizTalk artifacts.
16
This is another way in which resolvers are used in the BizTalk ESB Toolkit to ensure that, although itineraries can be used to stitch services together, the stitching stays flexible, able to be changed based on business needs. Figure 10 shows the SimpleItinerary again, this time with text explaining the steps.
Figure 10: Simple itinerary with explanations
If you look at the application from a BizTalk point-of-view, building an orchestration could accomplish the same functionality, but using the BizTalk ESB Toolkit has ensured that:
a) We have built an extensible application which is loosely-coupled (more loosely-coupled than a typical BizTalk application).
b) The techniques used are standardized as part of the BizTalk ESB Toolkit, and therefore are more manageable, as well as more likely to be understood by other members of the team.
c) The itinerary can be changed (such as new services can be inserted into the flow) without code changes or re-deployments.
If you look at this application from a non-BizTalk point-of-view, what you can see is that the BizTalk ESB Toolkit provides a standardized way to build declarative services that can be easily modified due to the mediated layers provided by the toolkit.
In response to the question “Isn’t the OnRamp just point-to-point connected to the OffRamp, isn’t that just point-to-point integration?”, the answer is no – because multiple Receive Locations can be added, new service consumers can consume this service in different ways (e.g. using different endpoints) without affecting older consumers. Because the BRE Resolver is used, the OffRamp can be changed
17
without deploying anything except a new BRE Policy version (which can be done globally without changing the itinerary or BizTalk configuration). And of course the itinerary itself can be changed, without having to change the BizTalk configuration. This means the itinerary could be extended to allow multiple OffRamps, to add message transformation, or even to place an Orchestration into the mix if necessary.
All of these capabilities make building a BizTalk ESB Toolkit application (even a simple one like the one presented here) guaranteed to be a more flexible integration than using point-to-point.
Example Two: Adding additional ESB Toolkit functionalityOne of the powerful capabilities of the BizTalk ESB Toolkit is being able to change an application without affecting the endpoints. (Not having to change client endpoints is especially important when trying to keep integrations agile.)
In this example we are going to change two things from the first example.
1) Instead of having a static itinerary resolver, the itinerary will be resolved dynamically by the OnRamp.
2) An additional itinerary will be added that will use different resolvers for determining the OffRamp properties, and will also add a dynamic transformation to the message processing.
To make the first change we need to modify the Receive pipeline configuration in the OnRamp, as in Figure 11.
Figure 11: New OnRamp pipeline configuration
The change made is to use the BRE resolver (named BRI in this case for itinerary resolution) instead of a static (database lookup) resolution. Given this change, we can now apply any number of itineraries to messages coming in through this OnRamp. In Figure 12 you can see a simple policy that changes the
18
itinerary based on the SOAP Action header, but any number of algorithms could be applied. In this case, because we've used a policy, we can switch out the policy at any time to change the itinerary resolution in my OnRamp).
Figure 12: Simple itinerary resolution policy
19
Next we need the additional itinerary, as shown in Figure 13.
Figure 13: Slightly more complex itinerary
You can see that we've added an extender that uses the messaging extender. It can be configured with one or more resolvers. In this case, we’re using the BRE resolver again, only this time the policy (SimpleMapResolve, which we've set in the BREMapResolver properties) is going to be used to determine what map to apply to the message as it’s passing through the system.
20
Figure 14 shows a portion of this policy. Note that we are using an ESB-provided fact to set the name of the transformation (BizTalk map) to be applied.
Figure 14: Simple map resolving policy
To summarize the ESB Toolkit application at this point we have: 1) An OnRamp that dynamically resolves the message itinerary using a BRE Policy.
2) Two itineraries that can currently be applied (we can add more without having to change the application at all).
3) At least one itinerary that will dynamically resolve the map to be applied to messages based on a BRE Policy.
4) A BRE Policy that will determine the adapter and URI of the OffRamp.
It is important to remember that there are other resolvers that can be used in place of the BRE resolvers (and of course you can write your own custom resolvers as well). The main point is that by using the dynamic capabilities of the BizTalk ESB Toolkit, each of your integrations are guaranteed to be dynamic, extensible, maintainable, and less costly to change.
21
ESB Management PortalAnother valuable capability of the BizTalk ESB Toolkit is that it includes an ASP.NET Portal sample. This portal sample is preprogrammed to provide functionality immediately useful for administering your BizTalk ESB Toolkit applications.
It includes the functionality shown here:
Figure 15: Functionality of the ESB portal
For example, the portal could be used by a developer or administrator to create notification alerts (email by default) when certain conditions occur. An alert could be created for developers to be notified whenever there is a particular kind of error, or when an error occurs in a specific application.
If there is an error, a user can see all the information about the message in the portal, and can actually edit the message and resubmit (assuming that the error condition can be fixed by editing the raw message). Shown below are some screen captures of the portal to illustrate how these capabilities are exposed to
22
Graphical metrics Repair and resubmit functionality
Alerting based on exception events
Auditing trails for repair and resubmit
Unified view of the .NET Exception data + BizTalk
message + BizTalk context properties
Historical views of exception data
Remote web based access
Filtering of exceptions based on application
business users:
Figure 16: ESB Portal – Exception Report & Charts
23
Figure 17: ESB portal – filtering Exceptions
Figure 18: ESB portal – Viewing Exception Details
Figure 18: ESB portal – Managing Exception Alerts and Subscriptions
The portal sample can be augmented or enhanced to cater to specific organization needs since all its source code is provided with the ESB Toolkit.
It should be noted though that although the portal is provided as a sample program with the ESB Toolkit, there are other ways that you can interact with the components. For example, consider an ordering process where a SharePoint portal has been provided to the order management team where they can see process metrics; and perform any human intervention that is often required in automated processes, and more. Leveraging the ESB Toolkit’s exception management framework, you could create a “listener” that picks up exception messages (technical exceptions as well as business exceptions) related to the order process and publishes them to the order process portal.
24
ESB as a Service
Another way to use the BizTalk ESB Toolkit (which can be in conjunction or separate from the capabilities already described so far) is to use the component directly as services.
Most of the ESB services discussed so far are exposed as standard Web services. For example, if you want to use transformation as a service rather than hard-coding a map inside of an Orchestration or from .NET code, you could instead call out to the transformation service.
Itinerary resolution is exposed as a service, so if you wanted to query that service and get an itinerary dynamically before sending a message into an OnRamp you could do that as well.
The fact that the BizTalk ESB Toolkit’s capabilities are exposed as standard Web services means that the possibilities for different patterns and configurations are almost endless.
SummaryThe BizTalk ESB Toolkit is a rich addition to BizTalk Server. It provides powerful capabilities on top of BizTalk Server that when used provides deep technical and business value by laying the groundwork for building loosely-coupled mediated services as well as allowing rapid business change in response to changing market conditions.
Itineraries and itinerary routing provide a dynamic, manageable, and versioning-friendly layer of top of existing BizTalk features such as routing, message and protocol transformation, and processing (orchestration).
The Itinerary Designer is built using a developer-friendly toolset hosted inside of Visual Studio to leverage existing developer skills. Also, the components that make up the BizTalk ESB Toolkit itinerary processing are built on top of BizTalk features, meaning that existing BizTalk skill sets are automatically leveraged. And each of these components is not only flexible, but also extensible, enabling customization when necessary.
The ESB Management Portal enables automatic visibility into the ESB’s message processing, as well as adding powerful features such as message re-submit. Adding to that the exposure of the ESB’s main features as services in their own right means that the BizTalk ESB Toolkit can be leveraged from other applications outside of BizTalk itself.
The BizTalk ESB Toolkit provides a stable and powerful platform for services that can change as fast as your business needs them to.
