Did you know? DZone has great portals for Python, Cloud, NoSQL, and HTML5!

SOA

  • submit to reddit

Mule 3

Simplifying SOA

By John DEmic

9,941 Downloads · Refcard 140 of 151 (see them all)

Download
FREE PDF


The Essential Mule 3 SOA Cheat Sheet

Mule 3 is an open-source integration platform and Enterprise Service Bus (ESB) designed to support high-performance, multi-protocol transactions between heterogeneous systems and services. It provides a foundation for service-oriented architecture (SOA) and it can run standalone or embedded directly within your application. This DZone Refcard covers Mule 3 and serves as a reference for users new to Mule or users of Mule 2 who are transitioning to Mule 3. The card covers new features such as orchestration flows, native REST support, Ajax/JavaScript integration, cloud connectors, and more.
HTML Preview
Mule 3: Simplifying SOA

Mule 3:Simplifying SOA

By John D’Emic

ABOUT MULE

Mule is the world’s most widely used open-source integration platform and Enterprise Services Bus (ESB). Mule is designed to support high-performance, multi-protocol transactions between heterogeneous systems and services. It provides the basis for service-oriented architecture. It is lightweight and can run standalone or embedded directly in your application.

This Refcard covers the use of Mule 3. If you are a new user, it will serve as a handy reference when building your integration flows in Mule. If you are an existing user, especially of Mule 2, it will help ease your transition into using Mule 3.

MULE XML

The programming model for Mule is XML using namespaces that provide a DSL authoring environment to orchestrate your integration applications. The diagram below demonstrates the structure of a typical Mule configuration.


<?xml version=”1.0” encoding=”UTF-8”?>
<mule xmlns=”http://www.mulesoft.org/schema/mule/core”
	xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
	xmlns:vm=”http://www.mulesoft.org/schema/mule/vm”
	xmlns:jms=”http://www.mulesoft/org/schema/mule/jms”
	xmlns:file=”http://www.mulesoft.org/schema/mule/file”
	xsi:schemaLocation=”
		http://www.mulesoft.org/schema/mule/core
		http://www.mulesoft.org/schema/mule/core/3.1/mule.xsd
		http://www.mulesoft.org/schema/mule/vm
		http://www.mulesoft.org/schema/mule/vm/3.1/mule-vm.xsd
		http://www.mulesoft.org/schema/mule/jms
		http://www.mulesoft.org/schema/mule/jms/3.1/mule-jms.xsd
		http://www.mulesoft.org/schema/mule/file
		http://www.mulesoft.org/schema/mule/file/3.1/mule-file.xsd
		“>
		
<description>Demonstrate Mule Configuration Elements</description>

<jms:connector name=”jmsConnector”
			connectionFactory-ref=”connectionFactory”
			username=”guest”
			password=”guest”/>

<file:connector name=”fileConnector” streaming=”true”/>

<flow name=”Route messages dynamically using a message property”>
  <vm:inbound-endpoint path=”input”/>
  <vm:outbound-endpoint path=”#[header:INBOUND:destination-queue]”/>
</flow>

<simple-service name=”random-number-service”
			address=”http://localhost:8080/rest”
			component-class=”com.mulesoft.refcard.
RandomNumberResource”
			type=”jax-rs”/>
</mule>

Mule XML consists of the following:

  • An opening mule element containing the namespaces used in the configuration.
  • A description of the purpose of the configuration.
  • Connector definitions.
  • Global configuration elements like endpoints, transformers and notification listeners.
  • Flows, patterns and services.

Flows

Flows, new to Mule 3, provide a free-form method of orchestrating message processing in Mule. A flow consists of a message source, typically an inbound-endpoint, followed by a sequence of message processors.

Flow Chart
Message Processor

Message Processors are used in flows to route, transform, filter and perform business logic on messages. MessageProcessor is a single method interface implemented by almost everything in a Flow.

Exception Strategy

An exception strategy can be added to the end of the flow to route errors that occur during the flow’s execution.

Hot Tip

Optionally end a flow with an outbound router or endpoint to send to another flow or external service.
Sending a JMS Message with a Flow

Sending a JMS message is easy with a flow. Here’s how you can use a flow to read files from a directory and send their payload to a JMS queue.


<flow name=”File to JMS Message”>
	<file:inbound-endpoint path=”data/files”>
		<byte-array-to-string-transformer/>
	</file:inbound-endpoint>
	<jms:outbound-endpoint queue=”output”/>
</flow>

The file:inbound-endpoint will read files from the given path, transforming their contents into Strings. The content is then passed as JMS TextMessages to the “output” queue.

Using Patterns

Mule Configuration Patterns, extending the Enterprise Integration Patterns, encapsulate common integration paradigms in a compact configuration format.

Creating a RESTful Web Service

A RESTful web service can be quickly created using the simpleservice pattern.


<simple-service name=”random-number-service”
		address=”http://localhost:8080/rest”
		componentclass=”org.refcard.RandomNumberResource”
		type=”jax-rs”/>

In this case, we are using the simple-service pattern to expose a JAX-RS resource that returns a random number from an HTTP GET request.

Hot Tip

Flows and patterns now supersede services, which were the predominant integration paradigm in Mule 2. While services will always be supported, you should favor flows and patterns for new applications.

MESSAGES

Messages encapsulate data entering and leaving Mule. The content of a message is called its payload. The payload is typically a Serializable Java class, an InputStream or an array of bytes.

Attachments

A message can have zero or more mime attachments in addition to the payload. These can be used to associate files, documents and images with the message.

Properties

Properties, also called headers, are metadata associated with a message. Mule, the various transports and you the developer can add properties to a message. Examples of message properties are JMS message headers, HTTP response headers or Mule-specific headers like MULE_MESSAGE_ID. The following table contains examples of message properties set by Mule.

Property Description
MULE_MESSAGE_ID A GUID assigned to the message.
MULE_CORRELATION_ID A GUID assigned to a group of messages.
MULE_CORRELATION_GROUP_SIZE The amount of messages expected in the correlation group.
MULE_CORRELATION_SEQUENCE The order of a correlation group.
MULE_SESSION A property that holds encoded session data./td>
Scopes

Properties are scoped differently depending on when they’re set or accessed during message processing. The following table contains the available scopes.

Scope Description
inbound Set by message sources, typically an inbound-endpoint.
outbound Any properties in this scope will get attached to an outbound or response message. Properties set by the messageproperties- transformer default to the outbound scope.
session Properties in the session scope are available between flows and services without explicit propagation.
invocation Used internally by Mule and lasts only for an invocation of a flow or service.

CONNECTIVITY

Mule connects to over 100 applications, protocols and APIs. Mule endpoints enable connectivity to protocols such as JMS, HTTP and JDBC. Cloud Connectors enable connectivity to applications and social media like SalesForce and Twitter.

Endpoints

Messages can be received with an inbound endpoint and sent with an outbound endpoint. Inbound and outbound endpoints are configured using the XML namespace prefix of the transport.

Connectors

A connector is used to configure connection properties for an endpoint. Most endpoints don’t require a connector. However, some (like JDBC or JMS) do require connector configuration, as we’ll see below.

Configuring an SMTP connector

The following example illustrates how an SMTP connector is configured.


<smtp:connector name=”smtpConnector”
	fromAddress=”user@foo.com”
	bccAddresses=”admins@foo.com”
	subject=”A Message from Mule” />

The SMTP connector allows you to specify properties that will be shared across SMTP endpoints. In this case, the connector sets the “from” and “bcc” addresses as well as the subject of the messages. A connector is referenced by its name, allowing you to define multiple connectors for the same transport.

Endpoints can be generically referenced using an endpoint URI as well as having specific XML configuration elements.

The following table describes some common endpoints supplied by Mule.

Endpoint Description
http://[host]:[port]:[path]?[query] Send and receive data over HTTP.
ajax://[channel] Pub / Sub to browser apps using CometD.
file://[path] Read and write files.
ftp://[user]@[host]:[port]/[path] Read and write files over FTP or SFTP.
jms://[type]:[destination]?[options] Full support for JMS topics and queues.
smtp://[user]@[host]:[port] Send email over SMTP.
imap://[user]@[host]:[port]/[folder] Receive email via IMAP.
jdbc://[sql query] Send and receive data from a SQL database.
vm://[path] Uses memory-based queues to send between services and flows.

The full list of transports is available in the Mule documentation.

Hot Tip

Use exchange patterns to define how a message is recieved by an endpoint. For endpoints that generate a response(synchronous) use the request-response, for asynchronous endpoints use the one-way exchange pattern.

Cloud Connectors

Cloud connectors enable easy access to SaaS, social media and infrastructure services such as Amazon WS and Facebook.

These connectors can be used anywhere in a flow to invoke a remote service. A cloud connector usually has a ‘config’ element where service credentials are set and then one or more elements that invoke service methods. The following demonstrates how the Twitter cloud connector can be used to post a tweet:


curl –-data “status=go mule!” http://localhost
<twitter:config name=”twitter” format=”JSON”
	consumerKey=”${twitter.consumer.key}”
	consumerSecret=”${twitter.consumer.secret}”
	oauthToken=”${twitter.access.token}”
	oauthTokenSecret=”${twitter.access.secret}” />

<flow name=”updateStatusFlow”>
	<http:inbound-endpoint host=”localhost” port=”80”/>
	<twitter:update-status
		status=”#[header:INBOUND:status]”/>

</flow>

We can now post a tweet to the inbound-endpoint with curl:


curl –-data “status=go mule!” http://localhost

Polling

Mule has a poll tag that allows data from a remote service to be received periodically. To get updates from a Twitter timeline:


<flow name=”getStatusFlow”>
	<poll>
		<twitter:public-timeline />
	</poll>
</flow>


MODULES

Modules extend Mule’s functionality by providing namespace support for a certain set of message processors. The following table contains some of the modules provided by Mule.

Module Description
JSON JSON support, including marshalling, transformation and filtering.
CXF SOAP support via Apache CXF
Jersey JAX-RS support for publishing RESTful services.
Scripting Support for JSR-223 compliant scripting language, like Groovy or Rhino.
XML XML support, including XML marshalling, XPath and XSLT support.

The full list of available modules is available in the official Mule documentation. Additional modules are available on MuleForge.

Hot Tip

Use MuleForge.org to locate community-written extensions.
Bridging REST to SOAP

The following demonstrates how the CXF module can be used to bridge a RESTful service to a SOAP service.

The inbound-endpoint accepts HTTP POST requests to http://localhost:8080/service. The POST data is then sent to the SOAP service defined by the CXF jaxws-client.

Routers

Routers implement the Enterprise Integration patterns (EIP) and determine how messages are directed in a flow.

The following table contains commonly used routers

Router Description
all Sends the message to each endpoint.
choice Sends the message to the first endpoint that matches.
recipient-list Sends the message to all endpoints in the expression evaluated with the given evaluator.
round-robin Each message received by the router is sent to alternating endpoints.
wire-tap Sends a copy of the message to the supplied endpoint then passes the original message to the next processor in the chain.
first-successful Sends the message to the first endpoint that doesn’t throw an exception or evaluates the failureExpression to true.
splitter Will split the current message into parts using an expression or just split elements of a List.
aggregator Will collect related messages and create a message collection.

Transformers

Transformers modify the message and pass it to the next message in the chain.

The following table contains commonly used transformers.

Name Description
message-propertiestransformer Adds and removes properties from a message, optionally specifying their scope.
byte-array-to-stringtransformer Many basic type transformers are included.
xml:jaxb-xml-to-objecttransformer Transforms JAXB objects explicitly.
auto-transformer Will automatically find the best transformer for a specified type.
xml:xslt_transformer Transforms a message using the given stylesheet.
json:object-to-jsontransformer Transforms message payloads to and from JSON.
gzip-compresstransformer Compresses and uncompress message payloads using gzip.
encrypt-transformer Encrypts and decrypts message payloads.

Hot Tip

Endpoints often include their own transformers. JMS for instance, allows transformers to convert message payloads to and from JMS messages autormatically.

Components

Components allow business logic to be executed in a flow. Any Java object or script can be used as a component. Components are configured by either identifying the class or providing a reference to a Spring bean for dependency injection.

The following snippet shows how a class called MyService can be configured as a component using a class and via dependencyinjection via Spring.


<bean class=”com.acmesoft.service.MyService”/>

<flow name=”test”>
	<http:inbound-endpoint host=”foo.com”>
	<component>
		<spring-object bean=”myService”/>
	</component>
</flow>

Mule will use the type of the payload of the message being processed to determine what method to invoke. It’s often necessary, however, to explicitly specify the method to invoke. Entry point resolvers are used for this purpose. The following table contains a list of available resolvers.

Resolver Description
method-entry-point-resolver Resolves the method using the specified name.
property-entry-point-resolver Resolves the method using the specified message property.
custom-entry-point-resolver A Java class that implements org.mule.api.model.EntryPointResolver or extends org.mule.model.resolvers. AbstractEntryPointResolver.

The use of entry point resolvers allows you to use POJO’s as components, decoupling your code from Mule. Sometimes, you will want access to the MuleMessage or MuleContext when processing a message. In cases like this, you can implement the org.mule.api.lifecycle.Callable interface. Callable includes a single method, onCall, to implement that provides direct access to the MuleMessage when the method is invoked.

In addition to custom components, Mule provides the following utility components.

Component Description
<log-component> Logs messages.
<echo-component> Returns and passes along.
<test:component> Helps test message flows (in the test namespace).

Hot Tip

Try to avoid implementing Callable to keep your component code decoupled from Mule’s API.

Filters

Filters selectively pass messages to the next processor in the chain.
The following table contains commonly used filters.

Name Description
expression-filter Passes messages using any of the expressions languages supported by Mule.
regex-filter Decides what messages to pass by applying the supplied regular expression to the message payload.
payload-type-filter Passes messages only of the given type.
custom-filter Specifies the class of a custom filter that implements the org.mule.api.routing.filter.Filter interface.
and-filter, or-filter, not-filter Logic filters that work with other filters.
Using Filters with XPath

The following example demonstrates how the xpath-filter can be used to only pass certain XML documents. In this case, only order XML documents containing a certain ZIP code are allowed to pass.


<flow name=”Filter messages using the XPath filter”>
		<vm:inbound-endpoint path=”input”/>
		<mulexml:xpath-filter pattern=”/order/zipCode”
				expectedValue=”11209”/>
		<vm:outbound-endpoint path=”output”/>
</flow>

EXPRESSIONS

Mule provides a rich expression language to evaluate data at runtime using the message currently being processed.

Evaluators

The following are commonly used expression evaluators.

Name Description
xpath Query the message payload using XPath
payload Use the message’s payload for evaluation.
map-payload A Java class that implements org.mule.api.model. EntryPointResolver or extends org.mule.model.resolvers. AbstractEntryPointResolver.
regex Perform a regular expression evaluation against a message payload.
bean Evaluates the message payload as a JavaBean.
groovy Use Groovy to evaluate an expression. Mule provides certain variables, like payload, properties and muleContext, to the script context to aid in evaluation.
header:[scope] Return the given header for a specific scope, as demonstrated below
Routing Messages Dynamically

The following illustrates how a message can be dynamically routed to a JMS queue passed on a message header.


<flow>
	<vm:inbound-endpoint path=”input”/>
	<jms:outbound-endpoint
		queue=”#[header:INBOUND:destination-queue]”/>
</flow>

This example accepts a message off the given VM queue. The outbound-endpoint uses the header evaluator to dynamically route the message to the queue defined by an inbound message property called “destination-queue”.

ANNOTATIONS

Annotation support, introduced in Mule 3, further simplifies Mule configuration by reducing or eliminating the XML needed to configure components and transformers.

The following table contains a list of commonly used annotations.

Name Type Description
@Transformer Method Indicates the method can be used as a transformer.
@ContainsTransformerMethods Class Indicates the annotated class contains transformation methods.
@Schedule Method Schedules a method for periodic execution.
@Lookup Field,Parameter Looks up the annotation field or parameter for dependency injection from the Mule registry.
@Payload Parameter (component or transformer) Injects the payload into a method. If the param type is different from the payload type, Mule will attempt to transform it.
@InboundHeaders Parameter (component or transformer) Injects any inbound headers at runtime. Can be filtered by name and used to inject individual headers.
@OutboundHeaders Parameter (component or transformer) Injects a Map of outbound headers that can be used to add headers to the outgoing message.
@Inbound Attachments Parameter (component or transformer) Injects any inbound attachments at runtime. Can be filtered by name.
@Outbound Attachments Parameter (component or transformer) Injects a Map of outbound attachments that can be used to add attachments to the message.

You can use the Mule 3 annotation support to quickly implement transformers and components.

Implementing a Transformer with Annotations

Here’s an example of a transformer that lowercases a message’s payload.


@ContainsTransformerMethods
public class LowercaseTransformer {

	@Transformer
	public String toLowercase(String string) {
		return string.toLowerCase();
	}
}

This class can be used a message source in a flow to generate a timestamp every minute.

Implementing a Component with Annotations

Components can also be implemented with annotations. The following class demonstrates how the @Schedule annotation can be used to periodically generate data.


public class HeartbeatMessageSource {
	@Schedule(interval = 60000)
	public long pulse() {
		return new Date().getTime();
	}
}

This class can be used as a message source in a flow to generate a timestamp every minute.

HANDLING ERRORS

Exceptions thrown during message processing are handled by exception strategies. The default-exception-strategy, configured at the end of the flow, allows you to route the exception to another endpoint for handling.

Sending Messages to a DLQ

Messages that can’t be delivered, for instance if an exception is thown during their processing, it can be sent to a Dead Letter Queue (DLQ). The following example will send any errors that occur in the flow to the “dlq” VM queue.


<flow>
	<vm:inbound-endpoint path=”input”/>
	<vm:outbound-endpoint
		path=”#[header:INBOUND:destination-queue]”/>
	<default-exception-strategy>
		<vm:outbound-endpoint path=”dlq”/>
	</default-exception-strategy>
</flow>

Building upon the previous example, the default-exceptionstrategy will route messaging failures (for example, when the destination-queue inbound header is null) to the VM queue named “dlq”.

Hot Tip

Exceptions routed by the default-exceptionstrategy are instances of org.mule.api.message. ExceptionMessage, which gives you access to the Exception that was thrown along with the payload of the message.

FUNCTIONAL TESTING

Functional testing is an important part of testing Mule applications. Mule provides a helper class, org.mule.tck. FunctionalTestCase, which you can extend to simplify setting up a TestCase.

Functionally Testing a Flow

public class XPathFilterFunctionalTestCase extends FunctionalTestCase
{
	@Override
	protected String getConfigResources() {
		return “xpath-filter-config.xml”;
	}

	public void testMessageNotFiltered() throws Exception {
		String xml = “<order><zipCode>11209</zipCode></order>”;
		
		MuleClient client = new MuleClient(muleContext);
		client.dispatch(“vm://input”, xml, null);

	assertNotNull(client.request(“vm://output”, 15000).
		getPayloadAsString());
	}

	public void testMessageIsFiltered() throws Exception {
		String xml = “<order><zipCode>11210</zipCode></order>”;

		MuleClient client = new MuleClient(muleContext);
		client.dispatch(“vm://input”, xml, null);

		assertNull(client.request(“vm://output”, 5000));
	}
}

Hot Tip

The test-component can be used to simulate remote services during functional testing.

CONCLUSION

This Refcard is just a glimpse into what you can do with Mule 3. Consult the full documentation on the MuleSoft website. The full code and test cases for the examples used in this Refcard are available on GitHub:

https://github.com/johndemic/essential-mule-refcard

About The Authors

John D’Emic

John D’Emic is a software developer and author. He has used Mule extensively since 2006 and is the “despot” of the MongoDB transport. He also co-authored Mule in Action with David Dossot in 2009. You can read about what he’s up to in his blog: johndemic.blogspot.com.

Recommended Book

Mule in Action

Mule in Action covers Mule fundamentals and best practices. It is a comprehensive tutorial that starts with a quick ESB overview and then gets Mule to work. It dives into core concepts like sending, receiving, routing and transforming data. Next, it gives you a close look at Mule’s standard components and how to roll out custom ones. You’ll pick up techniques for testing, performance tuning, BPM orchestration and even a touch of Groovy scripting.

Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

Effective Process Modeling with BPM & BPMN

By Matjaz B. Juric and Ana Sasa

16,392 Downloads · Refcard 51 of 151 (see them all)

Download
FREE PDF


The Essential BPM & BPMN Cheat Sheet

Business Process Management (BPM) is a set of related activities, such as process modeling and design, process execution, process monitoring, and process optimization. BPM is based on a flowcharting technique similar to activity diagrams from Unified Modeling Language (UML). This DZone Refcard provides an overview of the BPM lifecycle together with the roles and results of business process modeling. It gives an overview of the Business Process Modeling Notation and presents the most important BPM patterns. The Refcard also covers BPMN workflow patterns, other constructs, hot tips and more.
HTML Preview
Effective Process Modeling with BPM & BPMN

Effective Process Modeling: with BPM & BPMN

By Matjaz Juric and Ana Sasa

About Business Process Management

BPM (Business Process Management) is a set of related activities, such as process modeling and design, process execution, process monitoring, and process optimization. This Refcard provides an overview of the BPM lifecycle together with the roles and results of business process modeling. It gives an overview of the BPMN (Business Process Modeling Notation) and presents the most important BPM patterns.

BPM: Business Process Lifecycle

A business process lifecycle covers the following phases (Figure 1):

  • Process modeling - definition of the process models using the selected methodology and notation (such as BPMN).
  • Process implementation – implementation of end-to-end IT support for the process. SOA provides technologies and tools to make the implementation phase quick and efficient.
  • Process execution and monitoring – execution of the process and monitoring of the process to gather the Key Performance Indicators (KPI).
  • Process simulation – simulated execution of the process with the objective gathering KPIs and identifying optimization points.
  • Process optimization – improving the process efficiency, effectiveness, agility, flexibility, and transparence.
BPM process lifecycle

Figure 1: BPM process lifecycle

Hot Tip

KPIs are financial and non-financial metrics used to help an organization define and measure process efficiency. Examples of a KPI are “Average revenue per customer”, “Average time for response to a customer call”, “ Average order amount”, etc.

Hot Tip

Business activity monitoring (BAM) is real-time observation of key performance indicators.

BPM: Modeling

Why do we Model Business Processes?

Design new business processes Focus on business goals, KPIs, customer needs, and business partner expectations.
Model existing business processes Assure the right flow of activities. Identify normal flows and possible exceptional flows.
Identify inputs and outputs of activities.
Identify key documents and sources.
Identify business rules.
Restructure existing business processes Focus on the activities and their added value. Focus on lines of business and their relations. Model responsibilities and roles.
Development of endto-end IT support for business processes Detailed modeling of process flow.
Detailed modeling of data, documents, business objects, and
interfaces.
Detailed exception handling.

Who should take part in process modeling?

The team should include different profiles and encourage looking at the process from different angles. This is particularly important for optimizations. Four to six people is usually an optimal team size. The following table lists the various profiles that should comprise the team:

Role Responsibility
Line of Business Expert Good, in-depth knowledge of the process.
Process Owner Responsible for
Moderator Responsible for the meeting, for asking questions for leading the discussion into the right direction.
Modeling Expert Responsible for design the process model (during and after the meeting).
QA Owner Responsible for the alignment of processes in aspect of total quality management.

How do we model?

Approach Problems
Top-down We start with the process architecture. First we identify the major process activities and their flow. Then we model each activity into more detail.
  • High level process modeling requires good knowledge about the process and some experience.
  • Modeling lower levels can reveal inconsistencies on higher-levels.
Bottom-up We start with the identification of activities. We model sub processes and business transactions and merge them into processes.
  • We get lost in the details.
  • Getting overview of processes and their relations can become very difficult.
  • We can focus on too many details.
Inside-out We start with core processes. We expand them with adding support processes around core processes.
  • It can be difficult to identify core processes and how to progress into the right direction.

Hot Tip

The Inside-out approach is usually the most pragmatic approach to prcess modeling. Provide a brief explanation of why it is the most pragmatic approach.
As-Is model We model the process as it is currently executed. Knowing the current as-is state is necessary for any future optimizations. We need to clarify whether we will model the process as it should be performed, or as it is performed in reality. Often there are significant differences between the two. When we model the as-is process we should not make on-the-fly modifications - not even those which seem obvious. We should however make notes of all possible modifications for the to-be process model.
To-Be model We model the optimized model, where we should consider:
  • Extent of changes – do we want evolutionary or revolutionary changes
  • How radical the changes to the process can be
  • Organizational and other limitations
  • How the to-be model will be accepted by the involved people and what organizational changes will it require

How to approach designing a process model:

We should model the process to understand the detailed structure of it. We should identify at least the following:

  • Process activities, on various levels of details (depending on the selected approach)
  • Roles responsible for carrying-out the process activities
  • Events, which trigger the process execution and events that interrupt the process flow
  • Input and output documents exchanged within the process
  • Business rules that are part of the process

Below is the most conventional approach for designing a process model, in order of occurrence:

  1. Identify the roles
  2. Identify the activities
  3. Connect the activities with roles
  4. Define the order of activities
  5. Add events
  6. Add documents
Process model

Figure 2: Process model for each individual process

Process model 2

Figure 3: Results of Business Process Modeling

About BPMN

BPMN (Business Process Modeling Notation) is a graphical notation for business process modeling. The objective of BPMN is to support business process modeling for business and technical users. It provides a notation that is intuitive yet able to represent complex process semantics. BPMN is maintained by the Object Management Group.

Flow Objects

Flow objects are the main BPMN constructs that define the behavior of a business process. There are three categories of flow objects:

  • Activities: they represent the work performed within a business process (see Figure 4).
  • Gateways: they represent how a sequence flow diverges or converges in a business process (see Figure 5).
  • Events: they depict that something happens in a business process (see Figure 6).
Figure 4

Figure 4: Activity types and markers

Figure 5

Figure 5: Types of gateways

Figure 6

Figure 6: Events, event triggers and results

Connecting Objects

Connecting objects are used to connect flow objects to each other and to other information. There are three categories of connecting objects: Sequence flow ( see Figure 7), Message flow ( see Figure 9), Association ( see Figure 11).

arrow right Defines the order of execution of flow objects.
sequence Sequence flow with a condition (conditional flow).
Default Flow Default flow, which is chosen if none of the conditions are satisfied.

Figure 7: Sequence Flow

Figure 8

Figure 8: Construct that can be connected via sequence flow (blue shaded field represent a legal connection)

Figure9 Shows the flow of messages between two entities.

Figure 9: Message Flow

Figure 10

Figure 10: Construct that can be connected via message flow (blue shaded field represent a legal connection)

Figure 11

Figure 11: Association

Other constructs

swimlanes

Figure 13: Swimlanes and pools

Artifacts

Figure 14: Artifacts

Exception Flow

In order to model an exception flow, we use intermediate events attached to the boundary
of an activity. If such event is triggered during the activity execution, the flow is redirected
through the intermediate event.
Example: The activity
Check With Supplier
of the example
process has an
intermediate timer
event attached to
its boundary. If the
supplier does not
provide a response
within a certain
timeframe, we
remove the item from
the order.
Exception Flow

WorkFlow patterns with bpmn

Sequence
Workflow Pattern Description:
An activity starts after completion of another activity.
BPMN:
Activities are connected by a sequence flow directed towards the subsequent activity.
Example: After
checking if the
supplier can provide
the necessary items
in the Check With
Supplier task, we
notify the customer
about their order in
the Notify Customer
task.
Workflow
Parallel Split
Workflow Pattern Description: A path diverges into two or more parallel subsequent paths. The subsequent paths execute concurrently.
BPMN: The pattern can be implemented in several ways:
  • We use several outgoing sequence flows for a flow object;
  • We use a parallel gateway to divide a sequence flow into several sequence flows.
  • We use an expanded sub-process in which we place the activities to be performed in parallel.
  • We use an inclusive gateway with equivalent conditions.
Example 1: After receiving payment for the order we prepare the ordered items for shipment and issue the receipt concurrently.
Solution 1: Parallel split
with outgoing sequence
flows.
solution1
Solution 2: Parallel split using
a parallel gateway solution2
Solution 3: Parallel split using
an expanded sub-process solution3
Example 2: If the order items are in stock we send the confirmation of the order to the
customer and reserve the ordered items in the inventory. These tasks are performed in
parallel. Otherwise we check if the supplier can deliver the items
Solution 1: Parallel split using an inclusive gateway
solution1.1
Solution 2: Parallel split using a parallel gateway
solution2.1
Synchronization
Workflow Pattern Description: Two or more paths converge into one subsequent path. The
subsequent path is enabled when all the preceding paths complete (and-join).
BPMN: The Pattern can be implemented in two ways:
  • We use a parallel gateway to merge several sequence flows into a single flow. The outgoing flow activates when all the incoming sequence flows are enabled.
  • We use an expanded sub-process in which we place the activites to be performed in parallel. Expanded sub-process completes after all the activities it contains complete.
Example 1: After preparing the ordered items for shipment and issuing the receipt, we ship
the package to the customer.
Solution 1: Synchronization using a parallel gateway.
solution2.1
Solution 2: Synchronization using an expanded sub-process.
solution2.1
Exclusive Choice
Workflow Pattern Description: A path diverges into two or more subsequent paths. When
the incoming path is enabled exactly one of the subsequent paths is selected and enabled.
BPMN: We use an exclusive gateway.
After analyzing the order
we check whether the
customer has provided
a promotional code.
If a promotional code
is provided we collect
discount information and
use it to calculate final
price. Otherwise, we
calculate final price for the
order without discounts.
Example 1: Exclusive choice with data-based exclusive gateway
Example1
After we notify the
customer about the
earliest possible delivery
of the ordered items, the
customer may change the
ordered items, confirm the
proposed date or cancel
the order. If the customer
does not respond in a
certain timeframe an
intermediate timer event is
triggered.
Example 2: Exclusive choice with event-based exclusive gateway
Example2
Simple Merge
Workflow Pattern Description: Two or more alternative paths converge into a single
subsequent path.
BPMN: The pattern can be implemented in two ways:
  • We use an exclusive merge gateway to merge alternative paths.
  • We use a flow object with two or more incoming sequence flows. The incoming sequence flows represent the ends of alternative paths. Any one of the incoming sequence flows trigger the flow object.
  • Note: The behavior is the same in both cases provided that the incoming sequence flows are alternative.

Example: The two alternative paths used to calculate the final price of the ordered items are merged using the exclusive merge or by sequence flows leading to the “Check Inventory” task.
Solution 1: Simple merge with exclusive merge gateway
Solution 1
Solution 2: Simple merge with sequence
flows to a flow object
Solution 2

Advanced Branching and synchronization pa tterns

Multi-Choice
Workflow Pattern Description: A path is diverged into two or more subsequent paths. One or
more subsequent paths may be executed.
BPMN: The pattern can be implemented in several ways:
  • We use an inclusive gateway.
  • We use a collection of contidional sequence flows.
  • We use a complex gateway.
Example 1: Based on requirements the customer specified in the order, we confirm the order
via e-mail, by regular mail or both. Example solutions 1 nd 2 represent equivalent behavior.
Solution 1: Multi-Choice with an inclusive gateway
Sol 1 inclusive
Solution 2: Multi-Choice with conditional sequence flows
Solution 2 sequence
Example 2: An order from the received order
list may concern one or more departments.
Depending on this, one, two or all three
subsequent branches can be executed.
Example 2 branches
Structured Synchronizing Merge (Synchronizing join)
Workflow Pattern Description: Two or more paths converge into a single subsequent path. Several incoming paths may be enabled, in which case they are synchronized before the subsequent path is activated. In different process instances different number of incoming paths may be taken.
BPMN: We use an inclusive gateway.
Example: Based on requirements the
customer specified in the order, we confirm
the order via e-mail, by regular mail or both.
If both activities are required to be executed,
paths have to be synchronized before the
process can continue.
Diagram1
Multi-Merge (Multiple Merge)
Workflow Pattern Description: Two or more paths converge into a single subsequent path. Each Incoming path activates the subsequent path.
BPMN: We use sequential flow for every ending of a converging path directed towards the flow object of the beginning of the subsequent path.
Example: We confirm the order via e-mail, by
regular mail or both. if either of the activities
takes place, the order information file needs
to be updated.
Diagram2

Iteration based patterns

Arbitrary Cycles (Unstructured Loop)
Workflow Pattern Description: Loops that have more than one entry or exit points.
BPMN: Sequence flow connected to an upstream activity.
Updated Inquiry
Structured Loop
Workflow Pattern Description: A task or a subprocess is repeated while or until some condition is true.
BPMN: We set the attributes of the activity as follows:
  • We set the value of the LoopType attribute to “Standard”,
  • We set the contidion expresion for the attribute LoopCondition,
  • to model a “while” loop we set the value of the attribute TestTime to “Before”,
  • to model an “until” loop we set the value of the attribute TestTime to “After”.
Example: After receiving a list of orders the
Process Order subprocess is performed
for every order until the end of orders is
reached in the list.
List 1

Multiple instance patterns

Multiple Instances without Synchronization
Workflow Pattern Description: Multiple instances of a task or a subprocess are created. They run concurrently and are not synchronized on completion.
BPMN: We set the values of activity attributes as follows:
  • LoopType attribute to “multiInstance”,
  • MI FlowCondition to “None”.
  • we set the value of the MI_Ordering attribute to “Parallel”
  • Example: For every order in the order list an
    instance of the Process Order subprocess
    is invoked. The subprocess instances are
    executed concurrently. Every instance
    generates a token that continues after the
    instance is completed.
    Process Order
    Multiple Instances with a Priori Design-Time Knowledge
    Workflow Pattern Description: Multiple instances of a task or a subprocess are created. The number of instances is known at design time. They run concurrently and are synchronized at completion before the process continues.
    BPMN:
    We set the attributes of the activity as follows:
    • we set the value of the LoopType attribute to “MultiInstance”,
    • the expression of the MI_Condition attribute returns an integer representing the number
      of instances known at design time,
    • we set the value of the MI_FlowCondition attribute to “All”.
    • we set the value of the MI_Ordering attribute to “Parallel”
    Example: If a request for a loan exceeds
    1000 USD the loan needs to be checked for
    approval by 3 eligible employees.
    checked
    Multiple Instances with a Priori Run-Time KnowledgeWorkflow Pattern Description: Multiple instances of a task or a subprocess are created. The number of instances depends on various run-time factors. Instances run concurrently and are synchronized at completion before the process continues.
    Workflow Pattern Description: Multiple instances of a task or a subprocess are created. The number of instances depends on various run-time factors. Instances run concurrently and are synchronized at completion before the process continues.
    BPMN: We set the attributes of the activity as follows:
    • we set the value of the LoopType attribute to “MultiInstance”,
    • the expression of the MI_Condition attribute is based on run-time factors and returns
      the actual number of required instances at run-time,
    • we set the value of the MI_FlowCondition attribute to “All”.
    • we set the value of the MI_Ordering attribute to “Parallel”
    Example: The process receives a list of all
    orders. The expression of the MI_Condition
    attribute depends on the number of orders
    in the list, which can be different for every
    process instance. For every order in the
    order list an instance of the Process Order
    subprocess is created. The subprocess
    instances are executed concurrently. After all
    the subprocess instances are completed, the
    process continues.
    checked 2

    Termination Patterns

    Implicit Termination
    Workflow Pattern Description: A process or a subprocess instance terminates when there is nothing else to be done and it is not deadlocked. The instance has completed successfully.
    BPMN: The pattern can be implemented in one of the following ways:
    • We end every path of the process or subprocess with an end event. If we use a start event we must use at least one end event.
    • An end of a path in the process is indicated by a flow object without an outgoing sequence flow. The process completes when all tokens that were generated for the instance are consumed.
      • Note: We must either conclude all paths with an end event (with an exception of compensation activities) or not use end event for the given process/subprocess.
    Example: In the example process there are two alternative paths that the process instance can take. If the order cannot be fulfilled, the customer is notified. After this the end event is reached and the process completes. If the order can be fulfilled several activities take place and ordered items are shipped. After this the process reaches an end event and completes.
    paths1
    Explicit Termination
    Workflow Pattern Description: Aprocess or subprocess terminates and the remaining work is cancelled.
    BPMN: We use a terminate end event.
    Example: In the example, the process splits into two parallel paths after order analysis.
    If additional documentation is required, the customer is notified. Even though order
    preprocessing activities already take place, if the customer does not send the required
    documentation in time, the process terminates explicitly and all the remaining activities are
    cancelled.
    Parallel Paths

    Conclusion

    BPM is essential for continuous improvement of business process efficiency and effectiveness with the overall goal to produce business results faster, cheaper, better. This Refcard has provided the overview of the BPM lifecycle, presented the BPMN notation and demonstrated the most important patterns.

    Other References and Resources

    M.B. Juric, R. Loganathan, P. Sarang, F. Jennings: SOA Approach to Integration, November 2007. OMG: Business Process Modeling Notation (BPMN), Version 1.2, January 2009.

    M.B. Juric, P. Sarang, B. Mathew: Business Process Execution Language for Web Services 2nd Edition, January 2006. H. Gaur, M. Zirn, et al.: BPEL Cookbook: Best Practices for SOAbased integration and composite applications development, July 2006.

    Wil van der Aalst, Arthur ter Hofstede, et al.: Workflow Patterns, http://www.workflowpatterns.com/.

    About The Author

    Photo of author Matjaz B Juric

    Matjaz B Juric

    Matjaz B Juric, Ph. D., is associate professor at the University of Maribor and the head of the SOA Competency Centre. He has been consultant for several large companies on the BPM/SOA projects and has worked on projects, such as SOA Maturity Model, SOA in Telcos, performance analysis and optimization of RMI-IIOP, etc. Matjaz is author of courses for the BPEL and SOA consulting company BPELmentor.com. He is also a member of the BPEL Advisory Board.

    Photo of author Ana Sasa

    Ana Sasa

    Ana Sasa has dedicated the last five years exclusively to the field of SOA. She has worked on a number of IT projects for government agencies and large companies and has specialized knowledge and experience in the technical, business and organizational aspects of BPM and SOA. She is a researcher at the University of Ljubljana.

    Recommended Book

    BPMN and BPEL

    Learn how to model business processes in an SOA-compliant approach using BPMN, translate them into BPEL and execute them on the SOA platform. A practical guide with real-world examples illustrating all key concepts. This book is for CIOs, executives, SOA project managers, business process analysts, BPM and SOA architects, who are responsible for improving the efficiency of business processes through IT, or for designing SOA. It provides a high-level coverage of business process modeling, but it also gives practical development examples on how to move from model to execution. We expect the readers to be familiar with the basics of SOA.


    Share this Refcard with
    your friends & followers...

    DZone greatly appreciates your support.


    Your download should begin immediately.
    If it doesn't, click here.

    SOA Governance

    By Todd Biske

    12,269 Downloads · Refcard 41 of 151 (see them all)

    Download
    FREE PDF


    The Essential SOA Governance Cheat Sheet

    Service-Oriented Architecture (SOA) governance is the combination of people, policies, and processes that an organization leverages to achieve the desired behavior in their SOA adoption efforts. SOA needs a solid foundation that is based on standards and includes policies, contracts, and service level agreements. SOA Governance must enable an organization to answer two fundamental questions: What are the right services to build? How do I build services the right way? This DZone Refcard provides Enterprise Architects, senior IT managers, and SOA program managers with an overview of the considerations, approaches, and technologies necessary for successful SOA governance.
    HTML Preview
    SOA Governance

    SOA Governance

    By Todd Biske

    ABOUT SOA Governance

    SOA governance is the combination of people, policies, and processes that an organization leverages to achieve the desired behavior in their SOA adoption efforts. This refcard provides Enterprise Architects, senior IT managers, and SOA program managers with an overview of the considerations, approaches, and technologies necessary for successful SOA governance.

    The hardest part of SOA adoption is the cultural change required. This can include changes not only in the technologies utilized in building solutions, but also the processes by which solutions are initially defined, funded, resourced, managed in production, and changed per the needs of the business. SOA governance must enable an organization to answer two fundamental questions. What are the right services to build? How do I build services the right way?

    Beyond these fundamental questions, SOA governance must go deeper to ensure that the desired behaviors of the adoption effort are achieved. This is especially critical, because the typical desired behaviors, such as increased agility and reduction or elimination of redundancy, will not be achieved through a single project or program, but over the successful execution of many projects, and sustained through the projects that follow. It is governance that must provide the policies that guide how those projects are defined, executed, and maintained so that goals are met.

    People

    Governance must begin with, but not end with, people. People must take the desired outcome of the SOA effort, determine what the desired behaviors are, and define the policies that comprise that behavior. These people must be viewed as authorities in the organization. Without authority, the organization will not respect the policies that have been established. At the same time, the selection of the people that will be associated with your governance effort and how they interact with the people executing the projects is the area of most risk in implementing SOA governance. From the people chosen, to the method of interaction with the project teams, to the decision making culture of the organization, the people can be the difference between effective and ineffective governance.

    Determining your Organizational approach

    When determining the people that will be part of your SOA governance effort, you must determine how those people will be organized. The challenge you will face is that an SOA effort will touch all areas of your IT organization, and perhaps even outside of IT. It is likely that you have teams in your organization today that are involved with some aspects of governance, such as Enterprise Architecture, but it is unlikely that a single team has domain over all aspects of the SOA effort. Are your architects setting policies that determine what projects get funded? Are your managers defining policies that guide how solutions get built? The SOA effort will challenge the existing organizational structure and create dependencies across teams where they did not exist before.

    Hot Tip

    The most important thing is to have your governance efforts match the culture of the organization, presuming that your organization is already effective in its decision making and policy setting processes. If it is not, then perhaps a change is in order.

    Command-and-Control Organizations

    Organizations that operate in a manner that always looks up the organization chart for policy definition and decisionmaking are typically known as command-and-control organizations. While the rigidity of the processes and recognition of the chain of command may be a turn off for many, this does not mean that the organization cannot be effective with this approach. Sometimes it is only way to get things back in line if the current processes are very chaotic and ineffective. In such an organization, having the SOA governance team show up on the organization chart either as a new entity or as a collection of dotted lines that report in to the highest levels within IT may be required to ensure that the authority of the team is recognized.

    Determining your Organizational Approach, continued

    Consensus-Driven Organizations

    At the opposite extreme are organizations whose decision making process is completely independent from the organizational structure. These consensus-driven organizations focus on bringing the right people together for the question at hand, and allow them to reach the appropriate decision. Their effectiveness is based on how quickly the people involved can reach consensus. The risk with this approach is that decisions are not made in a timely manner, because consensus is very difficult to reach. If your organization is an effective consensus-driven organization, a formal SOA governance team may not be necessary. Rather, the right people will naturally come together to establish policy because that’s how the organization operates.

    SOA Center of Excellence

    In all likelihood, your organization will fall somewhere in the middle. The commonality across both approaches is that the SOA governance effort must recognize that SOA will touch many, if not all areas of your IT organization, and will likely cross the existing organizational boundaries. By drawing people from multiple areas into the effort, your chances of missing some piece of the SOA governance puzzle are lessened. This collection of people will form your SOA Center of Excellence. You can choose to formally represent that within your organization chart, if necessary, or it can simply be another one of many “virtual teams” that a consensus-driven organization naturally leverages to ensure success.

    Center Excellence

    Roles

    Because of the domains of policies that will be necessary for your SOA efforts, there are many roles in the organization that will need to be involved with the governance effort.

    Role Description
    Enterprise
    Architect
    Responsible for establishing technical policies, expressed in the form
    of reference architectures, that define how services will be built and infrastructure leveraged.
    Solution Architect Responsible for the technical leadership and decisions on a project or program, and for the project’s compliance with the technical designtime
    policies.
    Information
    Architect
    Responsible for ensuring the consistent representation of information
    on service messages.
    Technical Lead/
    Domain Architect
    Responsible for technical policies within a smaller domain, expressed
    through domain-specific reference architectures.
    Business Analyst Responsible for the functional aspects and business analysis activities of the solution, including defining what services get built.
    Security Architect Responsible for ensuring that the technology solutions of the company take appropriate measures to protect its corporate assets and
    intellectual property.
    Platform Manager Responsible for the technology platform where services (and consumers, for internal consumers) are hosted, and the entry criteria for
    deployment.
    Service Manager Responsible for managing the relationships with service consumers>br>via service contracts, managing the service lifecycle, and ensuring that proper run-time governance is in place.
    IT Manager Responsible for managing personnel, work, and budget.

    Policies

    Establishing a center of excellence is not enough to ensure success with your SOA governance and SOA adoption efforts. While governance begins with people, it cannot end there. If all decisions must be made and/or reviewed by the center of excellence, your efforts will not scale as the COE will become a bottleneck for all of your projects.

    In order to allow your governance efforts to scale, the people must focus first on establishing policies. Policies are the standards and guidelines that enable the staff executing your SOA projects to make appropriate decisions and achieve the desired behaviors. There are three key timeframes for which policies are necessary:

    Pre-Project
    Governance
    In this timeframe, the primary concern is with determining what IT projects
    to fund and execute, which is frequently associated with the broader
    subject of IT governance. While SOA governance should not introduce new
    governance processes associated with deciding what projects to fund and
    execute, policies associated with SOA governance should be included in the
    criteria. This is the timeframe where the policies must aid in answering the
    question, “What are the right services to build?”
    Project
    Governance
    This is also referred to as design-time governance; however, the activities
    in this time frame are concerned with much more than solution design. The
    policies at this stage must aid in answering the question, “How do I build my
    services the right way?”
    Run-Time
    Governance
    Governance does not end when the projects are complete. SOA adoption
    can increase the number of moving parts in any given solution, and if
    the dependencies aren’t managed appropriately chaos can ensue. You
    need to have appropriate run-time policies to ensure the systems behave
    appropriately while in use and that the relationships between service
    consumers and service providers are managed.

    Pre-Project Governance

    There are key artifacts that can assist in defining policies at this timeframe.

    Timeframe

    Pre-Project Governance, continued

    Organization Charts Can influence how funds are distributed in an organization,
    and this can be a barrier to SOA adoption. The organization
    chart must be leveraged and modified where necessary to
    account for clear service ownership and funding approaches. if
    the organizational issues aren’t discussed at the time projects
    are defined and funded, it can severly hamper the efforts of
    the project team.
    Business Process Models,
    Business Domain/
    Capability Models and
    Application Portfolio
    All are closely related. These are analysis artifacts that should
    be used to guide the decisions on what services should
    be created. Business process models and the application
    portfolio provide excellent context on existing applications
    and processes, but each on their own runs the risk of focusing
    on application silos or process silos. By combining these
    two with a capability map against the business domains, in
    essence a heat map of business capabilities, the areas where
    shared services will provide the most value can more easily be
    identified. It is important to do this outside of the context of
    Business Process Models,
    Business Domain/Capability Models and Application Portfolio,
    continued
    any project, as within a project, there is significant pressure
    to constrain analysis efforts and avoid “analysis paralysis.” A
    good reference for this approach is the book “Enterprise SOA
    Adoption Strategies” by Steve Jones of CapGemini.
    Service Porfolio The last artifact is a catalog of services that have been built
    and are available in production, but it is much more powerful
    when it is used as a planning tool. When the organization
    has taken the time to perform business process analysis and
    business domain/capability analysis, an outcome should be
    the definition of key services that the organization needs to
    create to fully leverage SOA.

    Policies for Pre-Project Governance
    The following are questions/policies that you should consider in your pre-project governance efforts:

    Pre-project governance
    Has the proposed project identified candidate services?
    Has the proposed project mapped candidate services to the business domains as represented in the business domain/capability models?
    Has the proposed project reviewed the service portfolio against the list of candidate services?
    Has an appropriate team of project stakeholders been identified based upon candidate services?
    Has the proposed program/project been appropriately structured and scheduled to properly manage the development and integration of new and existing services?
    Have all funding details been determined based upon the services proposed and the organizations involved?
    Does the roadmap include the development of services with high potential for reuse?
    Are projects encouraged to reuse existing services, where appropriate, based upon the business domain models and business objectives?
    Are projects allowed to create redundancies, where appropriate, based upon the business domain models and business objectives?
    Have existing systems been taken into account in the definition of the proposed services?
    Is the organizational structure being reviewed on a regular basis based upon continued service analysis?
    Does the organization have a clear approach to resolving service ownership models?
    Are business processes properly leveraging services?
    Does your service portfolio properly account for any globalization impact?
    Does the service portfolio properly account for any planned areas for growth by acquisition?

    Project Governance

    In order to perform project governance, the following artifacts are recommended as sources of policy:

    Artifacts

    Service technology reference architectureHelps ensure that appropriate technologies are used for the service being developed. It should first define the appropriateservice types for the organization and then map those types to specific service technologies. Service types can include:

    Composite
    Services
    These are services that are built by combining the output of two or
    more services and aggregating the respective responses into a single
    response.
    Automated
    (Orchestrated)
    Processes
    These are services that are built by executing a fully automated
    sequence of actions as represented in a graphical process model.
    Integration
    Services
    These are services whose purpose is to enable a system that does not
    support service standards to speak with service consumers.
    Presentation
    Services
    These are services that provide information in a presentation-friendly
    format, providing information in such a way that it is easily consumed by
    user interface technologies.

    Service Technology Reference Architechture, continued

    Management
    Services
    These are services that expose management and administrative
    functionality. There are technologies, such as SNMP, JMX, and WSManagement
    that have been tailored for this purpose.
    Information
    Services
    These are services that are used to retrieve information from a variety
    of data sources, aggregating the results into a single response. These
    services differ from composite services in that they are specifically
    designed to talk only to data sources on the back end, rather than any
    arbitrary web service.
    Content
    Subscription
    Services
    These are services that provide content feeds, typically adhering to feed
    syndication standards such as RSS and ATOM.
    General Business
    Services
    This is a catch-all category for any service that doesn’t fit into any of the
    other categories.

    These service types are mapped to service technologies, defining both the service platform as well as the servicecommunication technologies. The service platformdefines where the service implementation will be hosted and executed, such as a Java Application Server or a BPEL orchestration platform, while the service communication technology defines the message formats and thecommunication technologies used to interact with the service, such as XML, SOAP, and HTTP. The mapping effort links service types to technologies.

    The service technology reference architecture must also rovide policies on how the non-functional capabilities associated with services interactions will be provided by the underlying infrastructure. This includes:

    • Security
    • Routing and load balancing
    • Transport and mediation
    • High availability and failover
    • Monitoring and management
    • Versioning and transformations

    These non-functional capabilities should be enforced through policy-driven infrastructure that is configured, rather than coded.

    Hot Tip

    These non-functional capabilities are a key aspect of SOA, because they are the foundation of run-time governance. At the same time, they must be factored into the design-time decisions, because if the development teams don’t utilize the technology appropriately, the ability to enforce run-time governance policies will disappear.

    Service security reference architecture The next artifact is the service security reference architecture. This can be included as a subset of the Service Technology Reference Architecture, or created as a standalone artifact. Regardless of the approach, there are two questions that must be answered by the reference architecture. What security policies must be enforced? What technologies are used for enforcing those policies?

    Reference Architecture

    Service Security Reference Architecture, continued

    These questions must guide the developers of services and their consumers on security policies for authentication, authorization, encryption, digital signatures, and threat prevention. This includes how identity is represented on messages, how it flows through a chain of service invocations, how and where authorization decisions are made, what type of data should be encrypted and where it should be encrypted, how and where incoming and outgoing messages are checked for potentially malicious content, and more.

    Service blueprints and frameworks
    Service blueprints are examples of common patterns that demonstrate a policy-compliant way of solving a particular problem. Their use can make the collection of policies associated with a reference architecture must less daunting to a project team. Common patterns may include integration scenarios with legacy systems, exposing services to external parties, and consuming services provided by external parties.

    Service frameworks are reusable libraries that, when used, allow implementations to be compliant with the policies of the organization, such as using the correct security credentials on service messages. Compliance is easiest when it is the path of least resistance, and making it so a developer only needs to write one or two lines of code, or even none, can make that happen.

    Standard information models and schemas
    Standard information models and schemas do not present one universal representation that everyone agrees on, because it probably doesn’t exist. Rather, they ensure consistency in the way information is represented, minimizing the number of representations. Industry verticals, such as SWIFT (financial services), HIPAA (healthcare), and ACORD (insurance), can be leveraged as starting points, and are likely required when exposing services externally.

    Policies for project governance
    The following are questions/policies that you should consider in your project governance efforts, in addition to all those that were specified in the pre-project governance if not enforced at that time:

    Policies for project governance
    Have all services been mapped to an appropriate type?
    Are the service technologies chosen for each service consistent with the type to technology mapping specified in the reference architecture?
    Does the service use the standard communication technologies specified in the reference architecture?
    Does the service interface comply with all naming conventions for URLs as specified in the reference architecture?
    Does the service interface properly reference all external schema definitions, rather than copying them locally?
    Does the service interface use the standard schema definitions properly?
    Do external facing services only expose industry standard schemas, where they exist?
    Is the service interface compliant with industry standards, such as WS-I?
    Does the service require identity on its messages?
    Are all service consumers properly specifying identity on outgoing requests?
    Have appropriate authorization policies been established for the service?
    Is the service communication infrastructure being leveraged appropriately?
    Are all internal consumers properly leveraging the standard service frameworks?
    Is all sensitive information properly encrypted according to the service security policies?

    Policies for Project Governance, continued

    Have service contracts been established between all consumers and providers?
    Are all aspects of the service contract fully specified including message schemas, versions,
    delivery schedule, points of contact, and expected usage rates?
    Have all services been thoroughly and adequately tested, with testing results available to service consumers, if required by the service contract? For internal consumers, testing results
    should always be available to help counter the natural tendency for developers to resist using things they didn’t personally write.
    Have service managers been assigned for all new services?
    Are the service boundaries identified in the solution consistent with the business domain models?
    Has the solution incorporated existing services appropriately?
    Has the solution properly published information about new services into the Service Registry/ Repository?
    Has the solution avoided creating redundant services where not appropriate according to the business domain models?

    Run-time Governance

    During this timeframe, the major concern is the correct behavior of service consumers and service providers so that the infrastructure remains operational and in a healthy state at all times. At its core, the run-time infrastructure consists of three things: infrastructure used to execute the logic associated with the service consumer, infrastructure used to execute the logic associated with the service provider, and infrastructure used to allow communication between the two. Three core principles that should be adopted:

    Service Consumer Responsible for ensuring that all messages they send are compliant with the service communication standards.
    Service Providers Responsible for ensuring that they expose endpoints that can consume messages that are compliant with the service
    communication standards.
    Service Communication
    Infrastructure
    Enforces all non-functional capabilities for all messages that are compliant with the service communication standards,
    including mediation between those standards.
    Reference Architecture

    If a consumer or provider is not capable of being compliant with the service communication standards, adapters are leveraged not in the middle (the service communications infrastructure), but rather on the endpoints where the service consumers and service providers are deployed.

    Service Contract The service contract is the collection of policies that govern the interaction between a service consumer and a service provider, much as a legal contract is used to govern the relationship between two parties. This includes the messaging schemas, such as those defined in a WSDL file for a Web Service, but must also include the policies that govern the run-time behavior, such as the expected usage by the consumer in an appropriate level of detail, as well as the expected response time from the provider when the system is

    Service Contract, continued

    behaving as expected. It is the responsibility of the run-time infrastructure to enforce the policies of the service contract.

    Policies for Run-time Governance

    The following are questions/policies that you should consider in your run-time governance efforts:

    Policies for run-time governance
    What is the normal rate of requests for a given service consumer?
    What is the expected response time for the service provider for typical requests from that service consumer?
    What actions are taken when the request rate for a given service consumer exceeds each of the agreed upon thresholds?
    What actions are taken when the response time for a given service consumer exceeds each of the agreed upon thresholds?
    Are there any time restrictions on when a particular consumer can access a service?
    For services with multiple entry points via different technologies (e.g. SOAP/HTTP, XML/HTTP, SOAP/JMS), is policy enforcement defined and consistent (if needed) for each entry point?
    Are all security policies configured and being enforced?
    Are service requests routed to the appropriate version for each consumer, or have appropriate transformations applied preserving backward compatibility?
    Are all service messages being logged appropriately per any enterprise auditing requirements?
    Are all service messages being logged and preserved for the purpose of debugging?
    Are usage metrics being properly collected?
    Are usage reports being generated and distributed appropriately?
    Are the recipients of these reports properly reviewing them and accounting for any discrepancies in behavior?
    Are all policies associated with message structure being enforced by the run-time infrastructure?
    Are non-compliant messages being logged, rejected, and reported to appropriate personnel?

    The four processes of governance

    There are four key processes that must be executed as part of your governance processes: policy definition, communication and education, enforcement, and measurement and feedback.

    Policy definition Concerned with establishing the policies that the governance team feels will result in the desired behavior if they are followed. Without policy, the rest of the organization must either guess what the correct decisions are to get to the desired outcome, or involve someone from the governance team on every single project. The first option is unlikely to lead to success, and the second option has both scalability issues as well as being prone to variation based upon the “tribal knowledge” of the particular person from the governance team involved. Defining and documenting the policies is step one toward gaining consistency in the outcome.
    Education and
    Comunication
    Just because the governance team has reached agreement and documented the policies doesn’t mean they’re going to be followed,
    or even known for that matter. A formal, planned communication effort to educate the organization on why you’re adopting SOA, the desired
    behavior you hope to achieve, and the policies that are being put in place to achieve them is required. It’s not a one time presentation to all
    of IT, but rather a series of targeted communications for the various roles in the organization, large group presentations, small team presentations,
    blogs, wikis, and appropriate surveys and followups to ensure that the communication is effective.
    Enforcement Even if your communication efforts are incredibly successful, you still need to put processes in place to ensure the policies are being
    followed. What you will find, however, is the better job you can do on communication and education, the easier your enforcement processes
    can be. If education is poor, enforcement will likely need to be more heavy-handed. Where possible, automated testing and reporting can
    certainly make the processes more efficient and cost-effective.
    Measurment and
    Feedback
    The governance group must have measurement and feedback processes to ensure that progress is being made toward the desired
    behavior. If the desired behavior is not reached, something needs to be changed, and it could easily be the policies, the processes, or the
    people involved with governance. Accountability is lost if the team puts policies and processes in place, but then does nothing to verify that all
    that effort actually paid off.
    Processes

    SOA Governance Technologies

    The role of SOA governance technology is not to be your governance, but rather support your governance processes by making them more efficient. To do this, there are two areas where technology can be part of your SOA governance effort.

    Registry/Repository

    This is the first area in support of service metadata and policy management. The registry/repository allows you to track services, service consumers, and the policies that govern their interactions. It can be the critical tool in performing service portfolio management and service lifecycle management. Through domain modeling, process analysis, and other techniques, the portfolio can be populated with planned services, updated as they are implemented, and decommissioned when all consumers disappear, with the registry/repository being the tracking point.

    In addition to service metadata, the registry/repository can also be used for policy management. Policies also have a lifecycle associated with them that must be managed. If the teams that are expected to adhere to policies aren’t aware the policies exist, your governance efforts will be sub-optimal.

    Policy Enforcement

    The second area is policy enforcement where multiple technologies can be leveraged.

    Design Time

    production.
    Technologies Description
    Service Testing Can evaluate service interfaces for compliance with enterprise policies
    (some registry/repository solutions can also do this at time of registration). A key factor
    to consider is integration with other designtime testing solutions, whether for functional testing, regression
    testing, load testing, or performance testing. These same test cases can also be leveraged at run-time for
    active monitoring of services in production.
    Service Framework Shared libraries that a developer leverages when constructing service
    consumers or service providers. These frameworks can make policy compliance automatic, and thus the path of least resistance.

    Run-Time

    At run-time, there are three types of technologies that can be leveraged for enforcement of the run-time contract between a service consumer and service provider.

    Technologies Description
    Enterprise Service Bus (ESB) enabling standards-based connectivity between consumers and
    providers, typically with stronger appeal to developers.
    XML Appliance These appliances tend to excel in XML security and threat
    protection, but may not have the flexibility of some ESB solutions.
    Service
    Management
    Platforms
    The focus here is on the instrumentation and analysis of service
    interactions, allowing alerts to be issued, requests to be throttled
    or prioritized, and reports generated. There is overlap with the ESB
    space, as many management platforms also provide gateways with
    similar capabilities, but the platforms typically have agents that can
    integrate nicely with your existing infrastructure, including ESBs,
    appliances, and hosting platforms.
    Key Pitfalls for SOA Governance Efforts
    Lack of
    Communication and
    Education
    A common approach to governance is to assign some senior people
    to a review board and mandate that all projects go through a review
    as part of the development process. The problem with this is that the
    people involved frequently don’t define the policies that teams are
    required to follow, and even if they do, they aren’t communicated. As
    a result, the project teams are left to guess what the expectations are,
    and the reviews are likely unsuccessful.

    Service Contract, continued

    No Formal
    Contracts
    The interaction between a consumer and a provider is more than just
    making sure the messages are compatible. If multiple consumers
    are involved, the run-time interactions must be carefully managed to
    ensure that one badly behaved consumer (or provider), doesn’t cause
    all consumers to experience problems.
    Lack of Service
    Ownership
    and Lifecycle
    Management
    If the desire of the organization is to reuse existing services, nothing
    will stop that effort in its tracks more than having nobody responsible
    for the service after it is put into production. If it requires changes to
    support a new consumer, and no one is there to make those changes,
    or if funding can not be allocated properly, the project teams will take
    the path of least resistance and build their own version of the service.
    Lack of Analysis
    Outside of Projects
    All services are not created equal. Some services may be used by
    many consumers, others may only have one consumer. Treating all
    services as if they will have many consumers may cause an overinvestment in many, just
    as holding off until the second consumer
    comes along can cause an under-investment. The proper way is to perform analysis outside
    of any particular project to model the
    organization and provide some context to make good decisions on the level investment
    necessary for any given service.

    About The Author

    Photo of author Todd Biske

    Todd Biske

    Todd Biske is a Senior Enterprise Architect with Monsanto in St. Louis, Missouri. He has over 15 years of experience in Information Technology, both as a corporate practitioner and as a consultant, working with companies involved with Agriculture, Atmospheric Sciences, Financial Services, Insurance, and Travel and Leisure. His interests include Service Oriented Architecture, Systems Management Technologies, Usability, and Human-Computer Interaction. He has a M.S. degree in Computer Science from the University of Illinois at Urbana-Champaign, is a member of the SOA Consortium, is a frequent conference presenter, and writes a popular blog on strategic IT topics at http://www.biske.com/blog/

    Recommended Book

    SOA Governance

    SOA Governance is the key to a successful adoption of service-oriented architecture. It is the process of establishing a desired outcome for your efforts, and then leveraging people, policies, and processes to make that outcome a reality. This includes technical policies and standards that guide your design-time activities, preproject policies that impact your project selection and funding decisions, and finally, run-time policies that impact your operational management activities. The adoption of SOA is intended to improve the efficiency and productivity of your company, and your SOA governance efforts are critical in achieving your goals in quality, consistency, predictability, change management, and interdependencies of services.


    Share this Refcard with
    your friends & followers...

    DZone greatly appreciates your support.


    Your download should begin immediately.
    If it doesn't, click here.

    SOA Patterns Refcard Available - Download Now

    SOA patterns describe common architectures, implementations, and their areas of application to help in the planning, implementation, deployment, operation, and ongoing management and maintenance of complex systems. The patterns in this refcard are classified...

    1 replies - 7515 views - 01/04/09 by Jill Tomich in Articles

    SOA Patterns

    By Eugene Ciurana

    29,122 Downloads · Refcard 38 of 151 (see them all)

    Download
    FREE PDF


    The Essential SOA Patterns Cheat Sheet

    Service-Oriented Architecture (SOA) patterns describe common architectures, implementations, and their areas of application to help in the planning, implementation, deployment, operation, and ongoing management and maintenance of complex systems. The patterns in this DZone Refcard are classified into four major groups, and listed in alphabetical order within each group. Each pattern includes a Pattern name, Icon, Summary, Problem, Solution, Application, Diagram, Results, and Examples. This Refcard also covers SOA Fundamentals, Pattern Language, Basic Service Patterns, Architectural Patterns, Compound Patterns, and more.
    HTML Preview
    SOA Patterns

    SOA Patterns

    By Eugene Ciurana

    ABOUT SOA PATTERNS

    SOA patterns describe common architectures, implementations, and their areas of application to help in the planning, implementation, deployment, operation, and ongoing management and maintenance of complex systems.

    SOA FUNDAMENTALS

    Fundamentals

    Systems are described as services independent of the underlying technology
    Services are implemented through messaging.
    A SOA involves service providers and service consumers.
    Any participating system may act as either a provider or a consumer depending on the application's workflow.
    Services and messages are stateless.
    Services and consumers are often implemented in different programming languages, execute in different run-time environments, or both.
    SOA involves the services themselves, a directory of available services in some form (service discovery), and public contracts for consumers to connect and use each service (service negotiation).

    SOA differs from client/server architecture in that services are universally available and stateless, while client/server requires tight coupling among the implementation participants.

    SOA implementation must provide consistent designs that leverage the target environment; design consistency is attained through the application of the eight SOA principles. Service must provide:

    1. Normalized service contract.
    2. Loose coupling between consumers and services, and between the services themselves.
    3. Abstraction from implementation details; the consumers only know the contract without worrying about implementation details.
    4. Ability to compose other services regardless of the complexity of the composition.
    5. Run-time environment autonomy.
    6. Statelessness.
    7. Reusability.
    8. Discoverability through meta data or public contract definitions.

    These principles guide the SOA patterns described in the rest of this refcard.

    PATTERN LANGUAGE

    Pattern Language

    Each pattern includes a Pattern name, Icon, Summary, Problem, Solution, Application, Diagram, Results, and Examples.

    The icon and diagram symbols were selected for their ease of whiteboard use and availability in most diagramming tools.

    The patterns in this guide are classified into four major groups, and listed in alphabetical order within each group. A complete example appears at the end of this guide showing how to combine various patterns to describe a system.

    BASIC SERVICE PATTERNS

    These are the building blocks of more complex patterns.

    Aggregator

    Combines individual messages to be handled as a single unit.

    Aggregator

    Problem Stateless messages will not arrive at the service endpoint in a predetermined sequence. Messages may be processed by different services at different speeds and messages will arrive at an endpoint out of order. SOA systems guarantee message delivery but not delivery order.
    Solution Define an aggregator that receives a stream of data and groups related messages as a single entity for delivery to an endpoint for further processing. Aggregators are stateful intermediate processing units but deliver atomic payloads in a stateless manner.
    Application Group messages flowing through a service bus based on payload type or common attributes for further routing and processing.
    Results Flexibility in implementation because individual service providers can process data asynchronously without concern about state or sequence, delegating this to a workflow engine or to aggregators running in the SOA infrastructure.

    Service Bus

    A communications channel for message delivery from a single inbound endpoint to one or more outbound endpoints and optional "on the fly" message processing as data flows through the bus.

    Service Bus

    Problem Applications must communicate among them, some times using different protocols and technologies. Naïve implementations rely on point-topoint or hub-and-spoke, dedicated conduits that increase complexity, implementation time, and integration difficulty due to tight coupling between components.
    Solution Provide a data- or protocol-neutral conduit with abstract entry and exit points for interconnecting applications independently of their underlying technology.
    Application Heterogeneous system integration, legacy and new system interoperability, protocol abstraction.
    Results Message-Oriented Middleware (MOM): publish/subscribe queuing and enterprise service buses.

    Dynamic Routing

    An efficient mechanism for dispatching messages to one or more destinations based on configurable, non-filtering rules applied to the message payload.

    Dynamic Routing

    Problem Routing messages through a distributed system based on filtering rules is inefficient because messages are sent to every destination’s filter and router for inspection and rules resolution, whether the message could be processed or not.
    Solution Define a message router that includes both filtering rules and knowledge about the processing destination paths so that messages are delivered only to the processing endpoints that can act upon them. Unlike filters, message routers do not modify the message content and are only concerned with message destination.
    Application Message dispatching based on application-specific data elements such as customer attributes, message type, etc.
    Results Better overall message delivery and processing performance at the cost of increased delivery system complexity since the router must implement both knowledge of the destinations and heuristic, arbitrary rules. Excellent for decoupling applications by removing routing information from discrete systems.

    Event-Driven Consumer

    A setup that delivers messages to a services provider as soon as they appear on the channel.

    Event-Driven Consumer

    Problem Messaging systems based on blocking listeners or polling use unnecessary resources or idle for no good reason if the channel is starved. The message target blocks threads that the service could otherwise use for other tasks.
    Solution Implementation of a bus-based or application-specific callback mechanism that’s invoked only if a message appears in its inbound channel. The messaging system may invoke the callback asynchronously or synchronously.
    Application Distributed systems with a varying set of consumers and service providers with varying degrees of CPU usage based on message payload; atomic transaction processing systems that require large scalability independent of the number of service consumers.
    Results Message processing is single-threaded scaling linearly with the number of dispatched messages. Threads consume messages as they become available and free up resources when done, to be reactivated when another message becomes available. Better run-time resource utilization.

    Filter

    A conduit that extracts data from a message or applies a function to it as it flows between consumers and services through a messaging channel.

    Filter

    Problem A need to implement flexible message processing between systems in a platform-independent manner and without introducing system dependencies or unnecessary coupling.
    Solution Implement conduits with a simple inbound/processing/outbound interface modeled after a function or pipe that facilitates composition of daisy-chained filters by allowing data transfers from the output of one filter to the input of the next. All filters, regardless of their internal structure, must share the same external interface to facilitate integration and recombination.
    Application Use of discrete functions on messages like encryption, data consolidation, redundancy elimination, data validation, etc. Filters split larger processing tasks into discrete, easy to manage units that can be recombined for use by multiple service providers.
    Results Filters eliminate data and dependencies by uniform defining a contract (inbound/outbound interface) that encourages reusability through composition. Filters are also interchangeable components that enable different workflow functionality without changing the filter itself.

    Router

    A general mechanism for dispatching messages to one or more destinations based on configurable rules or filters applied to message payloads.

    Router

    Problem An application must connect with one or more application endpoints without coupling itself with any of them.
    Solution Use a conduit that allows configurable delivery rules based on the message payload, data filters, or content type. Routing may be sequential (endpoints receive the payload one after another) or in parallel (all endpoints receive the payload at virtually the same time).
    Application Content delivery in service buses, message dispatching, message proxies, enterprise integration applications, and other systems where messages must be delivered to endpoints following a sequence of applying a rule set.
    Results The router abstraction is in use in all modern SOA systems in some forms, whether available in queuing or bus-based systems out of the box, or implemented in custom-made applications and message delivery systems because they provide an elegant and simple mechanism for systemindependent message delivery.

    Translator or Transformer

    A mechanism for converting a message payload from one representation to another as it flows through the messaging system.

    Translator or Transformer

    Problem Heterogeneous systems integration (legacy, in-house, and vendorprovided) may use different message representation for input or output.
    Solution Provide a system-independent mechanism for altering the message payload and metadata (envelope) prior to delivery to an application endpoint.
    Application Message translation at the application endpoint because these translations are system- or protocol-dependent, unlike filters which are generic.
    Results Translators are one of the most effective message transformation mechanisms because they allow application developers and integrators to insulate, implement, test, and maintain these system components without modifying existing application workflow or business logic.

    ARCHITECTURAL PATTERNS

    Architectural patterns reflect solutions specific to common design issues in the definition of service-oriented system implementations.

    Asynchronous Processing

    A mechanism for queuing messages between one or more endpoints to decouple processing time and resources for each stage.

    Asynchronous Processing

    Problem Synchronous processing may result in poor server performance and reduced reliability.
    Solution Consumers exchange messages with the services through a processing queue that decouples front-end (message capture) from the back-end (processing); messages arrive into the queue at a rate different from that of processing.
    Application Any application that requires independent scalability of the front- and back-end functionality such as mainframe data consolidation (back-end) of e-commerce order fulfillment (front-end, middleware).
    Results Processing queues are well-understood and scale horizontally or vertically, depending on the application requirements. Plenty of open-source and commercial implementations, and several reference implementations and APIs are available.

    Bridge

    A mechanism for connecting two or more applications over a common data path, each using a different protocol and in which messages may require processing or analysis as they flow between endpoints for routing.

    Bridge

    Problem Application endpoints may reside in different parts of the enterprise network, use different protoocls, or may require processing based on specific message attributes.
    Solution Define a bridge between applications that provides a mechanism for routing messages, filtering them, and transform them.
    Application SOA proxies between application endpoints on the cloud and application endpoints in the middleware or back-end; ESB processing.
    Results Good for extending applications by focusing development only on intermediate processing between system and using existing systems as-they-are. Bridging allows easy integration of legacy and SOA systems.

    Cross-Service Operation

    A mechanism for coordinating multiple run-time activities which together comprise a service with guaranteed completion or roll-back capabilities.

    Cross-Service Operation

    Problem Two or more services, possibly running across multiple systems, must complete successfully; if one or more fail all the services associated with it and the application response must roll-back to their previous state for maximum application integrity.
    Solution Granular services may be wrapped in another service that provides integrity checks and ensures successful completion or graceful degradation, if any, if the granular services fails.
    Application Transactional systems.
    Results May require a transaction processor (commercial, potential vendor lock-in) wrapper to collaborate with the rest of the SOA infrastructure; consumes more resources to preserve original state for each granular service in case roll-back is necessary.

    Event-Driven Dispatching

    A mechanism for routing messages to consumers in response to specific events in the SOA or triggered by specific applications running in the SOA.

    Event-Driven Dispatching

    Problem Consumers must process messages as they become available in a system but polling for such messages is inefficient.
    Solution The consumers are implemented as reentrant, blocking applications that subscribe to a coummuniations channel. The consumers remain dormant until an event or message awakens them; the SOA dispatches the message or event in response to system or application states.
    Application Publish/subscribe systems to support asynchronous processing applications.
    Results Event-driven dispatching is hard to implement in cross-service operations. This pattern is better applied to granular services, or to treat a crossservice operation as a black box by ignoring the intermediate steps involved in the operation.

    Process Aggregation

    A method of combining two or more non-sequential, inter-dependent processing steps.

    Process Aggregation

    Problem Multiple services may be required to complete an operation but not all are known at design time, the sequence may vary depending on changing business rules, and it's not necessary to successfully complete all granular processing successfully (i.e. it requires no transactional capability).
    Solution A processing service executes the granular service calls, maintains internal state, determines processing steps, and provides synchronous or asynchronous service responses to the consumers.
    Application Systems that require multiple processes running in parallel but are not transactional, or have a mix of transactional and non-transactional components.
    Results Process aggregation provides integration flexibility but it's hard to manage. Break it down into smaller application clusters (cross-service operations or aggregations) down functional lines, synchronicity requirements, or any other criteria.

    Routing and Filtering

    A formal mechanism for routing messages to application endpoints between endpoints.

    Routing and Filtering

    Problem Messages must be dispatched to various applications based on their payload, attributes, protocol, or all of these.
    Solution Provide a formal mechanism for routing messages by recursive definition of filter, one or more routers, filters, routers, and so on.
    Application Rules-based processing, workflow, event-driven dispatchers
    Results The recursive nature of the definitions simplifies management. Naïve implementaters some times define filters or routers without formalizing their order, resulting in unintentional application coupling or resource exhaustion due to excessive use of filters or routers, respectively.

    Replicator

    Messages or payloads must be replicated across multiple endpoints with identical configurations.

    Replicator

    Problem Decoupled, horizontally scalable services get stuck in a bottleneck caused by shared access to a common message pool or data source.
    Solution Message or data replication features are implemented as part of the SOA message flow so that independent applications or endpoints may consume them in parallel.
    Application Read-only data resources or messages flowing through the SOA to increase throughput.
    Results Additional cost, complexity management if replicators are allowed to proliferate unchecked. Excellent way of providing scalability when the replicators are confined to specific problem domain service paths.

    COMPOUND PATTERNS

    Compound patterns aggregate the basic patterns to define a cohesive representation of a system. Patterns are never used in isolation, nor are they a goal by themselves. A subsystem may be built around two or more patterns. This section shows how the basic patterns defined earlier in this refcard can be combined into more sophisticated system descriptions.

    Centralized Schema

    Defines a method of sharing schemas across application boundaries to avoid redundant data representation and service definition.

    Centralized Schema

    Problem Similar data sets must be processed by services or applications with different capabilities, resulting in unwieldy service contracts or data schemas.
    Solution Define rich data schemas as entities that are separate from the service contracts and from the physical manifestation of the data as it flows through the system.
    Application Any contract-first web services, regardless of implementation technology (JMS, SOAP, other) in which more than one system will transmit, transform, process, or store data.
    Results Easy to implement if the developers make a conscious decision to separate the schema from the services where it's used. A good centralized schema implementation can generate different format definitions that, although incompatible with one another all have a 1:1 mapping to the data model.

    Concurrent Contracts

    Method for allowing multiple consumers with different abstractions or implementations to simultaneously consume the same service.

    Concurrent Contracts

    Problem The service contract may not be suitable for all the services potential consumers.
    Solution Multiple contracts may exist for the same service, each with a different level of abstraction than the others in the same group, to fit corresponding service level agreements or to accommodate legacy systems.
    Application Problem domains where various consumers need must process different subsets of the same data, like a customer master or a stock tracking system.
    Results Easy to implement if it's based on a centralized schema and it uses automated transformers or rule-based systems for generating each application contract; it may become unwieldy if the application contracts are manually generated or managed instead of handled by the centralized schema or an automated catalogue.

    Decomponse Capability

    A way of designing services to reduce the impact of functional deconstruction if it becomes necessary due to bloat or evolution of business processes and workflow.

    Decomponse Capability

    Problem A service may need decomposition without altering its core functionality, including the service's contract itself.
    Solution Maintain physical separation of the data schemas from the services definition, combining them only for generating specific service implementations, so that data and services may change independently of one another. Define evolutionary service changes in terms of the existing services and basic patterns like filtering, routing, and transformations.
    Application Evolution of large, mission-critical systems which provide additional functionality as business requirements are implemented. Any application where incremental features built into a service result in bloat or performance bottlenecks.
    Results Capability decomposition almost always results in the definition of a new service topology that supports the original functionality for legacy or older consumers while providing new functionality or additional features as needed. Decomposition should be transparent to the consumers but lead to modular service design and implementation.

    Enterprise Service Bus

    A communications channel for message delivery from a single inbound endpoint to one or more outbound endpoints and provides protocol handling, message filtering, transformation, and routing, and optional "on the fly" message processing.

    Enterprise Service Bus

    Problem Applications must communicate among them, some times using different protocols and technologies. Naïve and legacy implementations rely on point-to-point, dedicated conduits that increase complexity, implementation time, and integration difficulty due to tight coupling between components.
    Solution Provide a data- or protocol-neutral conduit with abstract entry and exit points for interconnecting applications independently of their underlying technology.
    Application Enterprise integration, heterogeneous system integration, legacy and new system interoperability, protocol abstraction.
    Results The emergence of a family of products that implement this concept under the guise of Message-Oriented Middleware (MOM): publish/ subscribe queuing and enterprise service buses.

    Fault-Tolerant Service Provider

    Mechanism for deploying a service platform to achieve near-zero downtime in case one of the services providers or the platform itself have a catastrophic failure.

    Fault-Tolerant Service Provider

    Problem Mission-critical applications are the main candidates for SOA implementations and must provide appropriate fault-tolerance and recovery in case of catastrophic failure.
    Solution Provide redundant service containers and message brokers complemented by network-level load balancing and routing; ensure that application services are stateless and re-entrant when possible.
    Application Services in fast growing, high-availability environments with near-zero downtime service level agreements.
    Results Easy to implement for stateless services. This pattern may be used for providing both scalability and fault-tolerance.

    Wrapper

    Encapsulate a legacy service API inside a generic, stateless service.

    Wrapper

    Problem Legacy systems may offer limited service capabilities, or their only interface with other applications may be through file data exchanges or legacy APIs.
    Solution Wrap the interoperation mechanisms within a service façade that operates with the legacy system as if it were a legacy consumer, and exposes a normalized SOA interface to new consumers.
    Application Integration with legacy mainframe or client/server systems to expose their capability to new services and consumers.
    Results Many legacy client/server applications are tightly coupled and even a wrapper may not be enough to expose their capabilities as a service. Extensive rewrites may be required or a service may offer only readonly capabilities. If reimplementation is necessary, then implement as a stateless service and draw a migration plan to phase out the existing legacy service or system.

    About The Author

    Photo of author Eugene Ciurana

    Eugene Ciurana

    Eugene Ciurana is an open-source evangelist who specializes in the design and implementation of mission-critical, high-availability large scale systems. As Director of Systems Infrastructure for LeapFrog Enterprises, he and his team designed and built a 100% SOA-based system that enables millions of Internet-ready educational hand held products and services. As chief liaison between Walmart.com Global and the ISD Technology Council, he led the official adoption of Linux and other open-source technologies at Walmart Stores Information Systems Division.

    Publications

    • Developing with the Google App Engine
    • Best Of Breed: Building High Quality Systems, Within Budget, On Time, and Without Nonsense
    • The Tesla Testament: A Thriller

    Website

    http://eugeneciurana.com

    Recommended Book

    Book cover

    SOA Design Patterns is the de facto catalog of design patterns for SOA and service-orientation. The 85 patterns in this full-color book provide the most successful and proven design techniques to overcoming the most common and critical problems to achieving modern-day SOA.


    Share this Refcard with
    your friends & followers...

    DZone greatly appreciates your support.


    Your download should begin immediately.
    If it doesn't, click here.