Open Source

  • submit to reddit

Getting Started with BIRT

By Virgil Dodson

12,393 Downloads · Refcard 49 of 151 (see them all)

Download
FREE PDF


The Essential BIRT Cheat Sheet

Eclipse Business Intelligence and Reporting Tools (BIRT) is an open source, Eclipse-based reporting system that integrates with your Java/J2EE application to produce compelling reports. BIRT is the only top-level Eclipse project focused on business intelligence.This DZone Refcard provides an overview of the BIRT components focusing on a few key capabilities of the BIRT Designer, BIRT Runtime APIs, and BIRT Web Viewer. This Refcard should be interesting to report designers as well as developers or architects involved in integrating BIRT reports into applications.
HTML Preview
BIRT

Getting Started with BIRT

By Virgil Dodson

What Is Birt?

Eclipse Business Intelligence and Reporting Tools (BIRT) is an open source, Eclipse-based reporting system that integrates with your Java/J2EE application to produce compelling reports. BIRT is the only top-level Eclipse project focused on business intelligence. BIRT provides core reporting features such as report layout, data access and scripting. This Refcard provides an overview of the BIRT components focusing on a few key capabilities of the BIRT Designer, BIRT Runtime APIs, and BIRT Web Viewer. This Refcard should be interesting to report designers as well as developers or architects involved in integrating BIRT reports into applications.

Design and Runtime components

BIRT has two main components: a report designer based on Eclipse, and a runtime component that you can add to your application. The charting engine within BIRT can also be used by itself, allowing you to add charts to your application.

Birt Components

Getting birt

Open Source BIRT can be downloaded from http://download. eclipse.org/birt/downloads/ or http://www.birt-exchange. com. There are several different packages containing BIRT depending on your needs.

BIRT All-In-One
Download
The fastest way to get started designing BIRT reports on Windows.
Includes everything you need to start designing BIRT Reports,
including the full Eclipse SDK.
BIRT Framework This download allows you to add the BIRT plug-in to your existing
Eclipse environment. (Make sure you check the dependencies and
update those too.)
RCP Designer Simple to use rich client version of the BIRT Report Designer
dedicated to creating reports without the rest of the Eclipse
development environment.
BIRT Runtime Deployment components of the BIRT project including a command
line example, API examples, and example web viewer.
BIRT Web Tools Integration Contains the plug-ins required to use the BIRT Web Project Wizard
and the BIRT Viewer JSP tag library.

Hot Tip

You can also get BIRT into your existing Eclipse environment through the Eclipse Update Manager. Be sure to also select the Data Tools Project when using this approach.

BIRT report designers

BirtReport

The BIRT report designers are easy-to-use, visual report development tools that meet a comprehensive range of reporting requirements. The report designers include taskspecific editors, builders, and wizards that make it easy to create reports that can be integrated into web applications. All BIRT report designers support:

  • Component-based model for reuse
  • Ease of use features
  • Support for a wide range of reports, layouts and formatting
  • Programmatic control
  • Data access across multiple data sources

BIRT FILE TYPES

Design File
(*.rptdesign)
An XML file that contains the data connection infromation, report layout and instructions. Created when making a report in the BIRT Designer.
Template File
(*.rpttemplate)
Ensures all reports you create start with some common elements such as a company header or predefined syles. The starting point for a BIRT report.
Library File
(*.rtplibrary)
Stores commonly used report elements, such as a company logo, so they are managed in one place for all reports.
Report Document
(*.rtpdocument)
The completed report including layout instructions and data. Can be transformed into final report output, such as HTML, PDF, and XLS.

BIRT Data Sources

BIRT supports a variety of data sources and can be extended to support any data to which you have access. In addition to the list below, BIRT also ships with a connection to the ClassicModels sample database and can be easily extended to connect to your custom data source. BIRT also includes a Joint Data Set which allows you to join data across data sources.

Flat File Data Source Supports tab, comma, semicolon, and pipe delimited data
JDBC Data Source Supports connections to relational databases
Scripted Data Source Allows you to communicate with Java objects or to any data you can get from you application.
Web Services Data
Source
Supports connections to a web service. A wizard helps you point at a service through a WSDL and select the data
XML Data Source Supports data from XML

Palette of report items

label Use to include static (or localized) text within a report. Typically for report titles, column headers or any other report text.
text Use to include richly formatted text to your report, including the ability to integrate HTML formatting with your dynamic data.
DynamicText Use to integrate your static text with dynamic or conditional data.
data Use to include data from your connection in the report.
image Use to include images from various embedded sources or dynamic locations.
grid Use to define the layout of a report. Can be nested within other grids to support complex layouts.
list Use to display Data elements from your data source that repeat and creates a new report row for each daata set row. Can contain multiple levels of grouping.
table Use to display repeating data elements within your report and has support for multiple columns and multiple levels of grouping.
chart Use to add rich, interactive charting to your BIRT report.
Cross Tab Use to display grouped and dynamic data by both the row and column level.
aggregation Use to build totals for tables and groups. Includes over 30 built-in functions like COUNT, SUM, MAX, MIN, AVE, RUNNINGSUM, COUNTDISTINCT, and RANK.

Chart types

Bar Chart1 Side-by-Side Bar Charts show bars from each series
one beside the other. These bars are arranged so that
they each have the same width. The width of the bars
depends on the number of series being plotted.
Stacked Bar Charts show bars stacked one above the
other. The positive and negative values are stacked
separately above and below the origin.
Percent Stacked Bar Charts show bars stacked one
over the other in such a way that the total height of the
stacked bar (from its lowest to its highest) is 100%
Line Chart2 Overlay Line Charts show lines from each series
independent of the others. The lines are shown joining
the values for the series.
Stacked Line Charts show lines stacked one above
the other. The positive and negative values are stacked
separately above and below the origin.
Percent Stacked Line Charts show lines stacked one
over the other in such a way that the total height of the
stacked lines (from the lowest point to the highest in
each unit) is 100%
Area Chart3 Overlay Area Charts show areas from each series
independent of the others. The areas are shown
joining the values for the series.
Stacked Area Charts show areas stacked one above
the other. The positive and negative values are stacked
separately above and below the origin.
Percent Stacked Area Charts show areas stacked one
over the other in such a way that the total height of the
stacked areas (from the lowest point to the highest in
each unit) is 100%
Pie Chart4 Pie Charts show values as slices of a pie. The size of
each slice is proportional to the value it represents. Pie
charts for multiple series are plotted as multiple pies,
one for each series.
Meter Chart5 Standard Meter Charts contain a single dial with
region(s). The background of the dial can be divided
into regions with different colors.
Superimposed Meter Charts contain multiple dials
with identical regions. The dials overlap together so
that it can represent multiple needles within a single
region.
Scatter Chart6 Scatter Charts show the values arranged on the plot
using the category and value data as coordinates. Each
data value is indicated by a marker.
Stock Chart7 A Candlestick Stock Chart contains a box with lines
extending up and down from the ends. The upper and
lower edges of the box are the stock open and close
values. The upper and lower points of the line are the
high and low values.
A Bar-Stick Stock Chart contains a vertical line with
two horizontal lines sticking to it. The upper and lower
points of the vertical line are the stock open and close
values. The two horizontal lines are the high and low
values.
Bubble Chart8 Bubble Charts show the values arranged on the plot
using the category and value data as coordinates. Each
data value is indicated by a marker.
Difference Chart9 Difference Charts use two fills to represent the
positive and negative areas
Gantt Chart10 Standard Gantt Charts contain a connection line with
start and end markers.
Tube Chart1 Side-by-Side Tube Charts show tubes from each
series one beside the other. These tubes are arranged
so that they each have the same width. The width of
the tubes depends on the number of series being
plotted.
Stacked Tube Charts show tubes stacked one above
the other. The positive and negative values are stacked
separately above and below the origin.
Percent Stacked Tube Charts show tubes stacked
one over the other in such a way that the total height
of the stacked tube (from its lowest point to its highest)
is 100%
Cone Chart11 Side-by-Side Cone Charts show cones from each
series one beside the other. These cones are arranged
so that they each have the same width. The width of
the cones depends on the number of series being
plotted.
Stacked Cone Charts show cones stacked one above
the other. The positive and negative values are stacked
separately above and below the origin.
Percent Stacked Cone Charts show cones stacked
one over the other in such a way that the total height
of the stacked cone (from its lowest point to its highest)
is 100%
Pyramid Chart12 Side-by-Side Pyramid Charts show pyramids from
each series one beside the other. These pyramids are
arranged so that they each have the same width. The
width of the pyramids depends on the number of
series being plotted.
Stacked Pyramid Charts show pyramids stacked one
above the other. The positive and negative values are
stacked separately above and below the origin.
Percent Stacked Pyramid Charts show pyramids
stacked one over the other in such a way that the total
height of the stacked pyramid (from its lowest point to
its highest) is 100%

Hot Tip

Creating your first report:
  • Create a new Report Project from the category of Business Intelligence of Reporting Tools. Change to the Report Design perspective.
  • File -> New ->Report. Select the template called “My First Report” which launches a cheat sheet containing a step-by-step tutorial assisting you with connecting to data sources, creating data sets, and laying out your report.

Localization

BIRT supports internationalization of report data including support for bidirectional text. BIRT also supports the localization of static report elements within a report allowing you to replace report labels, table headers, and chart titles with localized text. BIRT uses resources files with name/value pairs and a *.properties file extension. For example, a file called MyLocalizedText_de.properties can include a line that says “welcomeMessage=Willkommen”. To use these files within a BIRT report:

Assign Resource File to entire report Report -> Properties -> Resources -> Resource File
Assign individual keys to a label Label -> Properties -> Localization -> Text key

Styles

Reports designed with the BIRT report designer can be richly formatted with styles that match your existing web application

Built-in Styles Built-in styles can be shared in a report library for managing style across multiple reports.
CSS Style Sheet BIRT can import CSS files at design time or reference existing CSS files at run time.

Report Parameters

A BIRT report can contain parameters that effect the report. Parameters can be supplied by the user or passed in programmatically from the application. Parameters can be bound to a data set query effectively filtering the report data. Parameters can also be used in expressions and scripting. For example, a parameter can be used with a visibility expression to hide a column or entire table. Available report parameter values can be supplied from a static list, dynamically created from a data set, or even cascading dynamic parameters. For example, selecting a Country presents the available States, and selecting a State presents the available Cities. Related parameters can be grouped for easier user navigation.

Parameter collection from the user can be in several forms:

Text Box Empty text area where the user can type the values desired
Combo/List Box A list of values is presented to the user. This list can be provided as a
static list or dynamically generated based on a data set query. multiple
values can be accepted.
Radio Button Provides boolean Yes/No, True/False, On/Off of parameters
Custom Parameters can be passed in programmatically so you can creae your
own web front end to collect the parameters from the user.

Customization wi th expressions, scripting and Events

BIRT includes out-of-the-box functionality that is available through drag-and-drop or by setting some properties, but also supports more advanced customizations through expressions, scripting, and events. The expression builder in BIRT allows you to do conditional report processing just about anywhere you need to instead of hard coding values. For example, the expression below will display the shipped date for orders that have already shipped, otherwise, it will display the order date.


if (dataSetRow[“STATUS”] == “Shipped”) {
  dataSetRow[“SHIPPEDDATE”];
} else {
  dataSetRow[“ORDERDATE”];
}

Scripting of a BIRT report can be done in either JavaScript or Java depending on your skill set and needs. Scripting allows you to circumvent the traditional event processing of the BIRT report. You can add scripting to report object, data source, and data element event types. Each of these event types has several events that you can overwrite.

For example, you can use scripting to navigate your Java objects and add them to a BIRT Data Set.


favoritesClass = new Packages.SimpleClass();
favorites = favoritesClass.readData();
…
var favrow = favorites.get(currentrow);
var Customer = favrow[0];
var Favorite = favrow[1];
var Color = favrow[2];
row[“Customer”]=Customer;
row[“Favorite”]=Favorite;
row[“Color”]=Color;

Use scripting to change bar colors on a chart based on plotted data.


if (dph.getOrthogonalValue() < 1000) {
  fill.set(255,0,0); //red
} else if (dph.getOrthogonalValue() < 5000) {
  fill.set(255,255,0); //yellow
} else {
  fill.set(0,255,0); //green
}

Use scripting to add or drop a report table based on a user parameter.


if (params[“showOrders”] == false){
  reportContext.getReportRunnable().designHandle.getDesignHandle()
    .findElement(“table1”).drop();
}

Or use scripting to include dynamic images that are based on the report data.


if (row[“CREDITLIMIT”] <= 0) {
“down.jpg”
} else {
“up.jpg”
}

Report deployment options

Once you create your report designs, there are several different ways to generate the report output. Obviously, you can run these reports directly from the BIRT Designer, but you can also run BIRT reports from the command line, generate BIRT reports from you Java application using the BIRT APIs, integrate and customize the example web viewer, or deploy your reports with third-party components and report servers.

APIs

BIRT supplies several APIs and an example J2EE application for generating and viewing reports. The major APIs are the Design Engine API(DE API), Report Engine API(RE API) and the Chart Engine API (CE API). In addition to the APIs, BIRT supports scripting using either Java or JavaScript within report designs.

Design Engine API(DE API) Use the Design Engine API (DE API) to create a custom report designer tool, or to explore or modify BIRT report designs. The BIRT Designer uses this API. You can call this API within a BIRT script to modify the currently running report design.
Report Engine API(RE API) Use the Report Engine API to run BIRT reports directly from Java code or to create a custom web application front end for BIRT.
Chart Engine API(CE API) Use the Chart Engine API to create and render charts apart from BIRT.

Birt report engine tasks

There are several tasks supplied by the Report Engine API that can be used to generate report output. A few key tasks are listed below.

IRunAndRenderTask Use this task to run a report and create the output directly to one of the supported output formats. This task does not create a report document.
IRunTask Use this task to run a report and generate a report document, which is saved to disk.
IGetParameterDefinitionTask Use this task to obtain information about parameters and their default values.
IDataExtractionTask Use this task to extract data from a report document. The BIRT viewer uses this class to extract report data into CSV format.

World ’s Simplest birt engine example


static void executeReport() throws EngineException
{
  IReportEngine engine=null;
  EngineConfig config = null;
try{
  // start up Platform
  config = new EngineConfig( );
  config.setBIRTHome(“C:\\BIRT_231\\birt-runtime-2_3_1\\
  ReportEngine”);
  config.setLogConfig(“C:\\BIRT_231\\logs”, java.util.logging.Level.
  FINEST);
  Platform.startup( config );
  // create new Report Engine
  IReportEngineFactory factory = (IReportEngineFactory) Platform
  .createFactoryObject( IReportEngineFactory.EXTENSION_REPORT_
  ENGINE_FACTORY );
  engine = factory.createReportEngine( config );
  // open the report design
  IReportRunnable design = null;
  design = engine.openReportDesign(“C:\\BIRT_231\\designs\\param
  .rptdesign”);
  // create RunandRender Task
  IRunAndRenderTask task = engine.createRunAndRenderTask(design);
  // pass necessary parameters
  task.setParameterValue(“ordParam”, (new Integer(10101)));
  task.validateParameters();
  // set render options including output type
  PDFRenderOption options = new PDFRenderOption();
  options.setOutputFileName(“my_report.pdf”);
  options.setOutputFormat(“pdf”);
  task.setRenderOption(options);
  // run task
  task.run();
  task.close();
  engine.destroy();
}catch( Exception ex){
	ex.printStackTrace();
}
finally
{
  Platform.shutdown( );
}


Hot Tip

If you are deploying the BIRT Engine within an RCP application you should NOT set the BIRT Home variable or execute the Platform.startup() method.

WEb Viewer

Web Viewer

The BIRT WebViewer is an example application that illustrates generating and rendering BIRT report output in a web application. This viewer demonstrates report pagination, an integrated table of contents, report export to several formats, and printing to local and server side printers.

The BIRT Web Viewer can be used in a variety of ways:

Stand-alone Use as a pre-built web application for running and viewing reports.
Modify Viewer Source Use as a starter web application that you can customize to your needs.
RCP Application Use as a plug-in for your existing RCP application.
Integrated with existing web application The viewer can be integrated with URLs or BIRT JSP tag library.

The BIRT Web Viewer consists of two main Servlets, the ViewerServlet and the BirtEngineServlet. These Servlets handle three mappings: (/frameset, /run, and /preview).

/frameset Renders the report in the full AJAX viewer, complete with toolbar, navigation bar
and table of contents features. This mapping also generates an intermediate
report document from the report design file to support the AJAX based features.
For example.
http://localhost:8080viewer/frameset?_report=myreport .rptdesign&parm1=value
/run Runs and renders the report but does not create a report document. This
mapping does not supply HTML pagination, TOC or toolbar features, but does
use the AJAX framework to collect parameters, support report cancelling and
retrieve the report output in HTML format. For example.
http://localhost:8080/viewer/run?_report=myreport.rptdesign&parm1=value)
/preview Runs and renders the report but does not generate a report document, although
an existing report document can be used; in this case, just the render operation
occurs. The output from the run and render operation is sent directly to the
browser. For example
http://localhost:8080/viewer/prevew?_report=myreport.rptdesign&parm1=value)

Viewer URL Parameters

Below are a few of the key URL parameters available for the viewer. These parameters can be used along with the Servlet mappings, such as, run, frameset, and preview, listed in the Web Viewer section.

Attribute Description
__id Unique identifier for the viewer.
__title Sets the report file.
__showtitle Determines if the report title is shown in the frameset viewer. Defaults to true. Valid values are true and false.
__toolbar Determines if the report toolbar is shown in the frameset viewer. Defaults to true. Valid values are true and false.
__navigationbar Determines if the navigation bar is shown in the framset viewer. Defaults to true. Valid values are true and false.
__parameterpage Determines if the parameter page is displayed. By default, the frameset, run, and preview mappings automatically determine if the parameter page is required. This setting overrides this behavior. Valid values are true and false.
__report Sets the name of the report design to process. This setting can be absolute path or relative to the working folder.
__document Sets the name for the rptdocument. The document is created when the report engine separates run and render tasks, and is used to support features like table of contents and pagination. This setting can be an absolute path or relative to the working folder.
__format Specifies the desired output format, such as pdf, html, doc, ppt, or xls.
__Locale Specifies the locale for the specific operation. Note that this setting overrides the default locale.
__page Specifies page to render.
__pagerange Specifies page range to render such as, 1-4, 7.
__bookmark Specifies a bookmark in the report to load. The viewer automatically loads the appropriate page.

Viewer Web.xml settings

The BIRT Web Viewer has several configuration options. These settings can be configured by modifying the web.xml file located in the WebViewerExample/WEB-INF folder. Below are a few of the key settings available for the viewer.

Attribute Description
BIRT_VIEWER
_LOCALE
This setting sets the default locale for the Web Viewer.
BIRT_VIEWER
_WORKING
_FOLDER
This is the default location for report designs. If the report design specified
in a URL parameter is relative, this path is pre-pended to the report name.
BIRT_VIEWER
_DOCUMENT
_FOLDER
If the __document parameter is not used, a report document is generated
in this location. If this setting is left blank, the default value, webapp/
documents, is used. If the__document URL parameter is used and the value
is relative, the report document is created in the working folder.
BIRT_VIEWER
_IMAGE_DIR
Specifies the default location to store temporary images generated by the
report engine. If this setting is left blank, the default location of webapp/
report/images is used.
BIRT_VIEWER
_LOG_DIR
Specifies the default location to store report engine log files. If this setting
is left blank, the default location of webapp/logs is used.
BIRT_VIEWER
_LOG_LEVEL
Sets the report engine log level. Valid values are:
OFF, SEVERE, WARNING, INFO, CONFIG, FINE, FINER, and FINEST.
BIRT_VIEWER
_SCRIPTLIB_DIR
Specifies the default location to place JAR files used by the script engine
or JARs containing Java event handlers. These JARs are appended to the
classpath. If this setting is left blank the default value of webapp/scriptlib
will be used.
BIRT_
RESOURCE_
PATH
This setting specifies the resource path used by report engine. The
resource path is used to search for libraries, images, and properties files
used by a report. If this setting is left blank, resources are searched for in
BIRT_VIEWER
_MAX_ROWS
Specifies the maximum number of rows to retrieve from a dataset
BIRT_VIEWER
_PRINT
_SERVERSIDE
This setting specifies whether server side printing is supported. If
set to OFF the toolbar icon used for server side printing is removed
automatically. Valid values are ON and OFF.

Viewer JSP Tag library

The BIRT Web Viewer includes a set of tags to make it easy to integrate BIRT reports into browser pages. These tags are available from the BIRT Web Tools Integration download. Below are a few the key JSP tags and a description of their usage.

Tag Description
viewer Displays the complete Viewer inside an IFRAME. This tag allows you to use frameset and run Servlet mappings. The AJAX Framework is used.
report Displays the report inside an IFRAME or DIV tag without the Viewer. This tag allows you to use preview mapping and does not create an rptdocument. The AJAX Framework is not used.
param Used to set parameter values when using the viewer or report tags. This tag must be nested within the viewer or report tag.
value Used to specify multiple values for a given param tag.
parameterPage Used to launch the BIRT parameter dialog or to create a customized parameter entry page. This tag can be used with the frameset, run, or preview mappings to launch the viewer after the parameters are entered.
paramDef Used within a parameterPage tag to retrieve pre-generated HTML for specific parameter control types such as radio, checkbox, dynamic or cascaded parameters

Simple viewer jsp tag example


<%@ taglib uri=”/birt.tld” prefix=”birt” %>
…
<birt:viewer
id=”birtViewer” pattern=”preview”
reportDesign=”TopNPercent.rptdesign”
height=”600” width=”800”
format=”html”
title=”My Viewer Tag”
isHostPage=”false”
showTitle=”true” showToolBar=”true”
showNavigationBar=”true”
showParameterPage=”true”>
</birt:viewer> 

BIRT REPORT OUTPUT FORMATS

In addition to delivering paginated report content to a web browser, BIRT also supports several other output formats. These formats listed below are support by both the Report Engine API as well as the BIRT Web Viewer.

Paginated web output An example web viewer is included with BIRT allowing for on demand paginated web output.
DOC Microsoft Word Document.
HTML Suitable for creating HTML pages of report data deployable to any server.
PDF Adobe PDF output suitable for emailing or printing.
Postscript Output can be directed to a printer that supports postscript.
PPT Powerpoint output.
XLS Excel file output.

Birt extension points

The APIs in BIRT define extension points that let the developer add custom functionality to the BIRT framework. These extensions can be in the form of custom data sources, report items, chart types, output formats, and functions. Once implemented, these custom extensions will show along with the built-in types. For example, you can create a custom report item, like a rotated text label, that will show up in the BIRT Palette along with the existing items.

Data Sources BIRT supports the Open Data Access (ODA) architecture, which means it can be extended to support custom data sources.
Functions BIRT allows you to create custom functions that extend those available in BIRT Expressions.
Report Items Report Items can be extended, allowing you to create your own custom report item.
Chart Types Additional chart types can be added to BIRT as plug-ins.
Output Emitters BIRT can be extended to include your own custom output type. For example, a simple CSV emitter exists and can be added to BIRT.

Additional Birt resources

Eclipse BIRT Project Site http://www.eclipse.org/birt
BIRT Exchange Community Site http://www.birt-exchange.com
Submitting/Searching BIRT Bugs http://bugs.eclipse.org/bugs/enter_bug.cgi?product=BIRT
Online BIRT Documentation http://www.birt-exchange.com/modules/documentation/

About The Author

Photo of author Virgil Dodson

Virgil Dodson

Virgil Dodson is a Developer Evangelist at Actuate Corporation and blogger/forum moderator at the BIRT Exchange community site. Virgil has over 13 years experience as a software developer. For the past 6 years he has helped Java developers get started with Actuate’s embedded reporting products. He holds a Bachelor of Science degree in Computer Information Systems from DeVry.

Recommended Book

Birt

Topics Discussed Include: Installing and deploying BIRT Deploying a BIRT report to an application server Understanding BIRT architecture Scripting in a BIRT report design Integrating BIRT functionality in applications Working with the BIRT extension framework


Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

The Top Twelve Integration Patterns for Apache Camel

By Claus Ibsen

8,194 Downloads · Refcard 47 of 151 (see them all)

Download
FREE PDF


The Essential Apache Camel Cheat Sheet

Enterprise Integration Patterns (EIP) have become the standard way to describe, document and implement complex integration problems. Apache Camel is an open-source project for implementing the EIP simply in a few lines of Java code or XML configuration. This DZone Refcard will guide you through the most common Enterprise Integration Patterns and give you examples of how to implement them either in Java code or using Spring XML. While it is targeted toward software developers and enterprise architects, anyone in the integration space can benefit from this Refcard.
HTML Preview
Enterprise Integration Patterns with Apache Camel

Enterprise Integration Patterns: with Apache Camel

By Claus Ibsen

About Enterprise Integration Patterns

Integration is a hard problem. To help deal with the complexity of integration problems the Enterprise Integration Patterns (EIP) have become the standard way to describe, document and implement complex integration problems. Hohpe & Woolf's book the Enterprise Integration Patterns has become the bible in the integration space - essential reading for any integration professional.

Apache Camel is an open source project for implementing the EIP easily in a few lines of Java code or Spring XML configuration. This reference card, the first in a two card series, guides you through the most common Enterprise Integration Patterns and gives you examples of how to implement them either in Java code or using Spring XML. This Refcard is targeted for software developers and enterprise architects, but anyone in the integration space can benefit as well.

About Apache Camel

Apache Camel is a powerful open source integration platform based on Enterprise Integration Patterns (EIP) with powerful Bean Integration. Camel lets you implementing EIP routing using Camels intuitive Domain Specific Language (DSL) based on Java (aka fluent builder) or XML. Camel uses URI for endpoint resolution so its very easy to work with any kind of transport such as HTTP, REST, JMS, web service, File, FTP, TCP, Mail, JBI, Bean (POJO) and many others. Camel also provides Data Formats for various popular formats such as: CSV, EDI, FIX, HL7, JAXB, Json, Xstream. Camel is an integration API that can be embedded in any server of choice such as: J2EE Server, ActiveMQ, Tomcat, OSGi, or as standalone. Camels Bean Integration let you define loose coupling allowing you to fully separate your business logic from the integration logic. Camel is based on a modular architecture allowing you to plugin your own component or data format, so they seamlessly blend in with existing modules. Camel provides a test kit for unit and integration testing with strong mock and assertion capabilities.

Essential Patterns

This group consists of the most essential patterns that anyone working with integration must know.

Pipes and Filters

Diagram How can we perform complex processing on a message while maintaining independence and flexibility?
Pipes and Filters
Problem A single event often triggers a sequence of processing steps
Solution Use Pipes and Filters to divide a larger processing steps (filters) that are connected by channels (pipes)
Camel Camel supports Pipes and Filters using the pipeline node.
Java DSL

from("jms:queue:order:in").pipeline("direct:transformOrder", "direct:validateOrder", "jms:queue:order:process");

Where jms represents the JMS component used for consuming JMS messages on the JMS broker. Direct is used for combining endpoints in a synchronous fashion, allow you to divide routes into sub routes and/or reuse common routes.

Tip: Pipeline is the default mode of operation when you specify multiple outputs, so it can be omitted and replaced with the more common node:


from("jms:queue:order:in").to("direct:transformOrder",
"direct:validateOrder", "jms:queue:order:process");

TIP: You can also separate each step as individual to nodes:


from("jms:queue:order:in")
	.to("direct:transformOrder")
	.to("direct:validateOrder")
	.to("jms:queue:order:process");

Spring DSL

<route>
	<from uri="jms:queue:order:in"/>
	<pipeline>
		<to uri="direct:transformOrder"/>
		<to uri="direct:validateOrder"/>
		<to uri="jms:queue:order:process"/>
	</pipeline>
</route>
<route>
	<from uri="jms:queue:order:in"/>
	<to uri="direct:transformOrder"/>
	<to uri="direct:validateOrder"/>
	<to uri="jms:queue:order:process"/>
</route>

Message Router

Diagram How can you deouple indevidual processing steps so that messages can be passed to different filters depending on a set of conditions?
Message Router
Problem Pipes and Filters route each message in the same processing steps. How can we route messages differently?
Solution Filter using predicates to choose the right output destination.
Camel Camel supports Message Router using the choice node. For more details see the Content Based router pattern.

Content-Based Router

Diagram How do we handle a situation where the implementation of a single logical function (e.g., inventory check) is spread across multiple physical systems?
Content-Based Router
Problem How do we ensure a Message is sent to the correct recipient based on information from its content?
Solution Use a Content-Based Router to route each message to the correct recipient based on the message content.
Camel Camel has extensive support for Content-Based Routing. Camel supports content based routing based on choice, filter, or any other expression.
Java DSL

Choice


from("jms:queue:order")
.choice()
.when(header("type").in("widget","wiggy"))
.to("jms:queue:order:widget")
.when(header("type").isEqualTo("gadget"))
.to("jms:queue:order:gadget")
.otherwise().to("jms:queue:order:misc")
.end();

TIP: In the route above end() can be omitted as its the last node and we do not route the message to a new destination after the choice.

TIP: You can continue routing after the choice ends.

Spring DSL

Choice


<route>
	<from uri="jms:queue:order"/>
	<choice>
		<when>
			<simple>${header.type} in 'widget,wiggy'</simple>
			<to uri="jms:queue:order:widget"/>
		</when>
		<when>
			<simple>${header.type} == 'gadget'</simple>
			<to uri="jms:queue:order:gadget"/>
		</when>
		<otherwise>
			<to uri="jms:queue:order:misc"/>
		</otherwise>
	</choice>
</route>

TIP: In Spring DSL you cannot invoke code, as opposed to the Java DSL that is 100% Java. To express the predicates for the choices we need to use a language. We will use simple language that uses a simple expression parser that supports a limited set of operators. You can use any of the more powerful languages supported in Camel such as: JavaScript, Groovy, Unified EL and many others.

TIP: You can also use a method call to invoke a method on a bean to evaluate the predicate. Lets try that:


<when>
	<method bean="myBean" method="isGadget"/>
	...
</when>

<bean id="myBean" class="com.mycomapany.MyBean"/>
	
public boolean isGadget(@Header(name = "type") String type) {
	return type.equals("Gadget");
}

Notice how we use Bean Parameter Binding to instruct Camel to invoke this method and pass in the type header as the String parameter. This allows your code to be fully decoupled from any Camel API so its easy to read, write and unit test.

Message Translator

Diagram How can systems using different data formats communicate with each other using messaging?
Message Translator
Problem Each application uses its own data format, so we need to translate the message into the data format the application supports.
Solution Use a special filter, a messae translator, between filters or applications to translate one data format into another.
Camel Camel supports the message translator using the processor, bean or transform nodes. TIP: Camel routes the message as a chain of processor nodes.
Java DSL

Processor


public class OrderTransformProcessor
		implements Processor {
	public void process(Exchange exchange)
			throws Exception {
		// do message translation here
	}
}
from("direct:transformOrder")
	.process(new OrderTransformProcessor());

Bean

Instead of the processor we can use Bean (POJO). An advantage of using a Bean over Processor is the fact that we do not have to implement or use any Camel specific interfaces or types. This allows you to fully decouple your beans from Camel.


public class OrderTransformerBean {
	public StringtransformOrder(String body) {
		// do message translation here
	}
}
Object transformer = new OrderTransformerBean();
from("direct:transformOrder").bean(transformer);

TIP: Camel can create an instance of the bean automatically; you can just refer to the class type.


from("direct:transformOrder")
	.bean(OrderTransformerBean.class);

TIP: Camel will try to figure out which method to invoke on the bean in case there are multiple methods. In case of ambiguity you can specify which methods to invoke by the method parameter:


from("direct:transformOrder")
	.bean(OrderTransformerBean.class, "transformOrder");

Transform

Transform is a particular processor allowing you to set a response to be returned to the original caller. We use transform to return a constant ACK response to the TCP listener after we have copied the message to the JMS queue. Notice we use a constant to build an "ACK" string as response.


from("mina:tcp://localhost:8888?textline=true")
	.to("jms:queue:order:in")
	.transform(constant("ACK"));

Spring DSL

Processor


<route>
	<from uri="direct:transformOrder"/>
	<process ref="transformer"/>
</route>

<bean id="transformer" class="com.mycompany.
OrderTransformProcessor"/>

In Spring DSL Camel will look up the processor or POJO/Bean in the registry based on the id of the bean.

Bean


<route>
<from uri="direct:transformOrder"/>
<bean ref="transformer"/>
</route>
<bean id="tramsformer"
class="com.mycompany.OrderTransformBean"/>

Transform


<route>
<from uri="mina:tcp://localhost:8888?textline=true"/>
<to uri="jms:queue:order:in"/>
<transform>
<constant>ACK</constant>
</transform>
</route>

Annotation DSL

You can also use the @Consume annotation for transformations. For example in the method below we consume from a JMS queue and do the transformation in regular Java code. Notice that the input and output parameters of the method is String. Camel will automatically coerce the payload to the expected type defined by the method. Since this is a JMS example the response will be sent back to the JMS reply-to destination.


@Consume(uri="jms:queue:order:transform")
public String transformOrder(String body) {
	// do message translation
}

TIP: You can use Bean Parameter Binding to help Camel coerce the Message into the method parameters. For instance you can use @Body, @Headers parameter annotations to bind parameters to the body and headers.

Message Filter

Diagram How can a component avoid receiving unwanted messages?
Message Filter
Problem How do you discard unwanted messages?
Solution Use a special kind of Message Router, a Message Filter, to eliminate undesired messages from a channel based on a set of criteria.
Camel Camel has support for Message Filter using the filter node. The filter evaluates a predicate whether its true or false; only allowing the true condition to pass the filter, where as the false condition will silently be ignored.
Java DSL We want to discard any test messages so we only route non-test messages to the order queue.

from("jms:queue:inbox")
	.filter(header("test").isNotEqualTo("true"))
	.to("jms:queue:order");

Spring DSL For the Spring DSL we use XPath to evaluate the predicate. The $test is a special shorthand in Camel to refer to the header with the given name. So even if the payload is not XML based we can still use XPath to evaluate predicates.

<route>
	<from uri="jms:queue:inbox"/>
	<filter>
		<xpath>$test = 'false'</xpath>
		<to uri="jms:queue:inbox"/>
	</filter>
</route>

Dynamic Router

Diagram
Dynamic Router
Problem How can we route messages based on a dynamic list of destinations?
Solution Use a Dynamic Router, a router that can self-configure based on special configuration messages from participating destinations.
Camel Camel has support for Dynamic Router using the Dynamic Recipient List combined with a data store holding the list of destinations.
Java DSL We use a Processor as the dynamic router to determine the destinations. We could also have used a Bean instead.

from("jms:queue:order")
	.processRef(myDynamicRouter)
	.recipientList("destinations");
	
public class MyDynamicRouter implements Processor {
	public void process(Exchange exchange) {
		// query a data store to find the best match of the
		// endpoint and return the destination(s) in the
		// header exchange.getIn()
		// .setHeader("destinations", list);
	}
}

Spring DSL

<route>
	<from uri="jms:queue:order"/>
	<process ref="myDynamicRouter"/>
	<recipientList>
		<header>destinations</destinations>
	</recipientList>
</route>

Annotation DSL

public class MyDynamicRouter {
	@Consume(uri = "jms:queue:order")
	@RecipientList
	public List<String> route(@XPath("/customer/id")
String customerId, @Header("location") String location,
Document body) {
		// query data store, find best match for the
		//endpoint and return destination (s)
	}
}

TIP: Notice how we used Bean Parameter Binding to bind the parameters to the route method based on an @XPath expression on the XML payload of the JMS message. This allows us to extract the customer id as a string parameter. @Header wil bind a JMS property with the key location. Document is the XML payload of the JMS message.

TIP: Camel uses its strong type converter feature to convert the payload to the type of the method parameter. We could use String and Camel will convert the body to a String instead. You can register your own type converters as well using the @Converter annotation at the class and method level.

Recipient List

Diagram How do we route a message to a list of statically or dynamically specified recipients?
Recipient List
Problem How can we route messages based on a static or dynamic list of destinations?
Solution Define a channel for each recipient. Then use a Recipient List to inspect an incoming message, determine the list of desired recipients and forward the message to all channels associated with the recipients in the list.
Camel Camel supports the static Recipient List using the multicast node, and the dynamic Recipient List using the recipientList node.
Java DSL

Static

In this route we route to a static list of two recipients, that will receive a copy of the same message simultaneously.


from("jms:queue:inbox")
	.multicast().to("file://backup", "seda:inbox");

Dynamic

In this route we route to a dynamic list of recipients defined in the message header [mails] containing a list of recipients as endpoint URLs. The bean processMails is used to add the header[mails] to the message.


from("seda:confirmMails").beanRef(processMails)
	.recipientList("destinations");

And in the process mails bean we use @Headers Bean Parameter Binding to provide a java.util.Map to store the recipients.


public void confirm(@Headers Map headers, @Body String body} {
	String[] recipients = ...
	headers.put(""destinations", recipients);
}

Spring DSL

Static


<route>
	<from uri="jms:queue:inbox" />
	<multicast>
		<to uri="file://backup"/>
		<to uri="seda:inbox"/>
	</multicast>
</route>

Dynamic

In this example we invoke a method call on a Bean to provide the dynamic list of recipients.


<route>
	<from uri="jms:queue:inbox" />
	<recipientList>
		<method bean="myDynamicRouter" method="route"/>
	</recipientList>
</route>

<bean id="myDynamicRouter"
	class="com.mycompany.MyDynamicRouter"/>
	
public class myDynamicRouter {
	public String[] route(String body) {
		return new String[] { "file://backup", .... }
	}
}

Annotation DSL

In the CustomerService class we annoate the whereTo method with @RecipientList, and return a single destination based on the customer id. Notice the flexibility of Camel as it can adapt accordingly to how you define what your methods are returning: a single element, a list, an iterator, etc.


public class CustomerService {
	@RecipientList
	public String whereTo(@Header("customerId") id) {
		return "jms:queue:customer:" + id;
	}
}

And then we can route to the bean and it will act as a dynamic recipient list.


from("jms:queue:inbox")
	.bean(CustomerService.class, "whereTo");

Splitter

Diagram How can we process a message if it contains multiple elements, each of which may have to be processed in a different way?
Splitter
Problem How can we split a single message into pieces to be routed individually?
Solution Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item.
Camel Camel has support for Splitter using the split node.
Java DSL

In this route we consume files from the inbox folder. Each file is then split into a new message. We use a tokenizer to split the file content line by line based on line breaks.


from("file://inbox")
	.split(body().tokenize("\n"))
	.to("seda:orderLines");

TIP: Camel also supports splitting streams using the streaming node. We can split the stream by using a comma:


.split(body().tokenize(",")).streaming().to("seda:parts");

TIP: In the routes above each individual split message will be executed in sequence. Camel also supports parallel execution using the parallelProcessing node.


.split(body().tokenize(",")).streaming()
	.parallelProcessing().to("seda:parts");

Spring DSL In this route we use XPath to split XML payloads received on the JMS order queue.

<route>
	<from uri="jms:queue:order"/>
	<split>
		<xpath>/invoice/lineItems</xpath>
		<to uri="seda:processOrderLine"/>
	</split>
</route>

And in this route we split the messages using a regular expression


<route>
	<from uri="jms:queue:order"/>
	<split>
		<tokenizer token="([A-Z|0-9]*);" regex="true"/>
		<to uri="seda:processOrderLine"/>
	</split>
</route>

TIP: Split evaluates an org.apahce.camel.Expression to provide something that is iterable to produce each individual new message. This allows you to provide any kind of expression such as a Bean invoked as a method call.


<split>
	<method bean="mySplitter" method="splitMe"/>
	<to uri="seda:processOrderLine"/>
</split>

<bean id="mySplitter" class="com.mycompany.MySplitter"/>

public List splitMe(String body) {
	// split using java code and return a List
	List parts = ...
	return parts;
}

Aggregator

Diagram How do we combine the results of individual, but related messages so that they can be processed as a whole?
Message Router
Problem How do we combine multiple messages into a single combined message?
Solution Use a stateful filter, an Aggregator, to collect and store individual messages until it receives a complete set of related messages to be published.
Camel Camel has support for the Aggregator using the aggregate node. Camel uses a stateful batch processor that is capable of aggregating related messaged into a single combined message. A correlation expression is used to determine which messages should be aggregated. An aggregation strategy is used to combine aggregated messages into the result message. Camel’s aggregator also supports a completion predicate allowing you to signal when the aggregation is complete. Camel also supports other completion signals based on timeout and/or a number of messages already aggregated.
Java DSL

Stock quote example

We want to update a website every five minutes with the latest stock quotes. The quotes are received on a JMS topic. As we can receive multiple quotes for the same stock within this time period we only want to keep the last one as its the most up to date. We can do this with the aggregator:


from("jms:topic:stock:quote")
	.aggregate().xpath("/quote/@symbol")
	.batchTimeout(5 * 60 * 1000).to("seda:quotes");

As the correlation expression we use XPath to fetch the stock symbol from the message body. As the aggregation strategy we use the default provided by Camel that picks the latest message, and thus also the most up to date. The time period is set as a timeout value in milliseconds.

Loan broker example

We aggregate responses from various banks for their quote for a given loan request. We want to pick the bank with the best quote (the cheapest loan), therefore we need to base our aggregation strategy to pick the best quote.


from("jms:topic:loan:quote")
	.aggregate().header("loanId")
	.aggregationStrategy(bestQuote)
	.completionPredicate(header(Exchange.AGGREGATED_SIZE)
	.isGreaterThan(2))
	.to("seda:bestLoanQuote");

We use a completion predicate that signals when we have received more than 2 quotes for a given loan, giving us at least 3 quotes to pick among. The following shows the code snippet for the aggregation strategy we must implement to pick the best quote:


public class BestQuoteStrategy implements AggregationStrategy {
	public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
		double oldQuote = oldExchange.getIn().getBody(Double.class);
		double newQuote = newExchange.getIn().getBody(Double.class);
		// return the "winner" that has the lowest quote
		return newQuote < oldQuote ? newExchange : oldExchange;
	}
}

Spring DSL

Loan Broker Example


<route>
	<from uri="jms:topic:loan:qoute"/>
	<aggregate strategyRef="bestQuote">
		<correlationExpression>
			<header>loanId</header>
		</correlationExpression>
		<completionPredicate>
			<simple>${header.CamelAggregatedSize} > 2</simple>
		</completionPredicate>
	</aggregate>
	<to uri="seda:bestLoanQuote"/>
</route>

<bean id="bestQuote"
	class="com.mycompany.BestQuoteStrategy"/>

TIP: We use the simple language to declare the completion predicate. Simple is a basic language that supports a primitive set of operators. ${header. CamelAggregatedSize} will fetch a header holding the number of messages aggregated.

TIP: If the completed predicate is more complex we can use a method call to invoke a Bean so we can do the evaluation in pure Java code:


<completionPredicate>
	<method bean="quoteService" method="isComplete"/>
</compledtionPrediacate>
public boolean isComplete(@Header(Exchange.AGGREGATED_SIZE)
	int count, String body) {
	return body.equals("STOP");
}

Notice how we can use Bean Binding Parameter to get hold of the aggregation size as a parameter, instead of looking it up in the message.

Resequencer

Diagram How can we get a stream of related but out-of-sequence messages back into the correct order?
Resequencer
Problem How do we ensure ordering of messages?
Solution Use a stateful filter, a Resequencer, to collect and reorder messages so that they can be published in a specified order.
Camel

Camel has support for the Resequencer using the resequence node. Camel uses a stateful batch processor that is capable of reordering related messages. Camel supports two resequencing algorithms:

-batch = collects messages into a batch, sorts the messages and publish the messages

-stream = re-orders, continuously, message streams based on detection of gaps between messages.

Batch is similar to the aggregator but with sorting. Stream is the traditional Resequencer pattern with gap detection. Stream requires usage of number (longs) as sequencer numbers, enforced by the gap detection, as it must be able to compute if gaps exist. A gap is detected if a number in a series is missing, e.g. 3, 4, 6 with number 5 missing. Camel will back off the messages until number 5 arrives.

Java DSL

Batch:

We want to process received stock quotes, once a minute, ordered by their stock symbol. We use XPath as the expression to select the stock symbol, as the value used for sorting.


from("jms:topic:stock:quote")
	.resequence().xpath("/quote/@symbol")
	.timeout(60 * 1000)
	.to("seda:quotes");

Camel will default the order to ascending. You can provide your own comparison for sorting if needed.

Stream:

Suppose we continuously poll a file directory for inventory updates, and its important they are processed in sequence by their inventory id. To do this we enable streaming and use one hour as the timeout.


from("file://inventory")
	.resequence().xpath("/inventory/@id")
	.stream().timeout(60 * 60 * 1000)
	.to("seda:inventoryUpdates");

Spring DSL

Batch:


<route>
	<from uri="jms:topic:stock:quote"/>
	<resequence>
		<xpath>/quote/@symbol</xpath>
		<batch-config batchTimeout="60000"/>
	</resequence>
	<to uri="seda:quotes"/>
</route>

Stream:


<route>
	<from uri="file://inventory"/>
	<resequence>
		<xpath>/inventory/@id
		<stream-config timeout="3600000"/>
	</resequence>
	<to uri="seda:quotes"/>
</route>

Notice that you can enable streaming by specifying <stream-config> instead of .

Dead Letter Channel

Diagram What will the messaging system do with a message it cannot deliver?
Message Router
Problem The messaging system cannot deliver a message
Solution When a message cannot be delivered it should be moved to a Dead Letter Channel
Camel

Camel has extensive support for Dead Letter Channel by its error handler and exception clauses. Error handler supports redelivery policies to decide how many times to try redelivering a message, before moving it to a Dead Letter Channel.

The default Dead Letter Channel will log the message at ERROR level and perform up to 6 redeliveries using a one second delay before each retry.

Error handler has two scopes: global and per route

TIP: See Exception Clause in the Camel documentation for selective interception of thrown exception. This allows you to route certain exceptions differently or even reset the failure by marking it as handled.

TIP: DeadLetterChannel supports processing the message before it gets redelivered using onRedelivery. This allows you to alter the message beforehand (i.e. to set any custom headers).

Java DSL

Global scope


errorHandler(deadLetterChannel("jms:queue:error")
	.maximumRedeliveries(3));
	
from(...)

Route scope
from("jms:queue:event")
	.errorHandler(deadLetterChannel()
	.maximumRedeliveries(5))
	.multicast().to("log:event", "seda:handleEvent");

In this route we override the global scope to use up to five redeliveries, where as the global only has three. You can of course also set a different error queue destination:


deadLetterChannel("log:badEvent").maximumRedeliveries(5)

Spring DSL

The error handler is configured very differently in the Java DSL vs. the Spring DSL. The Spring DSL relies more on standard Spring bean configuration whereas the Java DSL uses fluent builders.

Global scope

The Global scope error handler is configured using the errorHandlerRef attribute on the camelContext tag.


<camelContext errorHandlerRef="myDeadLetterChannel">
...
</camelContext>

Route scope

Route scoped is configured using the errorHandlerRef attribute on the route tag.


<route errorHandlerRef="myDeadLetterChannel">
...
</route>

For both the error handler itself is configured using a regular Spring bean


<bean id="myDeadLetterChannel" class="org.apache.camel.
builder.DeadLetterChannelBuilder">
	<property name="deadLetterUri" value="jms:queue:error"/>
	<property name="redeliveryPolicy"
		ref="myRedeliveryPolicy"/>
</bean>

<bean id="myRedeliverPolicy"
		class="org.apache.camel.processor.RedeliverPolicy">
	<property name="maximumRedeliveries" value="5"/>
	<property name="delay" value="5000"/>
</bean>

Wire Tap

Diagram How do you inspect messages that travel on a point-to-point channel?
Wire Tap
Problem How do you tap messages while they are routed?
Solution Insert a Wire Tap into the channel, that publishes each incoming message to the main channel as well as to a secondary channel.
Camel Camel has support for Wire Tap using the wireTap node, that supports two modes: traditional and new message. The traditional mode sends a copy of the original message, as opposed to sending a new message. All messages are sent as Event Message and runs in parallel with the original message.
Java DSL

Traditional

The route uses the traditional mode to send a copy of the original message to the seda tapped queue, while the original message is routed to its destination, the process order bean.


from("jms:queue:order")
	.wireTap("seda:tappedOrder")
	.to("bean:processOrder");

New message

In this route we tap the high priority orders and send a new message containing a body with the from part of the order. Tip: As Camel uses an Expression for evaluation you can use other functions than xpath, for instance to send a fixed String you can use constant.


from("jms:queue:order")
	.choice()
		.when("/order/priority = ‘high’")
			.wireTap("seda:from", xpath("/order/from"))
			.to("bean:processHighOrder");
		.otherwise()
			.to("bean:processOrder");

Spring DSL

Traditional


<route>
	<from uri="jms:queue:order"/>
	<wireTap uri="seda:tappedOrder"/>
	<to uri="bean:processOrder"/>
</route>

New Message


<route>
	<choice>
		<when>
			<xpath>/order/priority = 'high'</xpath>
			<wireTap uri="seda:from">
				<body><xpath>/order/from</xpath></body>
			</wireTap>
			<to uri="bean:processHighOrder"/>
		</when>
		<otherwise>
			<to uri="bean:processOrder"/>
		</otherwise>
	</choice>
</route>

Conclusion

The twelve patterns in this Refcard cover the most used patterns in the integration space, together with two of the most complex such as the Aggregator and the Dead Letter Channel. In the second part of this series we will take a further look at common patterns and transations.

Get More Information

Camel Website http://camel.apache.org The home of the Apache Camel project. Find downloads, tutorials, examples, getting started guides, issue tracker, roadmap, mailing lists, irc chat rooms, and how to get help.
FuseSource Website http://fusesource.com The home of the FuseSource company, the professional company behind Apache Camel with enterprise offerings, support, consulting and training.
About Author http://davsclaus.blogspot.com The personal blog of the author of this reference card.

About The Author

Photo of author Claus Ibsen

Claus Ibsen

Claus Ibsen is a passionate open-source enthusiast who specializes in the integration space. As an engineer in the Progress FUSE open source team he works full time on Apache Camel, FUSE Mediation Router (based on Apache Camel) and related projects. Claus is very active in the Apache Camel and FUSE communities, writing blogs, twittering, assisting on the forums irc channels and is driving the Apache Camel roadmap.

About Progress Fuse

FUSE products are standards-based, open source enterprise integration tools based on Apache SOA projects, and are productized and supported by the people who wrote the code.

Recommended Book

ASP.NET

Utilizing years of practical experience, seasoned experts Gregor Hohpe and Bobby Woolf show how asynchronous messaging has proven to be the best strategy for enterprise integration success. However, building and deploying messaging solutions presents a number of problems for developers. Enterprise Integration Patterns provides an invaluable catalog of sixty-five patterns, with real-world solutions that demonstrate the formidable of messaging and help you to design effective messaging solutions for your enterprise.


Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

ServiceMix 4.2

The Apache Open Source ESB

By Jos Dirksen

8,730 Downloads · Refcard 65 of 151 (see them all)

Download
FREE PDF


The Essential ServiceMix Cheat Sheet

ServiceMix 4.2 is an enterprise-class open source ESB from Apache. In the open-source community, there are many different solutions for each problem. Although there are many open-source ESB projects, not all of them are mature enough to be use to solve enterprise mission-critical integration problems. This DZone Refcard will introduce you to whats new in version 4.2 and walk you through the OSGi based architecture, Web services, configuration and deployment options of ServiceMix 4.2 components with some key tips mixed in along the way.
HTML Preview
Getting Started with ServiceMix 4.0

Getting Started with ServiceMix 4.0

By Jos Dirksen

About Servicemix 4.0

In the open source community there are many different solutions for each problem. When you look for an open source ESB, however, you don't have that many options. Even though there are many open source ESB projects, not all of them are mature enough to be used to solve enterprise mission critical integration problems. ServiceMix is one of the open source projects that is mature enough to be used in these scenarios. ServiceMix, an Apache project, has been around for a couple of years now. It provides all the features you expect from an ESB such as routing, transformation, etc. The previous version was built based on JBI (JSR-208), but in its latest iteration, which we're discussing in this Refcard, ServiceMix has moved to an OSGi based architecture, which we'll discuss later on.

This DZone Refcard will provide an overview of the core elements of ServiceMix 4.0 and will show you how to use ServiceMix 4 by providing example configurations.

Servicemix 4.0 Architecture

Before we show how to configure ServiceMix 4.0 for use, let us first look at the architecture of ServiceMix 4.0. This figure shows the following components:

Architecture of ServiceMix

ServiceMix Kernel: In this figure you can see that the basis of ServiceMix 4 is the ServiceMix Kernel. This kernel, which is based on the Apache Felix Karaf project (an OSGi based runtime), handles the core features ServiceMix provides, such as hot-deployment, provisioning of libraries or applications, remote access using ssh, JMX management and more.

ServiceMix NMR: This component, a normalized message router, handles all the routing of messages within ServiceMix and is used by all the other components.

>ActiveMQ: ActiveMQ, another Apache project, is the message broker which is used to exchange messages between components. Besides this ActiveMQ can also be used to create a fully distributed ESB.

Web: ServiceMix 4 also provides a web component. You can use this to start ServiceMix 4 embedded in a web application. An example of this is provided in the ServiceMix distribution.

JBI compatibility layer: The previous version of ServiceMix was based on JBI 1.0. For JBI a lot of components (from ServiceMix, but also from other parties), are available. This layer provides compatibility with the JBI specification, so that all the components from the previous version of ServiceMix can run on ServiceMix 4. Be sure though to use the 2009.01 version of these components.

Camel NMR: ServiceMix 4 provides a couple of different ways you can configure routing. You can use the endpoints provided by the ServiceMix NMR, but you can also use more advanced routing engines. One of those is the Camel NMR. This component allows you to run Camel based routes on ServiceMix.

CXF NMR: Besides an NMR based on Camel, ServiceMix also provides an NMR based on CXF. You can use this NMR to expose and route to Java POJOs annotated with JAX-WS annotations.

Hot Tip

OSGi runtime
ServiceMix runs on an OSGi based kernel, but what is OSGi? In short an OSGi container provides a service based in-VM platform on which you can deploy services and components dynamically. OSGi provides strict classloasing seperation and forces you to think about the dependencies your components have. Besides that OSGi also defines a simple lifecycle model for your services and components. This results in an environment where you can easily add and remove components and services at runtime and allows the creation of modular applications. An added advantage of using an OSGi container is that you can use many components out of the box: remote administration, a web container, configuration and preferences services, etc.

Before we move on to the next part, let's have a quick look at how a message is processed by ServiceMix. The following figure shows how a message is routed by the NMR. In this case we're showing a reply / response (in-out) message pattern.

Reply/Response Pattern

In this figure you can see a number of steps being executed:

  1. The consumer creates a message exchange for a specific service and sends a request.
  2. The NMR determines the provider this exchange needs to be sent to and queus the message for delivery. The provider accepts this message and executes its business logic.
  3. After the provider has finished processing, the response message is returned to the NMR.
  4. The NMR once again queues the message for delivery. This time to the consumer. The consumer accepts the message.
  5. After the response is accepted, the consumer sends a confirmation to the NMR.
  6. The NMR routes this confirmation to the provider, who accepts it and ends this message exchange.

Now that we've seen the architecture and how a message is handled by the NMR, we'll have a look at how to configure ServiceMix 4.

Configuration of ServiceMix 4.0

ServiceMix 4 configuration is mostly done through Spring XML files supported by XML schemas for easy code completion. Let's look at two simple examples. The first one uses the File Binding component to poll a directory and the second one exposes a Web service using ServiceMix's CXF support.


<beans xmlns:file="http://servicemix.Apache.org/file/1.0"
		xmlns:dzone="http://servicemix.org/dzone/">
	<file:poller service="foo:filePoller"
		endpoint="filePoller"
		targetService="foo:fileSender"
		file="inbox" />
</beans>

In this listing you can see that we define a poller. A poller is one of the standard components that is provided by ServiceMix's file-binding-component. If we deploy this configuration to ServiceMix, ServiceMix will start polling the inbox directory for files. If it finds one, the file will be sent to the specified targetService.

Hot Tip

Service Addressing
An important concept to understand when working with ServiceMix is that of services and endpoints. When you configure services on a component you need to tell ServiceMix how to route messages to and from that service. This name is called a service endpoint. If you look back at the previous example we created a file:poller. On this file:poller we defined a service and an endpoint attribute. These two attributes together uniquely identify this file:poller. Note though that you can have multiple endpoints defined on the same service. You can also see a targetService attribute on the file:poller. Besides this attribute there is also a targetEndpoint attribute. With these two attributes you identify the service endpoint to sent the message to. The targetEndpoint isn't always needed, if only one endpoint is registered on that service.

In the following listing, we've again used a simple XML file. This time we've configured a webservice.


<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:jaxws="http://cxf.Apache.org/jaxws"
	xsi:schemaLocation="
	http://www.springframework.org/schema/beans
	http://www.springframework.org/schema/beans/spring-beans.xsd
	http://cxf.Apache.org/jaxws http://cxf.Apache.org/schemas/jaxws.
	xsd">
<import resource="classpath:META-INF/cxf/cxf.xml" /> 1
<import resource="classpath:META-INF/cxf/cxf-extension-soap.xml" />
<import resource="classpath:META-INF/cxf/cxf-extension-http.xml" />
<import resource="classpath:META-INF/cxf/osgi/cxf-extensionosgi.xml" />
	<jaxws:endpoint id="helloWorld"
		implementor="dzone.refcards.HelloWorld"
		address="/HelloWorld"/>
</beans>

In this listing we use a jaxws:endpoint to define a webservice. The implementor points to a simple POJO annotated with JAX-WS annotations. If this example is deployed to ServiceMix, ServiceMix will register a webservice based on the value in the address attribute.

Deployment of ServiceMix 4 Components

ServiceMix provides a number of different options which you can use to deploy artifacts. In this section we'll look at these options, and show you how to use these.

ServiceMix 4, deployment options

Name Description
OSGi Bundles ServiceMix 4 is built around OSGi and ServiceMix 4 also allows you to deploy your configurations as an OSGi bundle with all the advantages OSGi provides.
Spring XML files ServiceMix 4 support plain Spring XML files.
JBI artifacts You can also deploy artifacts following the JBI standard (service assemblies and service units) to ServiceMix 4.
Feature descriptors This is a Karaf specific way for installing applications. It will install the necessary OSGi bundles and will add configuration defaults. This is mostly used to install core parts of the ServiceMix distribution.

OSGi bundle deployment
The easiest way to create an OSGi based ServiceMix bundle is by using Maven 2. To create a bundle you need to take a couple of simple steps. The first one is adding the mavenbundle- plugin to your pom.xml file. This is shown in the following code fragment.


...
<dependencies>
	<dependency>
		<groupId>org.Apache.felix</groupId>
		<artifactId>org.osgi.core</name>
		<version>1.0.0</version>
	</dependency>
	...
</dependencies>
...
<build>
	<plugins>
		<plugin>
			<groupId>org.Apache.felix</groupId>
			<artifactId>maven-bundle-plugin</artifactId>
			<configuration>
				<instructions>
				<Bundle-SymbolicName>${pom.artifactId}</Bundle-SymbolicName>
				<Import-Package>*,org.Apache.camel.osgi</Import-Package>
				<Private-Package>org.Apache.servicemix.examples.camel</Private-Package>
				</instructions>
			</configuration>
		</plugin>
	</plugins>
</build>
...

The important part here is the instructions section. This determines how the plugin packages your project. For more information on these settings see the maven OSGi bundle plugin page at http://cwiki.Apache.org/FELIX/Apachefelixmaven-bundle-plugin-bnd.html.

The next step is to make sure your project is bundled as a OSGi bundle. You do this by setting the <packaging> element in your pom.xml to bundle.

Now you can use mvn install to create an OSGi bundle, which you can copy to the deploy directory of ServiceMix and your bundle will be installed. If you use Spring to configure your application, make sure the Spring configuration files are located in the META-INF/spring directory. That way the Spring application context will be automatically created based on these files.

If you don't want to do this by hand you can also use a Maven archetype. ServiceMix provides a set of archetypes you can use. A good starting point for a project is the Camel OSGi archetype which you can use by executing the following following Maven command:


mvn archetype:create -DarchetypeGroupId=org.Apache.servicemix.tooling
-DarchetypeArtifactId=servicemix-osgi-camel-archetype
-DarchetypeVersion=4.0.0.2-fuse
-DgroupId=com.yourcompany -DartifactId=camel-router
-DremoteRepositories=http://repo.fusesource.com/maven2/

There are many other archetypes available. For an overview of the available archetypes see: http://repo.fusesource.com/maven2/org/Apache/servicemix/tooling/

Spring XML Files Deployment

It's also possible to deploy Spring files without OSGi. Just drop a Spring file into the deploy directory. There are two points to take into account. First, you need to add the following to your Spring configuration file:


<bean class="org.Apache.servicemix.common.osgi.EndpointExporter" />

This will register the endpoints you've configured in your Spring file. The next element is optional but is good practice to add:


<manifest>
	Bundle-Version = 1.0.0
	Bundle-Name = Dzone :: Dzone test application
	Bundle-SymbolicName = dzone.refcards.test
	Bundle-Description = An example for servicemix refcard
	Bundle-Vendor = jos.dirksen@gmail.com
	Require-Bundle = servicemix-file, servicemix-eip
</manifest>

Using a manifest configuration element allows you to specify how your application is registered in ServiceMix.

JBI artifacts deployment

If you've already invested in JBI based applications, you can still use ServiceMix 4 to run them in. Just deploy your Service Assembly (SA) in the ServiceMix deploy directory and ServiceMix will deploy your application.

Feature descriptor based deployment

If you've got an application which contains many bundles and that requires additional configuration you can use a feature to easily manage this. A feature contains a set of bundles and configuration which can be easily installed from the ServiceMix console. The following listing shows the feature descriptor of the nmr component.


<features>
	<feature name="nmr" version="1.0.0">
	<bundle>mvn:org.Apache.servicemix.document/org.Apache.servicemix.document/1.0.0</bundle>
	<bundle>mvn:org.Apache.servicemix.nmr/org.Apache.servicemix.nmr.api/1.0.0</bundle>
	<bundle>mvn:org.Apache.servicemix.nmr/org.Apache.servicemix.nmr.core/1.0.0</bundle>
	<bundle>mvn:org.Apache.servicemix.nmr/org.Apache.servicemix.nmr.osgi/1.0.0</bundle>
	<bundle>mvn:org.Apache.servicemix.nmr/org.Apache.servicemix.nmr.spring/1.0.0</bundle>
	<bundle>mvn:org.Apache.servicemix.nmr/org.Apache.servicemix.nmr.commands/1.0.0</bundle>
	<bundle>mvn:org.Apache.servicemix.nmr/org.Apache.servicemix.nmr.management/1.0.0</bundle>
	</feature>
</features>

If you want to install this feature you can just type features/install nmr from the ServiceMix console.

Routing in ServiceMix 4.0

For routing in ServiceMix you've got two options:

  • EIP: ServiceMix provides a JBI component that implements a number of Enterprise Integration Patterns.
  • Camel: You can use Camel routes in ServiceMix. Camel provides the most flexible and exhaustive routing options for ServiceMix

EIP Component Routing

This routing is provided by the EIP component. To check whether this is installed in your ServiceMix runtime you can execute features/list from the ServiceMix commandline. This will show you a list of installed features. If you see [installed] [ 2009.01] servicemix-eip the component is installed. If it shows uninstalled instead of installed, you can use the features/install servicemix-eip to install this component. You can now use this router using a simple XML file:


<eip:static-routing-slip service="test:routingSlip"	endpoint="endpoint">
	<eip:targets>
		<eip:exchange-target service="test:echo" />
		<eip:exchange-target service="test:echo" />
	</eip:targets>
</eip:static-routing-slip>

When installed this component provides the following routing options (this information is also available in the XSD of this component):

XML Element Description
async-bridge The async bridge pattern is used to bridge an In-Out exchange with two In-Only (or Robust-In-Only) exchanges. This pattern is the opposite of the pipeline.
content-basedrouter Component that can be used for content based routing of the message. You can configure this component with a set of predicates which define how the message is routed.
content-enricher A content enricher can be used to add extra information to the message from a different source.
message-filter With a message filter you specify a set of predicates which determine whether to process the message or not.
pipeline The pipeline component is a bridge between an In-Only (or Robust-In- Only) MEP and an In-Out MEP. This is the opposite of the async bridge.
resequencer A resequencer can be used to re-order a set of incoming messages before passing them on in a the new order.
split-aggregator A split aggregator is used to reassemble messages that have been split by a splitter.
static-recipient-list A static recipient list will forward the incoming message to a set of predefined destinations.
static-routing-slip The static routing slip routes a message through a set of services. It uses the result of the first invocation as input for the next.
wire-tap The wire-tap will copy and forward a message to the specified destination.
xpath-splitter This splitter uses an xpath expression to split an incoming message in multiple parts.

Camel Routing

Apache Camel is a project which provides a lof of different routing and integration options. In this section we'll show how to use Camel with ServiceMix and give an overview of the routing options it provides. Installing the Camel component in ServiceMix is done in the same way as we did for the EIP component. We use the features/list command to check what's already installed and we can use features/add to add new Camel functionality. Once installed we can use Camel to route messages between our components. Camel provides two types of configuration: XML and Java based DSL, XML configuration was used for the following two listings:

Camel XML configuration - Listing 1: Camel configuration

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<import resource="classpath:org/Apache/servicemix/camel/nmr/camel-nmr.xml" />
	<camelContext xmlns="http://camel.Apache.org/schema/spring">
	<route>
		<from uri="ftp://gertv@localhost/testfile?password=secret"/>
		<to uri="nmr:IncomingOrders"/>
	</route>
</beans>				
	
Camel XML configuration - Listing 2: Target service

<beans xmlns:file="http://servicemix.Apache.org/file/1.0"
	xmlns:dzone="http://servicemix.org/dzone/">
	<import resource="classpath:org/Apache/servicemix/camel/nmr/camel-nmr.xml" />
	<file:sender service="nmr:IncomingOrders" directory="file:target/pollerFiles" />
</beans>			
	

In these two listings you can see how we can easily integrate the Camel routes with the other components from ServiceMix. We use the nmr prefix to tell Camel to send the message to the NMR. The other service, which can be seperately deployed will then pick-up this message since it's also configured to listen to a nmr prefixed service.

Now let's look at two listings that use Camel's Java based DSL to configure the routes. For this we need a small XML file describing where the routes can be found, and a Java file which contains the routing.

Camel Java configuration - Listing 1: Spring configuration

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
	http://www.springframework.org/schema/beans http://www.
	springframework.org/schema/
	beans/spring-beans-2.0.xsd
	http://activemq.Apache.org/camel/schema/spring
	http://activemq.Apache.org/camel/schema/spring/camel-spring.xsd">
		
<import resource="classpath:org/Apache/servicemix/camel/nmr/camelnmr.xml" />
	<camelContext xmlns="http://activemq.Apache.org/camel/schema/spring">
		<package>dzone.refcards.camel.routes</package>
	</camelContext>
</beans>
	
Camel Java configuration - Listing 2: Java route

public class SimpleRouter extends RouteBuilder {
	public void configure() throws Exception {
		from("timer:myTimerEvent?fixedRate=true")
			.setBody(constant("Hello World!")).
				to("nmr:someService");
	}
}
	

Camel itself provides a lot of standard functionality. It doesn't just provide routing, it can also provide connectivity for different technologies. For more information on Camel please see it's website at http://camel.Apache.org/ or look at the "Enterprise Integrations Patterns with Camel" Refcard.

Hot Tip

Differences between ServiceMix and Camel
If you've looked at the Camel website you notice that it provides much the same functionality as ServiceMix. It provides connectivity to various standards and technologies, provides routing and transformation and even allows you to expose Web services. The main difference though is that Camel isn't a container. Camel is designed to be used inside some other container. We've shown that you can use Camel in ServiceMix, but you can also use Camel in other ESBs or in ActiveMQ or CXF. So if you just want an routing and mediation engine Camel is a good choice. If you however need a full ESB with good support for JBI, a flexible OSGi based kernel, hot-deploy and easy administration ServiceMix is the better choice.

ServiceMix and web services

Support for Web services is an important feature for an ESB. ServiceMix uses the CXF project for this. Since CXF is also completely spring based, using CXF to deploy Web services is very easy.

Hosting Web services

When you want to expose a service as a webservice you can easily do this using CXF. Just create a CXF OSGi bundle using the archetype: servicemix-osgicxf-code-first-archetype. This will create an OSGi and CXF enabled maven project which you can use to develop webservices. Now just edit the src/main/ resources/META-INF/spring/beans.xml file and after you've run the mvn install command you can deploy the bundle to ServiceMix. The following listing shows such an example. This will create a Web service and host it on http://localhost:8080/cfx/HelloDzone.

CXF Host Web service example using CXF

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:jaxws="http://cxf.Apache.org/jaxws"
	xsi:schemaLocation="
	http://www.springframework.org/schema/beans
	http://www.springframework.org/schema/beans/spring-beans.xsd
	http://cxf.Apache.org/jaxws http://cxf.Apache.org/schemas/jaxws.
	xsd">
		
	<import resource="classpath:META-INF/cxf/cxf.xml" />
	<import resource="classpath:META-INF/cxf/cxf-extension-soap.xml" />
	<import resource="classpath:META-INF/cxf/cxf-extension-http.xml" />
	<import resource="classpath:META-INF/cxf/osgi/cxf-extensionosgi.xml" />
	<jaxws:endpoint id="helloDZone"
		implementor="dzone.examples.ws.HelloDZoneImpl"
			address="/HelloDzone"/>
</beans>	
	

In the previous example we hoseted a Web service which could be called from outside the container. You can also configure CXF to host the Web service internally by prefixing the address with nmr. That way you can easily expose JAX-WS annotated java beans to the other services inside the ESB. The following example shows this:

CXF Host Web service internally

<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:jaxws="http://cxf.apache.org/jaxws"
	xsi:schemaLocation="
	http://www.springframework.org/schema/beans
	http://www.springframework.org/schema/beans/spring-beans.xsd
	http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd">
		
	<import resource="classpath:META-INF/cxf/cxf.xml" />
	<import resource="classpath:META-INF/cxf/cxf-extension-soap.xml" />
	<import resource="classpath:META-INF/cxf/transport/nmr/cxftransportnmr.xml" />
	<jaxws:endpoint id="helloDzone"
		implementor="dzone.examples.ws.HelloDZoneImpl"
			address="nmr:helloDZone" />
</beans>	
	

You can also host a Web services using the servicemix-cxf-bc component.

Host Web service using the servicemix-cxf-bc component

<beans xmlns:cxfbc="http://servicemix.Apache.org/cxfbc/1.0"
		xmlns:dzone="http://dzone.org/refcard/example">
	<cxfbc:consumer wsdl="classpath:dzone-example.wsdl"
		targetService="dzone:ExampleService"
		targetInterface="dzone:Example"/>
</beans>	
	

Consuming Web services

Consuming Web services in ServiceMix is just as easy. ServiceMix provides two different options for this. You can use Camel or use the servicemix-cxf-bc component:

Consume Web servicemix using the servicemix-cxf-bc component

<beans xmlns:cxfbc="http://servicemix.Apache.org/cxfbc/1.0"
		xmlns:dzone="http://dzone.org/refcard/example">
	<cxfbc:provider wsdl="classpath:target-service.wsdl"
		locationURI="http://webservice.com/Service"
		endpoint="ServicePort"
		service="dzone:ServicePortService"/>
</beans>	
	

With this configuration you can consume a Web service which is located at http://webservice.com/Service and which is defined by the WSDL file target-service.wsdl. Other services can use this component by making a call to the dzone:ServicePortService.

You can also consume a Web service using Camel. For more information on how you can configure the Camel route for this look at the Camel CXF integration section of the Camel website: http://camel.Apache.org/cxf.html.

For Web services ServiceMix provides the following useful archetypes:

Name Description
servicemix-cxf-bc-service-unit Create a maven project which uses the JBI CXF binding component.
servicemix-cxf-se-service-unit Create a maven project which uses the JBI CXF service engine.
servicemix-cxf-se-wsdlfirstservice-unit Create a maven project which uses the JBI CXF service engine. This project is based on WSDL first development.
servicemix-osgi-cxf-codefirstarchetype Create a maven project which uses CXF and OSGi together. This project is based on code first development.
servicemix-osgi-cxf-wsdlfirstarchetype Create a maven project which uses CXF and OSGi together. This project is based on wsdl first development.

Servicemix Components

Besides integration with Web services through CXF, ServiceMix provides a lot of components you can use out of the box to integrate with various other standards and technologies. In this section we'll give an overview of these components. This list is based on the 2009.1 versions. Most of this information can also be found in the XML schemas of these components.

ServiceMix Components

XML Element Description
ServiceMix Bean
Endpoint Allows you to define a simple bean that can receive and send message exchanges.
ServiceMix File
Poller A polling endpoint that looks for a file or files in a directory and sends the files to a target service. You can configure various options on this endpoint such as archiving, filters, use of subdirectories etc.
Sender An endpoint that receives messages from the NMR and writes them to a specific file or directory.
ServiceMix CXF Binding Component
consumer A consumer endpoint that is capable of using SOAP/HTTP or SOAP/JMS.
Provider A provider endpoint that is capable of exposing SOAP/HTTP or SOAP/JMS services.
ServiceMix CXF Service Engine
Endpoint With the Drools Endpoint you can use a drools rule set as a service or as a router.
ServiceMix FTP
Poller This endpoint can be used to poll an FTP directory for files, download them and send them to a service.
Sender With a sender endpoint you can store a message on an FTP server.
ServiceMix HTTP
Consumer Plain HTTP consumer endpoint. This endpoint can be used to handle plain HTTP request (without SOAP) or to be able to process the request in a non standard way.
Provider A plain HTTP provider. This type of endpoint can be used to send non- SOAP requests to HTTP endpoints.
Soap-Consumer An HTTP consumer endpoint that is optimized to work with SOAP messages.
Soap-Provider An HTTP provider endpoint that is optimized to work with SOAP messages.
ServiceMix JMS
Consumer An endpoint that can receive messages from a JMS broker.
Provider An endpoint that can send messages to a JMS broker.
Soap-Consumer A JMS consumer that is optimized to work with SOAP messages.
Soap-Provider A JMS provider that is optimized to work with SOAP messages.
JCA-Consumer A JMS consumer that uses JCA to connect to the JMS broker.
ServiceMix Mail
Poller An endpoint which can be used to retrieve messages.
Sender An endpoint which you can use to send messages.
ServiceMix OSWorkflow
Endpoint This endpoint can be used to start an OSWorkflow proces.
ServiceMix Quartz
Endpoint The Quartz endpoint can be used to fire messages into the NMR at specific intervals.
ServiceMix Saxon
XSLT With the XSLT endpoint you can apply an XSLT transformation to the received message.
Proxy The proxy component allows you to transform an incoming message and send it to an endpoint. You can also configure a transformation that needs to be applied to the result of that invocation.
XQuery The XQuery endpoint can be used to apply a selected XQuery to the input document.
ServiceMix Scripting
Endpoint With the scripting endpoint you can create a service which is implemented using a scripting language. The following languages are supported: Groovy, JRuby, Rhino JavaScript
ServiceMix SMPP
Consumer A polling component which bind with jSMPP and receive SMPP messages and sends the SMPPs into the NMR as messages.
Provider A provider component receives XML message from the NMR and converts into SMPP packet and sends it to SMPP server.
ServiceMix SNMP
Poller With this poller you can receive SNMP events by using the SNMP4J library.
ServiceMix Validation
Endpoint With this endpoint you can provide schema validation of documents using JAXP 1.3 and XMLSchema or RelaxNG.
ServiceMix-VFS
Poller An polling endpoint that looks for a file or files in a virtual file system (based on Apache commons-vfs) and sends the files to a target service.
Sender An endpoint which receives messages from the NMR and writes the message to the virtual file system.
ServiceMix-wsn2005
Create-pullpoint Lets you create a WS-Notification pull point that can be used by a requester to retrieve accumulated notification messages.
Publisher Sends messages to a specific topic.
Registerpublisher An endpoint that can be used by publishers to register themselves.
Subscribe Lets you create subscriptions to a specific topic using the WSNotification specification.
Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

Getting Started with Caucho Resin

By Emil Ong

4,861 Downloads · Refcard 88 of 151 (see them all)

Download
FREE PDF


The Essential Caucho Resin Cheat Sheet

Caucho Technology’s Resin® is a Java Application Server with a reputation for being lightweight and fast, yet reliable and scalable enough to power the most demanding enterprise sites. Resin comes in two flavors: Resin Open Source and Resin Professional. Resin Open Source is licensed under the GPL and has all the features for Java EE 6 Web Profile development. Resin Professional builds on Resin Open Source and offers features such as clustering, fast native I/O, and OpenSSL integration. This DZone Refcard starts with the basics and covers many of the features Resin 4.0 has to offer emerging technologies.
HTML Preview
Getting Started with Caucho Resin

Getting Started with Caucho Resin

By Emil Ong

ABOUT RESIN

Caucho Technology's Resin® is a Java Application Server with a reputation for being lightweight and fast, yet reliable and scalable enough to power the most demanding enterprise sites. Beginning as a Servlet and JSP engine in 1998, Resin has since evolved to support the Java EE 6 Web Profile within highly integrated implementations of Servlet 3.0, CDI, and EJB 3.1 Lite. In addition to the Web Profile standards, Resin also includes a high performance JTA transaction manager, a JMS provider, clustering, connection pooling, and a management console.

Resin is available in two flavors: Resin Open Source and Resin Professional. Resin Open Source is licensed under the GPL and has all the features necessary for Java EE 6 Web Profile development. Resin Professional builds on Resin Open Source and offers advanced features such as clustering (both traditional and cloud), fast native I/O, proxy caching, and OpenSSL integration.

DOWNLOADING RESIN

Resin is maintained in two branches: stable and development. At the time of writing, the stable branch is Resin 3.1 and the development branch is Resin 4.0. Users should note that despite the name, each release of the development branch is production ready. It's termed “development” because all new features and APIs go into this branch, but core technologies like Web Profile APIs are stable. This Refcard will deal entirely with Resin 4.0 because it contains the Java EE 6 Web Profile implementation (Resin 3.1 focused on Java EE 5) and has many new exciting features that are useful for emerging technologies such as cloud deployment.

All currently available versions of Resin are listed for download at http://caucho.com/download. Most users will want to download Resin Professional, even if they haven't purchased a license. Without the license, Resin Professional operates just like Resin Open Source. If you decide to upgrade later, all you have to do is drop in a license file. Developers who want a purely GPL version can download Resin Open Source.

INSTALLING RESIN

Resin is available in tar.gz and .zip formats for Unix, Mac OS X, and Windows users. While Resin can run in a pure Java mode, it also features some native (JNI) code that offers functionality like dropping root privileges in Unix and OpenSSL integration. Because of these features, you'll need to compile the JNI code if you're running on Unix or Mac OS X. Windows DLLs are provided in the distribution. Ubuntu Linux users can use Caucho's Ubuntu repository to install Resin as a .deb.

Hot Tip

Resin 4.0 now requires Java SE 6. This latest version of Java introduces a number of API improvements and Caucho's internal testing shows performance benefits as well. Make sure to get the JDK, not just the JRE.

Unix and Mac OS X installation

To install Resin on Unix and Mac OS X, you'll need to compile the JNI code for Resin. Before you compile, you'll need an environment capable of compiling C code (gcc, etc.). You'll probably also want OpenSSL, though this isn't strictly required. Once you've unpacked Resin, it should create a directory with the full Resin version name (e.g. resin-pro-4.0.4). Change to this directory and run the following commands:


$ ./configure
$ make
$ sudo make install

You'll need to run the last command (make install) as root so you can install Resin system wide. By default, this installs Resin in /usr/local/resin-<version>, but you can change this behavior by passing the --prefix= to the configure command. For more options, pass -- help to the configure command.

Windows Installation

Resin includes precompiled DLLs for the native code, so no compilation is necessary. Simply unpack the Resin .zip file directly where you'd like to install Resin and you're done. Typically, we recommend C:\Resin.

Once you have Resin installed, you should see two .exe files in the top-level Resin directory, resin.exe and setup.exe. resin. exe launches Resin from the command line, but is used for backwards compatibility. setup.exe installs Resin as a Windows Service, which we will discuss in a later section.

Ubuntu Installation

Ubuntu users can use Caucho's Ubuntu repository to install Resin. Add the following lines to your /etc/apt/sources.list:


deb http://caucho.com/download/debian/ unstable universe
deb http://caucho.com/download/debian/ unstable multiverse

After adding these lines, then run


$ sudo apt-get update

This command will update your Ubuntu database with information about the latest Resin releases. To install Resin, run:


$ sudo apt-get install resin-pro

This will install Resin Professional. Use “resin” instead of “resin-pro” if you'd prefer to install Resin Open Source. This installation will start Resin for you automatically on startup.

RESIN DIRECTORY LAYOUT

Resin uses a number of directories for the server itself, application files, and logging. These are all configurable (as you saw in the installation section), but we'll need to refer to them by name later. The following table will give the names, descriptions, and standard locations of commonly used Resin directories:

Directory Description Recommended
Resin Home Contains Resin server jars, JNI, and licenses. /usr/local/resin-<version> (Unix)
C:\Resin (Windows)
Root Directory Contains application content, log files, and server data /var/www (Unix)
C:\www (Windows)
Web App Deploy Directory Contains applications (.war files and exploded applications) webapps/ (Subdirectory of root directory)
Log Directory Contains server and access log files log/ (Subdirectory of root directory)

Resin separates the Resin Home directory, where Resin's libraries and licenses are contained, from the root directory, where your applications and logs are maintained. This structure makes it easier for you to upgrade Resin without disturbing your applications. In other words, you can just replace your Resin Home directory with the new Resin installation to upgrade.

STARTING RESIN

Resin command line

Resin can be started from the command line for debugging and development purposes. To run Resin with its output sent to the console, use the following command:


$ java -jar $RESIN_HOME/lib/resin.jar console

Resin can also open a standard Java debugging port on startup using the -debug-port option:


$ java -jar $RESIN_HOME/lib/resin.jar -debug-port 8000 console

Similarly, you can have Resin open a JMX port using the jmxport option:


$ java -jar $RESIN_HOME/lib/resin.jar -jmx-port 9000 console

To have Resin run in the background with its log output placed in the log directory, run:


$ java -jar $RESIN_HOME/lib/resin.jar start

You may also want to set the root directory of Resin and a specific configuration file on the command line as well:


$ java -jar $RESIN_HOME/lib/resin.jar --root-directory /var/www --conf /etc/resin/resin.xml start

If you need to stop Resin, you can run:


$ java -jar $RESIN_HOME/lib/resin.jar stop

This command stops the Resin instance, but the Resin Watchdog will still be running. The Resin Watchdog (see the Hot Tip below for more info) and all Resin processes can be stopped by running:


$ java -jar $RESIN_HOME/lib/resin.jar shutdown

Starting Resin at Boot

Once you have installed Resin system-wide, you may want to have it start when your server boots up. Resin provides start scripts for Unix and Windows.

Unix boot-time start up

Resin provides an init.d script and installs it in /etc/init.d/resin. This script is editable and essentially just starts the Resin server using the command line interface shown above. For standard installations, it shouldn't be necessary to modify this file, but you can configure alternate directories.

Windows Server Installation

Resin includes a Windows installation program called setup. exe to create a Windows Service. You can set parameters such as the Resin Home, Resin Root, and Log Directory for the service. You can also set the Service name or remove an existing service.

Resin Setup

Hot Tip

Even though we showed Resin run as a single Java process above, there are actually two processes being run, Resin and a Watchdog. The Watchdog is in charge of launching and monitoring the Resin process. This architecture provides additional reliability by restarting Resin if there's an error.

CONFIGURING RESIN

Configuring the Resin Server

Resin's server configuration is XML-based and contained largely within one file called resin.xml. The default resin.xml should work fine for most users with single-server deployments or developers. For more advanced configurations however, you'll want to understand and modify the resin.xml file.

Structure of resin.xml

The XML structure of the resin.xml file models the organization of Resin. At the top level there is the full Resin deployment which contains clusters. Each cluster is a set of servers and (virtual) hosts. (Note that even when running a single server, Resin considers this to be a cluster with one server.) Each host contains web applications.

Resin XML Structure

With this hierarchical structure, you can share resources and policies across applications. For example, you could configure a database pool shared by all applications in a single host or a distributed cache shared by all servers in a cluster.

Configuring JVM parameters

One of the most common tasks that administrators first do when setting up a new server is to configure JVM parameters. Because of Resin's Watchdog architecture, you can configure these parameters directly in your resin.xml file. When the Watchdog launches (or relaunches) a server, it will start a new JVM with these parameters:


<resin>
	<cluster>
		<server id=“a”>
			<jvm-arg>-Xmx512m </jvm-arg>
		</server>
...

Configuring Applications with resin-web.xml

Resin supports a number of features for applications that go beyond what Java EE standards specify, such as database pooling, custom logs and log rotation, and security authenticators. All of these facilities can be configured in the top-level resin.xml or in a Resin-specific application deployment descriptor file named WEB-INF/resin-web.xml. This file is recognized by Resin and can be used alongside of the portable web.xml descriptor file to configure Resin-specific resources. You can think of the resin-web.xml file as providing the <web-app> configuration in the resin.xml structure above. The sections below will discuss how to configure resources in the resin-web.xml file.

MONITORING AND ADMINISTRATION

Setting up an administrator password

Once you've started Resin, one of the first tasks you'll want to do is set up a password for administration tasks. To do this, just browse to http://localhost:8080/resin-admin (replace localhost with the host on which you've installed Resin if it's different). There you should see a login page with a section called Don't have a login yet? Enter a username and password, then submit and follow the directions on the next page for copying the generated password file to your Resin installation.

Once you've installed the password file, Resin will restart automatically and you can login with the password you've set up. The administration application features a number of monitoring resources for web applications, the proxy cache, server configuration, cluster status, memory usage, CPU usage, sampling-based profiling, and post-mortem failure analysis.

Resin Admin

DEPLOYING APPLICATIONS

File system deployment

Resin offers file system-based “hot deployment” with which you can deploy an application by copying it to the Resin “webapps” directory. The application can be either in .war form or an “exploded” war. If you deploy the application as a .war, Resin will expand and start it automatically by default.

Many developers may also prefer to copy their application in “exploded” form for development. Resin is able to detect changes to the code base (both JSPs and Java files) within the application and automatically recompile and redeploy. This feature can help developers iterate quickly to see code changes in their application.

While file system-based deployment is great for developing applications and deploying applications on a single machine, it can be difficult to manage when deploying to multiple machines in a cluster. Resin offers a distributed deployment mechanism for these cases.

Distributed Deployment

Resin 4.0 introduced a new clustering mechanism and a new deployment tool for distributed applications. This tool lets users deploy their application once and have it distributed across all servers in a cluster, even dynamic servers that are added after the application is deployed! (See the Clustering section for more information about dynamic servers.)

This tool is accessible via Ant and Maven. To use the Resin Maven plugin to deploy, add Caucho's Maven repository to your pom.xml:


<pluginRepositories>
	<pluginRepository>
		<snapshots>
		<enabled>true</enabled>
			<updatePolicy>always</updatePolicy>
			<checksumPolicy>ignore</checksumPolicy>
		</snapshots>
	<id>caucho</id>
	<name>Caucho</name>
	<url>http://caucho.com/m2-snapshot</url>
	</pluginRepository>
</pluginRepositories>

Once you have the plugin available, add it to your build configuration and specify the administrator username and password that you setup in the administration application:


<build>
	<finalName>foo</finalName>
	<plugins>
		<plugin>
			<groupId>com.caucho</groupId>
			<artifactId>resin-maven-plugin</artifactId>
			<version>4.0.4</version>
			<configuration>
				<server>127.0.0.1</server>
				<port>80</port>
				<user>admin</user>
				<password>my-admin-pass</password>
			</configuration>
		</plugin>
	</plugins>
</build>

Once you have this configuration in your pom.xml, you can deploy to Resin simply by using the upload .war goal:


$ mvn resin:upload-war

For more Maven options, see http://wiki.caucho.com/Maven2 For the related Ant plugin, see http://wiki.caucho.com/Ant

CONNECTING DATABASES

Resin features a built-in database pool which allows multiple database servers, load balancing, and connection checking. Resin's database pools are integrated with Resin's CanDI (CDI implementation) so that developers can use annotations to inject the database easily into their code.

The following code shows a sample database pool configuration that you might include in your resin-web.xml for a pool of up to 150 simultaneous connections:


<web-app xmlns="http://caucho.com/ns/resin">
	<database jndi-name="jdbc/myDb">
		<driver type="org.postgresql.Driver">
			<url>jdbc:postgresql://localhost/test</url>
			<user>myUser</user>
			<password>myPassword</password>
		</driver>
	<max-connections>150</max-connections>
	</database>
</web-app>

Once you've configured the pool in your resin-web.xml, you can either use JNDI to access the DataSource or use CDI annotations as in the following class:


public class MyBusinessLogic {
	@javax.inject.Inject
	DataSource myDatabase;
	...
}

LOGGING

Resin uses standard java.util.logging facilities for all its internal logging and implements several custom log handlers to manage log output and log rotation. The default logging configuration in the resin.xml file provides INFO-level logging for all Resin output. The XML for this configuration is:


<log-handler name=“” level=“all” path=“stdout:” timestamp=“[%y-%m-%d %H:%M:%S.%s] {%{thread}}” />
<logger name=“com.caucho” level=“info”/>

You can configure additional loggers for your classes simply by adding another <logger> tag either to resin.xml or your application's resin-web.xml. The default log-handler will output all log messages to the log directory (or standard output, if running in console mode). You can also configure additional log-handlers to deal specifically with your classes' log messages. For example, if all of your packages started with “com.example”, you could configure a logger and log-handler:


<log-handler name=“com.example” level=“all” path=“example.log” archive-format=“example-%Y%m%d.log.gz” rollover-period=“1D”/>
<logger name=“com.example” level=“info”/>

Notice that the names of both the logger and log-handler are “com.example”. We also changed the path to an explicit file name. We also added a rollover-period and an archive format. In this case, the log will be rolled over (rotated) once a day and the old log will be stored as example-%Y%m%d.log.gz, where the %Y%m%d will be replaced with the year, month, and date when the log was rolled over. The .gz extension also indicates to Resin that this log should be gzipped.

THE RESIN HTTP SERVER

Resin includes its own powerful HTTP server which features comparable performance to C-based servers such as Apache or nginx without the overhead of requiring multiple processes. Using the Resin HTTP server is recommended. In addition to its solid performance, the Resin HTTP server also has a number of convenient features for configuring SSL, rewriting requests, and managing virtual hosts.

OpenSSL

One of Resin Professional's most useful features is OpenSSL integration which offers far faster SSL performance than pure Java solutions. To configure OpenSSL, add an <http> tag with an <openssl> tag to your server configuration:


<resin xmlns=“http://caucho.com/ns/resin” xmlns:resin=“urn:java:com.caucho.resin”>
	<cluster id=“app-tier”>
		<server id=“” address=“192.168.0.10” port=“6800”>
		<http port=“443”>
		<openssl>
			<certificate-file>example.crt</certificate-file>
			<certificate-key-file>example.key</certificate-key-file>
			<password>my-password</password>
		</openssl>
		</http>
		</server>
	</cluster>
	...
</resin>

Rewrite Dispatch

Resin offers a URL rewriting mechanism similar to Apache's mod_rewrite in its HTTP server. Rules for rewriting URLs can be configured on an application, host, server, or cluster level. For example, you may want to allow all requests for specific static resources (such as images, CSS, JavaScript, etc.) to be served as usual, but redirect all other requests to a central controller Servlet. You could achieve that within your resin-web.xml with the following configuration:


<web-app xmlns=“http://caucho.com/ns/resin” xmlns:resin=“urn:java:com.caucho.resin”>
	<resin:Dispatch regexp=“\.(php|js|gif|png|css|html)$”/>
	<resin:Dispatch regexp=“^” target=“/controller”/>
</web-app>

The <resin:Dispatch> tag here is an internal redirection (i.e. the request is passed within the server without an HTTP redirect). You can use tags for HTTP forwarding, FastCGI integration, load balancing, and more:

Tag Behavior
<resin:Dispatch> Redirect a request internally
<resin:Redirect> Send an HTTP redirect
<resin:Forbidden> Send an HTTP forbidden response
<resin:Forward> Send an HTTP forward
<resin:FastCgiProxy> Redirect requests to a backend FastCGI process
<resin:HttpProxy> Redirect requests to a backend HTTP service
<resin:LoadBalance> Redirect the request to a backend cluster for processing

Virtual Hosts

For many deployments, you may not need to use specialized virtual hosts (e.g. you only use example.com and www.example. com). In these cases, you can deploy to the standard web app deploy directory and Resin will serve all your applications regardless of the virtual host specified by the client.

If you have a deployment with more virtual hosts however (store.example.com, blog.example.com, etc.), you'll need to organize your applications in the appropriate virtual hosts. Virtual hosts are a native concept to Resin and you can create them two different ways:

  • Use Resin's host deploy directory
  • Create an explicit host with the <host> tag in resin.xml

The host deploy directory allows you to create a directory structure such as:


/var/www/hosts/store.example.com/webapps

Then any applications deployed in this webapps directory will be served from Resin as store.example.com.

You may prefer to use an explicit <host> tag in your resin.xml. This approach allows you to create a custom directory structure and make your hosts explicit in configuration.

SECURITY

Resin provides two aspects of security for web applications: authorization and authentication.

Authentication

Resin provides an authentication framework that allows applications to authenticate users from a wide variety of sources including LDAP, a JDBC database, JAAS, or a simple XML database.

Authenticator Description
XmlAuthenticator For basic applications with few users (such as the Resin administration console).
LdapAuthenticator Can reference an LDAP database for users and passwords. Specify a distinguished name prefix and/or suffix to select users.
JdbcAuthenticator Specify a table and columns against which users, passwords, and roles will be authenticated.
JaasAuthenticator Use any JAAS plugin to authenticate users.

For example, to configure the XmlAuthenticator, you might add this XML to your resin.xml or resin-web.xml:


<web-app xmlns=“http://caucho.com/ns/resin” xmlns:resin=“urn:java:com.caucho.resin”>
	<resin:XmlAuthenticator>
		<password-digest>md5-base64</password-digest>
		<user name='myuser' password='IXiMnz7P2cJU18MSJjKiaA=='>
			<role>user</role>
			<role>foo</role>
		</user>
	</resin:XmlAuthenticator>
</web-app>

Authorization

While Resin supports the Servlet standard <securityconstraint> mechanism, it also provides an easy-to-use, yet powerful alternative that is integrated with the Rewrite Dispatch architecture. This integration makes it possible to handle requests for authorized and unauthorized users with custom logic. As an example, you may want to allow all accesses to an “/admin” application only if the user is in the proper “admin” role:


<web-app xmlns=“http://caucho.com/ns/resin” xmlns:resin=“urn:java:com.caucho.resin”>
	<resin:Forbidden regexp=“^/admin”>
		<resin:Not>
			<resin:IfUserInRole role=“admin”/>
		</resin:Not>
	</resin:Forbidden>
	<resin:Dispatch regexp=“^/admin”>
		<resin:IfUserInRole role=“admin”/>
		<resin:AddHeader name=“Cache-Control” value=“no-cache”/>
	</resin:Dispatch>
</web-app>

Notice that we also changed the caching behavior of the response to indicate that browsers should not cache this secure content.

DEVELOPING WITH RESIN

Eclipse Integration

Resin features a development plugin for Eclipse based on the WTP framework. With this plugin, developers have all of the facilities of the WTP environment with the ability to deploy to Resin using a variety of file system and remote deployment options.

New server runtime dialog

The plugin includes built-in configuration files for the development environment, but you can use any configuration file as well.

To download and install the Eclipse plugin for Resin, add http://caucho.com/eclipse as an update site within Eclipse and install the Caucho Resin plugin.

Testing Resin

Resin features an embedded server interface which can be used in test frameworks such as JUnit. The following example code shows how this API can be used to test a service injected using Resin CDI implementation, CanDI:


package qa;
import org.junit.*;
import org.junit.runner.RunWith;
import static org.junit.Assert.*;
import com.caucho.junit.ResinBeanContainerRunner;
@RunWith(ResinBeanContainerRunner.class)
@TestConfiguration(beanXML="beans-test.xml")
public class AccountServiveTest {
	@Inject
	private AccountService accountService;
	@Test
	public void testGetAccount()
		throws Exception
	{
		Account account = accountService.getAccount(1007);
		assertNotNull(account);
	}
}

CLUSTERING RESIN

Resin provides clustering capabilities for both traditional clusters and cloud deployments. This functionality includes:

  • Smart load balancing
  • Distributed session replication
  • Distributed object caching
  • Dynamic server addition and removal
  • Distributed application deployment

To get started with Resin clustering, you can add configurations to a <cluster>:


<resin xmlns=“http://caucho.com/ns/resin” xmlns:resin=“urn:java:com.caucho.resin”>
	<cluster id=“app-tier”>
		<server id=“app-a” address=“192.168.0.10” port=“6800”/>
		<server id=“app-b” address=“192.168.0.11” port=“6800”/>
		...
	</cluster>

The default resin.xml file has distributed sessions already enabled, so by adding these servers you've already got a cluster that can share data.

The start up procedure for Resin changes a bit when you have a cluster. When you have multiple servers configured in your resin.xml, you need to specify which of the servers you will use:


$ java -jar $RESIN_HOME/lib/resin.jar -server app-a start

In this case, you would run this command from the machine with the network interface assigned the IP 192.168.0.10, as per the configuration above.

Load Balancing

Once you've got a backend cluster set up as we did above, you'll probably want to add load balancing. In the same resin.xml as the app-tier cluster, add the following cluster configuration for a web-tier:


<resin xmlns=“http://caucho.com/ns/resin” xmlns:resin=“urn:java:com.caucho.resin”>
	<cluster id=“app-tier”>
	...
	</cluster>
	<cluster id=“web-tier” root-directory=“web-tier”>
		<server id=“web-a” address=“123.45.67.89” port=“6800”>
			<http address=“*” port=“80”/>
		</server>
		<host id=“”>
			<web-app id=“/”>
			<resin:LoadBalance regexp=“” cluster=“app-tier”/>
		</web-app>
		</host>
	</cluster>
</resin>

This configures a third Resin instance that will load balance requests from the outside world back to the app-tier servers. Because the <resin:LoadBalance> tag is part of the Rewrite Dispatch architecture, you can route load balanced requests with custom dispatch rules.

About The Author

Photo of Emil Ong

Emil Ong

Emil Ong is the Chief Evangelist and a lead developer of Caucho Technology. He comes from an academic research background, having studied security, systems, and peer-to-peer technology to gain his M.S. in Computer Science at UC Berkeley. When Emil joined Caucho in 2006, he began by working on Quercus, Caucho's 100% Java implementation of PHP, and Resin, Caucho's screamingly fast Java application server. In 2007, Emil became the Chief Evangelist of Caucho, adding public speaking engagements, community management, and press relations to his engineering duties. Emil is based in the San Francisco Bay Area.

Recommended Book

EJB 3 in Action

A comprehensive tutorial on EJB 3 co-authored by Caucho's Reza Rahman, this book features code samples, performance tips, and design patterns for using EJB and JPA. Novice and experienced Java programmers alike will find this book useful and informative.


Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

Getting Started with BlazeDS

By Shashank Tiwari

8,910 Downloads · Refcard 75 of 151 (see them all)

Download
FREE PDF


The Essential BlazeDS Cheat Sheet

Adobe BlazeDS is an open source software that facilitates effective integration of Flex and Java. It enables remote procedure calls and message exchanges between the platforms, helping couple together rich and engaging Flash platform-based interfaces and robust enterprise servers. This DZone Refcard on Adobe BlazeDS is the perfect place to start for any developer looking to learn more about using this tool. Getting Started with BlazeDS illustrates some of the most important features of the software and introduces the reader to what BlazeDS is and how to get started.
HTML Preview
Getting Started with BlazeDS

Getting Started with BlazeDS

By Shashank Tiwari

ABOUT BLAZEDS

Adobe BlazeDS is an open source software that facilitates effective integration of Flex and Java. It enables remote procedure calls and message exchanges between the two platforms thereby helping couple together rich and engaging Flash platform based interfaces and robust enterprise servers. Being open source, BlazeDS is freely available and can be downloaded from http://opensource.adobe.com/wiki/display/blazeds/BlazeDS/.

This Refcard provides a quick overview of BlazeDS. It attempts to illustrate some of the most important features of the software and therefore acts as a starting point for developers who are interested in the subject. More details, with exhaustive sets of examples, are available in the book Professional BlazeDS (Wiley/Wrox, 2009).

WHAT EXACTLY IS BLAZEDS?

Although it's mentioned upfront that BlazeDS helps connect Flash platform applications to Java, it's important to define it a bit further. Therefore both its behavioral and structural aspects are tersely listed in this section.

Behavioral Definition

BlazeDS enables and facilitates:

  • Invocation of remote Java methods from a Flex application.
  • Translation of Java objects returned from a server, in response to the remote method call, to corresponding AS3 objects.
  • Translation of AS3 objects sent by a Flex application to corresponding Java objects for passing them in as method call arguments.
  • Communication between a Flash Player instance and a Java server over a TCP/IP-based application-level binary protocol.
  • Near real-time message passing between a Flex application and a Java server.
  • Management of the communication channels between Flex and Java.
  • Management of connection types between Flex and Java.
  • Provision for adapters for communication with serverside Java artifacts like JMS queues, and persistence layers like Hibernate and JPA. (Some of these are in-built, and some can be obtained from open source projects or can be custom built.)
  • Pushing data from the server to the client on the server's initiative and not as a response to a request.

Structural Definition

BlazeDS is a:

  • Java web application that leverages the Java Servlets specification.
  • Web application that runs within a Java Servlet container or a Java application server, for example Apache Tomcat, JBoss AS, IBM Websphere or BEA (now Oracle) Weblogic.
  • Set of services that can be managed using JMX agents.
  • Remoting and messaging program that can be extended by using its Java API.
  • Program that intercepts all communication between a Flash Player and a Java server.
  • Configurable web application that can be clustered and used in cases that desire a higher than normal performance. (The in-built data push mechanism has a few limitations as far as high throughput and high volume goes but there are ways to get around this shortcoming.)

With the definition of BlazeDS firmly in place, it's worthwhile to explore Flex client and Java server integration in the larger context of combining the two platforms. Look at the next two figures for some insight into the context.

Flex/Java integration

Flex/Java integration

INSTALLING BLAZEDS

Installing BlazeDS is as simple or as complex as deploying a web application in a Java servlet container. BlazeDS works seamlessly in most mainstream open source and commercial Java servlet containers, including JBoss, Jetty, Tomcat, Oracle Weblogic and IBM Websphere.

You can get both compiled and source versions of the software. In addition, one of the binary versions comes in the form of a turnkey distribution that includes a configured copy of the Apache Tomcat Servlet container within the bundle. For beginners, it's convenient, appropriate and advisable to get the latest release version of the binary turnkey distribution.

CONFIGURING BLAZEDS

Simply put, at the heart of BlazeDS is a Servlet that bootstraps the infrastructure that intercepts all calls between a Flex client and the BlazeDS instance. This Servlet, called MessageBrokerServlet, uses artifacts like channels, endpoints, and adapters to enable proxy, remoting, and messaging services. A default configuration file, called services-config. xml, which lies in the WEB-INF/flex folder of the BlazeDS installation, defines these artifacts and their relationships in the context of the MessageBrokerServlet.

Channels and endpoints connect a Flex client to a BlazeDS server. They are the primary components that enable communication between these two entities. Endpoints reside at the BlazeDS end. Flex clients use channels to connect to these endpoints.

The BlazeDS endpoints are Servlet-based endpoints. Each endpoint defines a type and format of communication. For example, endpoints exist for simple AMF data exchange, polling AMF, and AMF data streaming.

Analogously, the Flex client defines a set of channels that vary depending on the type and format of communication. For example, the HTTPChannel facilitates communication over non-binary AMF format, AMFX (AMF in XML), and the AMFChannel enables standard binary AMF-based communication.

Matching endpoints and channels are paired, and that's when a Flex client and BlazeDS server talk to each other. The binding of channels and endpoints to their implementation classes and their pairing is done in the services-config.xml configuration file.

In addition to the endpoints, BlazeDS includes adapters that provide the critical compatibility between the core BlazeDS Servlet that talks to Flex and a server-side resource such as a JMS resource, a persistent data store, or an Object-Relational mapping layer. Adapters are also configured in services-config. xml.

The configuration file services-config.xml, in BlazeDS, is logically split into four configuration files.

The top level and the first of these four is services-config.xml itself. The other three are as follows:

  • remoting-config.xml
  • proxy-config.xml
  • messaging-config.xml

Remoting-config, proxy-config and messaging-config contain configuration pertaining to remote procedure calls, proxy services and message services respectively

Explaining every bit of the configuration is beyond the scope of this Refcard and is therefore not included. Only a couple of quick examples are shown to give you a flavor the typical configuration elements. Detailed explanations are available in the book entitled Professional BlazeDS (Wiley/Wrox, 2009), which I spoke about earlier.

A channel can be configured as follows:


<channel-definition id="my-amf" class="mx.messaging.channels.AMFChannel">
	<endpoint url="http://{server.name}:{server.port}/{contex.root}/messagebroker/amf"
	class="flex.messaging.endpoints.AMFEndpoint"/>
</channel-definition>

An adapter can be configured as follows:


<adapters>
<adapter-definition id="java-object"
class="flex.messaging.services.remoting.adapters.JavaAdapter"
default="true"/>
</adapters>

With these brief configuration examples in place, let's explore BlazeDS's pull-based (or request-response based) communication abilities. The following sections include a few more in context configuration illustrations.

PULL-BASED COMMUNICATION

Off the shelf, the Flex framework includes three methods of pull-based communication and data interchange with external data sources:

  • HTTP request-response
  • Web services
  • Remote procedure calls involving objects

A possible remoting configuration could be as follows:


<service id="aRemotingService"
class="flex.messaging.services.RemotingService">
<adapters>
<adapter-definition id="java-object"
class="flex.messaging.services.remoting.adapters.JavaAdapter"
default="true"/>
</adapters>
<default-channels>
<channel ref="myAMFChannel"/>
</default-channels>
<destination id="myPOJODestination">
<properties>
<source>myPackage. MyPOJO</source>
<scope>session</scope>
</properties>
</destination>
</service>

Using BlazeDS's remoting you are enabled with automatic translation between the Java and ActionScript3(AS3) data types, marshalling, un-marshalling, serialization and deserialization across the endpoints and channels.

BlazeDS's remoting capabilities provide fast and efficient data transmission between a Flex client and a Java server with the help of the binary Action Message Format (AMF) protocol and the built-in endpoints, channels and adapters to support it. Adapters make it possible to hook-up specific server side entities. The JavaAdapter included in the configuration example, is a built-in adapter for plain Java objects, which are also sometimes referred to as POJOs.

For accessing managed entities like Spring Beans or Enterprise Java Beans (EJBs), you can use the JavaAdapter as the translator but you also need a custom factory to help you access these objects, as managed objects reside in a namespace separate from the one that BlazeDS uses to instantiate its own objects.

JavaAdapter diagram

Besides, remoting BlazeDS can also act as a proxy server and help access data from domains that are not explicitly trusted via a crossdomain.xml security definition. Among others, the BlazeDS proxy capabilities have three important use cases including:

  • Access control to proxy destinations
  • Logging of all proxy traffic
  • Handling of HTTP errors

Proxy diagram

COMMUNICATING IN REAL-TIME

Data pushing in a web application context implies the server sending data to a browser-based client without the client necessarily requesting it.

Real-Time Communication diagram

Long polling can provide near real-time data push by waiting till the response from the server is ready to be dispatched. It's not scalable though, as it blocks connections.

Direct connectivity over Real Time Messaging Protocol (RTMP) provides a better scalable connectivity over long polling but RTMP is not available with BlazeDS yet! The RTMP specification was proprietary until the beginning of this year and has most recently been opened up to public. Its inclusion into future versions of BlazeDS is anticipated.

Usage of Java NIO provides for scalable connections as blocking connections are replaced by non-blocking asynchronous counterparts. BlazeDS does not include Java NIO implementations for its communication channels but it's not very difficult to include one. The open source dsadapters project (http://code.google.com/p/dsadapters/) aims to provide extensions to BlazeDS that include java NIO based channels, specialized factories, filters and advanced adapters, a lot of which will be available in the near future.

Messaging systems involve two typical messaging domains: point-to-point and publish-subscribe. BlazeDS can logically support both messaging domains. It has first-class support for publish-subscribe messaging domain.

Messaging diagram

BlazeDS based message services leverage two types of built-in adapters, namely:

  • JMSAdapters
  • ActionScriptAdapters

The JMSAdapter connects with JMS server side messaging systems. The ActionScriptAdapter helps route messages between Flex clients via the server. Therefore JMSAdapter comes handy when Flex clients are wired up to send and receive messages to and from enterprise systems that use JMS, whereas ActionScriptAdapter is useful for building systems like chat applications to help communicate between two Flex clients.

For effective near real-time messaging use one of the following channels:

AMFChannel/Endpoint with Polling

The client sends a recurring request to the server at a predefined frequency. Polling is very resource and network intensive.

AMFChannel/Endpoint with Long Polling

The channel issues polls to the server to fetch data but if no data is available it waits until data arrives for the client or the configured server wait interval elapses. For high frequency updates this configuration has the overhead of a poll roundtrip for every pushed message and therefore messages can be queued between polls. The Servlet API, prior to its current version 3, utilizes blocking I/O in which case long polling soon exhausts connections as multiple concurrent clients utilize the channel.

Streaming AMFChannel/Endpoint

This channel opens an HTTP connection between the server and client, over which the server sends an unending response of messsages. This channel avoids the overhead of polling and keeps the connection open for the entire scope of communication between the client and the server. Like AMFChannel with long polling, the constraints of blocking Servlet I/O can exhaust connections as multiple concurrent clients utilize the channel.

JPA AND HIBERNATE WITH FLEX

Object Relational Mapping (ORM) and persistence frameworks like JPA and Hibernate facilitate effective handling of relational data sources with Java applications. Extending the scope of these frameworks to include interactions with Flex clients allows seamless persistence management.

Off-the-shelf BlazeDS has no special features to support JPA and Hibernate. However a number of open source projects provide adapters to combine JPA and Hibernate with Flex. Gilead (http://noon.gilead.free.fr/gilead/) and dphibernate (http://code.google.com/p/dphibernate/) are two such open source adapters. The dsadapters project (http://code.google.com/p/dsadapters/) is also in the process of releasing a robust open source JPA/Hibernate adapter.

BlazeDS is capable of serializing and transforming Java-based objects to their AS3 counterparts so that may make you wonder why wiring up JPA and Hibernate entities and their collections needs any special handling. The reason for this special need arises because of the way BlazeDS serializes data across the wire.

BlazeDS has a set of endpoints where a Flex application channel sends requests up to BlazeDS that resides within a Servlet container or an application server. Responses from BlazeDS follow the route back up from the endpoint to the channel. On endpoints that support translation and serialization between AS3 and Java (or even web services), a serialization filter is defined to intercept calls to the endpoint. When an incoming or outgoing message hits the filter, serialization and deserialization occur. During serialization, the serializer eagerly fetches all the JPA and Hibernate persistent objects and sends them across the wire. The JPA and Hibernate proxies are replaced with the data that they stand in place of. This breaks the lazy loading semantics, rendering the idea of proxies useless. Therefore, any Hibernate adapter needs to preserve the proxy characteristics while keeping the standard behavior of the essential serialization and deserialization mechanism between Flex and Java intact.

A common design pattern used to solve this problem is what I like to call the "Clone and Merge Transfer Pattern."

Clone and Merge Transfer Pattern diagram

INTEGRATING WITH SPRING FRAMEWORK

The Spring BlazeDS integration project, also called Spring Flex, is a joint effort between SpringSource and Adobe to create a version of BlazeDS that works seamlessly with the Spring Framework. Prior to the existence of this project developers relied on custom factories to integrate Spring and BlazeDS.

BlazeDS is a Java Servlet based web application, so it integrates and works with the Spring framework facility that addresses the web layer. The core Spring framework web layer implements an MVC framework and is called Spring MVC. Spring BlazeDS integrates smoothly with Spring MVC.

A MessageBroker component sits at the heart of BlazeDS. All messages in BlazeDS are routed through the MessageBroker. In standard BlazeDS deployments, the MessageBrokerServlet is declared in WEB-INF/web.xml as follows:


<servlet>
<servlet-name>MessageBrokerServlet</servlet-name>
<servlet-class>flex.messaging.MessageBrokerServlet</servlet-class>
<init-param>
<param-name>services.configuration.file</param-name>
<param-value>/WEB-INF/flex/services-config.xml</param-value>
</init-param>
<init-param>
<param-name>flex.write.path</param-name>
<param-value>/WEB-INF/flex</param-value>
</init-param>
<load-on-startup>1
</servlet>
<servlet-mapping>
<servlet-name>MessageBrokerServlet</servlet-name>
<url-pattern>/messagebroker/*</url-pattern>
</servlet-mapping>

In Spring MVC, all incoming requests are handled by a Front controller Servlet, called the DispatcherServlet. This means you must have a way that all requests coming from the Flex client and intended to be handled by the BlazeDS instance are routed by the DispatcherServlet to the MessageBroker. This is exactly what the Spring BlazeDS project implements.

In addition, the Spring BlazeDS project also defines an XML Schema and associated infrastructure so that BlazeDS artifacts can be configured within Spring configuration files as Spring managed objects.

As an example a default BlazeDS message broker configuration can then be like so:


<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:flex="http://www.springframework.org/schema/flex"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://www.springframework.org/schema/flex
http://www.springframework.org/schema/flex/spring-flex-1.0.xsd">
<flex:message-broker/>
</beans>

More about the possible configurations can be learned online at http://www.springsource.org/spring-flex

With the help of this project it becomes easy to configure Spring Beans as server side remoting counterparts of Flex clients.

For example a service class configured as a Spring Bean as follows:


<bean id="myService" class="myPackage.MyService" />

Can easily be used as a remoting destination simply by specifying a configuration as follows:


<flex:remoting-destination ref="myService" />

There is plenty more, including use of annotations, possible for configuring Spring Beans as remoting destinations and you may want to learn more online from the Spring BlazeDS project site mentioned earlier in this paragraph.

Besides, remoting the Spring messaging and security benefits also get extended to Flex applications. In a Spring BlazeDS server, three types of message service components can interact with the Flex message service. The messaging service in Flex itself is agnostic to the messaging protocol used on the server side. Therefore, multiple server-side messaging alternatives easily work with Flex messaging. The three alternative serverside message services in Spring BlazeDS are:

  • Native built-in BlazeDS AMF messaging
  • JMS messaging using the Spring-spec JMS components
  • Spring BlazeDS messaging integration

EXTENDING BLAZEDS

BlazeDS exposes a public Java API. You can leverage that API to customize the behavior of BlazeDS resources.

The flex.messaging.services.ServiceAdapter abstract class sits at the root of the hierarchy. All built-in adapter classes inherit from the ServiceAdapter abstract class. Abstract classes not only define a contract like interfaces do but also define behavior through partial method implementations. The ServiceAdapter class states the base behavioral characteristics of a BlazeDS adapter.

You can create custom BlazeDS adapters by extending either the ServiceAdapter or one of its subclasses like the JavaAdapter or the MessagingAdapter classes.

MAKING BLAZEDS APPLICATIONS SCALABLE

In web scale applications scalability is an important criteria for success. BlazeDS instances can be clustered and the following techniques can be applied to make applications scale better:

Clustering

Clustered instances of BlazeDS share data and messages across instances. Therefore, message producers connected to a clustered server instance can send messages to message consumers connected to a different server within the same cluster. Besides sharing state information and routing information through the clustered servers, clustering provides for channel failover support. Clustered of BlazeDS instances is powered by JGroups.

Data compression

BlazeDS is a Java Servlets based web application. As data from a BlazeDS server leaves for the Flex client, it can be intercepted using the familiar Java Servlet filter mechanism. Servlet filters can intercept requests to and responses from a Java Servlet. In order to zip the data transferred from the server to the client, a filter can be created to manipulate and compress the outgoing response.

Data format optimization

AMF3 facilitates a very efficient way of binary transmission of data between the server and the client. In addition, you can choose to go with a text-based format that could be well structured like XML or delimited like comma-separated or tabdelimited text. JSON (JavaScript Object Notation) and AMFX, where AMF is transmitted in XML, are also options.

Robust connection

Non-blocking channels allow for greater number of connections to be served provided they are not all active at the same time always. Life Cycle Data Services uses NIO channels for connection scalability. You can include the same robustness in BlazeDS as well.

Service orientation

Service oriented architecture (SOA) patterns that apply well when creating Flex and BlazeDS applications are:

  • Concurrent contracts
  • Cross-domain utility layer
  • Functional decomposition
  • Rules centralization
  • State repository

Caching

If the accessed data is not changing during the course of its reuse it always makes sense to cache it. Caching is a timetested way of increasing performance by avoiding data fetches across the network and using pre-fetched local data instead.

Resource Pooling

Many of these external systems and libraries, such as messaging infrastructure, database connections and stateless business services, lend themselves to pooling. When resources are pooled, they are shared over multiple clients. BlazeDS has acces to all resource pooling strategies that any Java EE web application running in an application server has.

Workload distribution

Distributing work optimally between a client and its server is an important challenge when architecting RIA. BlazeDS remoting services optimally combine a Flex client and a Java server and allow a developer to distribute workload across the wire in ways without necessarily imposing the overheads that loose coupling like XML based interactions over HTTP and web services do.

CONCLUSION

With no upfront costs and a great set of features BlazeDS is a compelling piece of software for most serious Rich Internet Application (RIA) stacks that involve Flex and Java.

About The Author

Photo of Shashank Tiwari

Shashank Tiwari

Shashank Tiwari is a Managing Partner & CTO at Treasury of Ideas, a technology driven innovation and value optimization company. As an experienced software developer and architect, he is adept in a multitude of technologies. He is an internationally recognized speaker, author and mentor. As an expert group member on a number of JCP (Java Community Process) specifications he has been actively participating in shaping the future of Java. He is also an Adobe Flex Champion and a common voice in the RIA community. Currently, he passionately builds rich high performance scalable applications and advises many on RIA and SOA adoption. His clients range from large financial service corporations to brilliant startups, whom he helps translate cutting edge ideas into reality. He is also actively engaged in training and mentoring developers and architects in leading edge technology. He is the author of a number of books and articles, including Advanced Flex 3 (Apress, 2008) and Professional BlazeDS (Wiley, 2009). He lives with his wife and two sons in New York. More information about him can be accessed at his website (www.shanky.org).

Recommended Book

Book cover

This informative resource provides you with detailed examples and walkthroughs that explain the best practices for creating RIAs using BlazeDS. You'll begin with the essentials of BlazeDS and then more on to more advanced topics. Along the way, you'll learn the real-world concerns that surround enterprise-based Java and Flex applications.


Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

Essential PostgreSQL

By Leo Hsu and Regina Obe

17,243 Downloads · Refcard 71 of 151 (see them all)

Download
FREE PDF


The Essential PostgreSQL Cheat Sheet

PostgreSQL is the world’s most advanced open-source database. It runs on a number of platforms and is simple to install. It also contains enterprise features such as advanced spatial support via PostGIS, windowing functions, and table partitioning. Targeted at both novices and professionals alike, this DZone Refcard will help you quickly navigate some of PostgreSQL’s most popular features as well as its hidden gems. It will cover topics such as configuration, administration, backup, language support, and advanced SQL features.
HTML Preview
Essential PostgreSQL

Essential PostgreSQL

By Leo Hsu and Regina Obe

About Postgresql

PostgreSQL is the world’s most advanced open source database. It runs on numerous platforms: Linux, Unix, Windows, Mac OSX. It is simple to install, fast, sports enterprise features such as advanced spatial support via PostGIS, windowing functions, and table partitioning. It supports almost all SQL-92, SQL:1999, SQL:2003 and many SQL:2006 and SQL:2008 standards. In addition to its enterprise features, it has the added benefit of supporting numerous languages for authoring stored functions and an extensible procedural language architecture to introduce new languages.

Targeted at novices and professionals alike, this Refcard will help you quickly navigate some of PostgreSQL’s most popular features as well as its hidden gems. It will cover topics such as configuration, administration, backup, language support, and advanced SQL features. Items marked with [8.4] were introduced in PostgreSQL 8.4.

Configuration

PostgreSQL uses three configuration files to control overall operations. You can find these files in the initialized data cluster (the folder specified during the initialization process using initdb -d).

Hot Tip

All these can be edited with a text editor. They can be edited via PgAdmin III if contrib/adminpack.sql is installed in master postgres db.
File Purpose
postgresql.conf Controls the listening port, IP, and default query planner settings,memory settings, path settings, and logging settings. Can be queried via pg_settings database view.
pg_hba.conf Controls the authentication models used by PostgreSQL and can be set per user, per database, per IP range or a combination of all.
pg_indent.conf Controls mapping of an OS user to a PostgreSQL user.

postgresql.conf

Many of these service settings can be overridden for each session, database, or user/role.

Option Description
listen_addresses Use ‘*’ to listen on all IPs of the server, ‘localhost’ to just local or a comma separated list of IPs to listen on.
port Defaults to 5432, but can be changed to allow multiple postgresql daemon clusters/versions to coexist.
search_path List of default schemas that don’t need schema qualification. First schema is where non-schema qualified objects are created.
constraint_exclusion Options (on, off, partial). Partial was introduced in 8.4 and is the new default. Allows planner to skip over tables if constraint ensures query conditions can not be satisfied by the table. Mostly used for table partitioning via table inheritance.

pg_hba.conf

PostgreSQL supports many authentication schemes to control access to the database. The pg_hba.conf file dictates which schemes are used based on the rules found in this file. You can mix and match various authentication schemes at the same time. The rules are applied sequentially such that the first match fitting a connection is the one that is used. This is important to remember because if you have a more restrictive rule above a less restrictive, then the more restrictive is the one that trumps.

The most commonly used authentication schemes are trust (which allows connections without a password) and md5 which authenticates with md5 encrypted passwords. Others include: reject, crypt, password (this is plain text), krb5, ident (authenticate simply by identity of user in OS), pam, ldap.

The example pg_hba.conf entries below allows all local connections to connect to all databases without a password and all remote connections to authenticate via md5.

# TYPE DATABASE USER CIDR-ADDRESS METHOD
host all all 127.0.0.1/32 trust
host all all 0.0.0.0/0 md5

Data types

PostgreSQL has numerous built-in types. In addition, you can define custom types. Furthermore, all tables are considered to be types in their own right, and can therefore be used within another table’s column. Below are the common built-in types.

Data and Time Types

Type Description
Date The date is a datatype to represent dates with no time. Default representation is ISO 8601 e.g. ‘YYYY-MM-DD’. Use datestyle configuration setting to control defaults.
Timestamp This includes both date and time and is timezone-neutral. ‘2009-07-01 23:00’
Timestamp with time zone Timestamp with timezone. ‘2009-07-01 23:00:00-04’
Time Time without date ‘23:14:20’
Time with time zone ‘23:14:20-04’
Interval SELECT TIMESTAMP ‘2009-07-01 23:14:20’ + INTERVAL ‘4 months 2 days 10 hours 9 seconds’
Constituents of datetime, use date_part function to extract century, day, decade, dow (starts Sunday), doy, epoch, hour,isodow (day of week starts on Monday), minute, month, quarter, week, year

Numeric Types

Type Description
int, int8 4 byte and 8 byte integers
serial, serial4, serial8 Sequential integers. This can be used during table creation to specify auto-numbered fields.
numeric(s,p) Decimal numbers. s is scale and p is precision.
double precision Floating point numbers

String Types

Type Description
varchar(n) (aka character varying) Max of n characters, no trailing spaces
char(n) Padded to n characters.
text Unlimited text

Other Types

array Arrays in PostgreSQL are typed and you can create an array of any type. To define a column as an array of a specific type, follow with a brackets. Example: varchar(30)[]. You can also autogenerate arrays in an SQL statements with constructs such as. SELECT ARRAY[‘john’,’jane’]; SELECT ARRAY(SELECT emp_name FROM employees); [Pre 8.4] SELECT array_agg(emp_name) FROM employees; [8.4]
enum Enumerators, introduced in version 8.3 CREATE TYPE cloth_colors AS ENUM (‘red’,’blue’,’green’); When used in a table, you define the column as the name of the enum. Sorting is always in the order the items appears in the enum definition.
boolean true/false
bytea Byte array used for storing binary objects, such as files.
lo Large object. Stored in a separate system table with object ID reference to the large object. Useful for importing files from file system and storing and exporting back to file system.

Common Global Variables

Variable Description
CURRENT_TIMESTAMP, now() Returns current date and time with timezone
CURRENT_DATE Returns current date with no time
CURRENT_TIME Just time with no date
CURRENT_USER returns user name of session user

Commonly Used Func tions

DateTime Functions and Operators

Function Description
age(timestamp, timestamp) Returns an interval spanned by timestamp1 and timestamp2
age(timestamp) Difference from current time
date_part(text,timestamp), date_part(text,interval)

Example:
date_part(‘day’, timestamp ‘2009-07-04 11:05:45’) => 4
date_part(‘hour’, interval ‘560 minutes’) => 9

operators +, -, / (for intervals only) You can add (orsubtract) intervals to datetimes. You can perform subtraction between two datetimes. You can divide intervals into smaller intervals.
generate_series(timestamp, timestamp, interval) [8.4] Generate rows of timestamps. SELECT generate_series(DATE ‘2009-09-12’, DATE ‘2009-09-14’, INTERVAL ‘10 minutes’);

Text Functions and Operators

Function Description
|| (string || string, string || number) Concatenation
length Number of characters in string
lpad, rpad

Left and right pad
lpad(‘A’, 5, ‘X’) => XXXXA
rpad(‘A’, 5, ‘X’) => AXXXX

lower, upper, initcap Lower, upper, proper case.
md5 md5 hash
quote_ident Quotes keywords and expressions not suitable for identity when unquoted. quote_ident(‘in’) => “in” quote_ident(‘big’) => big
quote_literal Escapes both single and double quotes.
quote_nullable Similar to quote_literal but doesn’t quote NULL
split_part Takes a delimited string and returns the nth item. split_part(‘abc|def’, ‘|’, 2) =>def
strpos(text,subtext) Returns numeric position of subtext within text.
trim, ltrim, rtrim Trim spaces in string.

Array Functions

Function Description
||
Array concatenation ARRAY[1,2,3] || ARRAY[3,4,5] => {1,2,3,3,4,5}
unnest [8.4]
Converts an array to rows SELECT anum FROM unnest(ARRAY[1,2,3]) AS anum
array_upper(anyarray, dimension)array_lower(anyarray,dimension) returns upper/lower bound of the requested array dimension array_upper(ARRAY[ARRAY[‘a’], ARRAY[‘b’]],1) => 2
array_to_string(anyarray, delimiter_text) Converts an array to a text delimited by the delimeter.array_to_string(ARRAY[12,34], ‘|’) => 12|34

Other Functions

Function Description
generate_series(int1,int2,[step]) Returns rows consisting of numbers from int1 to int2 with [step] as gaps. Step is optional and defaults to 1.
min, max, sum, avg, count Common aggregates.
row_number,rank,dense_rank,percent_rank, lead, lag,first_value, nth_value [8.4] window functions

Database objects

Here is a listing of what you will find in a PostgreSQL server or database. An * means the object lives at the server level, not the database level.

Object Description
Databases* PostgreSQL supports more than one database per service/daemon.
Tablespaces* Logical representation of physical locations where tables are stored. You can store different tables in different tablespaces, and control data storage based on database and user/group role.
User/Group roles* Roles can have child roles. A role with login rights can be thought of as a user.
Languages These are the procedural languages installed in the database.
Casts PostgreSQL has the unique feature of having an extensible cast system. It has built-in casts, but allows you to define your own and override default casts. Casts allow you to define explicit behavior when casting from one object to another, as well as autocast behavior.
Schemas These are logical groupings of objects. One can think of them as mini-databases within a larger database. An object always resides in a schema.
Tables, Views Views are virtual tables that encapsulate an SQL SELECT statement. In PostgreSQL, tables can inherit from other tables and a query of a parent table will drill down to its children.
Rules Rules are tied to tables or views. They are similar to triggers except they can only be written in SQL and they rewrite a statement rather than actually updating directly. Views are implemented as SELECT rules with optional DO INSTEAD inserts/update rules to make them updateable)
Functions, triggers, and aggregates Can be written in any enabled language in the database, live in schemas. You can define your own custom aggregate functions. Trigger functions are special classes of functions that have OLD and NEW variables available that hold a pointer to the OLD and NEW data. Triggers are bound to table and a trigger function and a trigger function can be reused by many triggers.
Operators, operator classes, operator families Live in schemas. Many are predefined, but more can be added and allow you to define things such as +, =, etc. for custom data types.
Sequences Autocreated when defining columns as serial. In PostgreSQL, sequences are objects in their own right and can be shared across many tables.
Types Live in schemas. Don’t forget that you have the flexibility to create your own custom data types in PostgreSQL.

Tools

PostgreSQL comes bundled with several tools useful for administration and query writing.

Tool Description
psql Command-line client packaged with PostgreSQL. Good for automating SQL jobs, copying data, outputing simple html reports.
createdb, dropdb For creating and dropping a database from the OS shell.
PgAdminIII Popular graphical user interface packaged with PostgreSQL.
pg_restore Command-line tool for restoring compressed or tar backups.
pg_dump Command-line tool for doing backups. Great for automated backups
pg_dumpall Command-line tool for dumping all databases into a single backup.
pgAgent A daemon/service that can be downloaded from http://www.pgadmin.org/download/pgagent.php. Used for scheduling SQL jobs and batch shell jobs. Jobs can be added easily and monitored using the PgAdmin III job interface.
pgsql2shp Packaged with PostGIS (free spatial extender for PostgreSQL at http://www.postgis.org ). Command-line tool to dump spatial data out to ESRI shapefiles and DBFs.
shp2pgsql Packaged with PostGIS. Command-line tool that can load ESRI shapefiles or plain DBF files into PostgreSQL.
phpPgAdmin Not packaged with PostgreSQL, but downloadable or installable via PostgreSQL application stackbuilder. Similar to phpMyAdmin, it allows administration of PostgreSQL via web interface. Also downloadable separately at http://phppgadmin.sourceforge.net

PSQL Common Ta sks

PSQL is a command-line tool that allows you to run ad-hoc queries, scripts, and other useful database management routines. PSQL runs in both a non-interactive mode (straight from the OS shell prompt) and an interactive mode (PSQL terminal prompt). In both modes, the following arguments apply:

Argument Description
-d Database. Defaults to the user (via system identification if no user is specified).
-h Server host. Defaults to localhost if not specified.
-p Port. Defaults to 5432 if not specified.
-U Username you are trying to log in with. Defaults to system user name.

PSQL Non-Interactive Mode

Getting help


psql –help

Execute an SQL script stored in a file


psql –h localhost -U postgres –p 5432 –f pgdumpall.sql

Output data in html format


psql -h someserver -p 5432 -U postgres -d dzone -H -c “SELECT * FROM
pg_tips” -o mydata.html

Execute a single statement against a db


psql -U postgres –p 5432 -d dzone -c “CREATE TABLE test(some_id
serial PRIMARY KEY, some_text text);”

Execute an SQL batch script against a database and send output to file


psql -h localhost -U someuser -d dzone -f scriptfile.sql -o outputfile.txt

PSQL Interactive Mode

To initiate interactive PSQL, type


psql –U username –p 5432 –h localhost –d dzone

Once you are in the the psql terminal you can perform a myriad of tasks. Below are some of the common ones.

Quit \q
Cancel out of more screen :q
Help on psql commands \?
Help on SQL commands \h some command
Switch database \connect somedatabase
List all databases \l
\dtv p* List tables and views that start with p.
\du List user/group roles and their group memberships and server level permissions
\d sometable List columns, data types, and constraints for a table
\i somefile Execute SQL script stored in a file.
\o somefile Output contents to file.
Retrieve prior commands Use up and down arrows
\timing Toggle query timing on and off. When on, query output includes timing information.
\copy Copy from client computer to server and from server to client computer. Example: The following command string copies data to local client computer in CSV format with header.
\copy (SELECT * FROM sometable) TO ‘sometable.csv’ WITH HEADER CSV FORCE QUOTE

ad min tasks

Backup and Restore

Below are common backup and restore statements

Create a compressed backup


pg_dump -h someserver -p 5432 -U someuser -F c -b -v -f “somedb.
backup” somedb

Create a compressed backup of select tables


pg_dump -h localhost -p 5432 -U someuser -F c -b -f “somedb.backup”
-t “someschema.table1” -t “someschema.table2” -v somedb

Create a compressed backup excluding a particular schema


pg_dump -h localhost -p 5432 -U someuser -F c -b -f “somedb.backup”
-N someschema -v somedb

Restore a compressed backup


pg_restore –h localhost –d db_to_restore_to –U someuser somedb.backup

Restore select schemas from backup


pg_restore –h localhost –d db_to_restore_to –U someuser -n
someschema1 -n someschema2 somedb.backup


Output a table of contents from backup file


pg_restore -l -f “toc.txt” “somedb.backup”

Restore only items in the table of contents


pg_restore -h localhost -d db_to_restore -U someuser -L “toc.txt”
“somedb.backup”

Hot Tip

pg_dumpall currently only dumps to plain text sql. pg_dumpall backups must be restored with psql. For space savings and flexibility, use pg_dump. With pg_dump compressed and tar backups, you can selectively restore objects. You can not selectively restore with plain text backups.

Below are common switches used with pg_dump [D], pg_ restore [R], pg_dumpall [A]. These tools are packaged with PostgreSQL and are in the bin folder. They are also packaged with pgAdmin III and are in the PgAdmin III/1.10/ folder.

Switch Tool Description
b, --blobs D Include large objects in dump.
-d, --dbname=NAME R Specify name of database to restore to.
-F, --format=c|t|p D R Specify backup file format (c = compressed, t = tar, p = plain text). Plain text backups must be restored with psql.
-g, --globals-only A Dump only global objects (roles, schemas, tablespaces), no databases.
-g, --globals-only A Dump only global objects (roles, schemas, tablespaces), no databases.
-i D This will ignore the version of the command-line tool and will allow processing even if an older command-line is used to dump a newer PostgreSQL server.
-j, --jobs=NUM [8.4] R Use this multiple parallel jobs to restore. This is especially useful for large backups and speeds them up significantly in many cases.
-l, --list R Print summarized TOC of the archive.
-L, use-list=filename R Use TOC from this file for selecting/ordering output.
-n, --schema=NAME D R Dump/restore only select objects in schema(s).
-N, --excludeschema=SCHEMA D R Exclude from dump/restore named schema(s).
-r, --roles-only A Dump only roles, no database or tablespace.
-t, --table=NAME D Backup only named table(s) along with associated indexes,constraints, and rules.
-T, --excludetable=NAME D Exclude named table(s) from backup.
-v --verbose D R A Controls verbosity.

User Rights Management

These are SQL commands you can use to control rights. They can be run in the PSQL interactive, via an SQL script, or via PgAdmin.

Creat a new role with login rights that can create objects CREATE ROLE somerole LOGIN NOSUPERUSER INHERIT CREATEDB NOCREATEROLE;
Create a group role with no login rights and members inherit rights of role CREATE ROLE somerole NOSUPERUSER INHERIT NOCREATEDB NOCREATEROLE;
Add a role to another role GRANT somerole TO someotherrole;
Add a role to another role Example uses: GRANT SELECT, UPDATE ON TABLE sometable TO somerole; GRANT ALL ON TABLE sometable TO somerole; GRANT EXECUTE ON FUNCTION somefunction TO somerole; -- Grant execute to all users GRANT EXECUTE ON FUNCTION somefunction TO public;
Revoke rights REVOKE ALL ON TABLE sometabled FROM somerole;
Give insert/update rights to select columns [8.4] GRANT INSERT, UPDATE (somecolumn) ON sometable TO somerole;

Data Defini tion (DDL)

Many of the examples we have below use named schemas. If you leave out the schema, objects created will be in the first schema defined in the search_path and dropped by searching the search path sequentially for the named object.

Create a new database CREATE DATABASE somedatebase WITH OWNER = someuser;
Creat a schema CREATE SCHEMA someschema;
Changing database schema search path Sets the default schema to someschema ALTER DATABASE somedatabase SET search_path = someschema, public;
Dropping objects with no dependents A drop without a CASCADE clause will not drop an object if there are objects that depend on it, such as views, functions, and tables. For drop database you should be connected to a database other than the one you’re dropping. DROP DATABASE somedatabase; DROP VIEW someview; ALTER TABLE sometable DROP COLUMN somecolumn; DROP FUNCTION somefunction;
Dropping object and all dependents. (Use with caution.) DROP SCHEMA someschema CASCADE;
Create a table in a schema CREATE TABLE sometable (id serial PRIMARY KEY, name character varying(150), status boolean NOT NULL DEFAULT true);
Create a child table CREATE TABLE somechildtable (CONSTRAINT pk_somepk PRIMARY KEY (id)) INHERITS (someparenttable);
Create a check constraint ALTER TABLE sometable ADD CONSTRAINT somecheckcontraint CHECK (id > 0);
Create or alter a view CREATE OR REPLACE VIEW someview AS SELECT * FROM sometable [Prior to version 8.4 adding new columns to a view requires dropping and recreating]
Add a column to a table ALTER TABLE sometable ADD COLUMN somecolumn timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP;
Add a functional index to a table CREATE INDEX idx_someindex ON sometable USING btree (upper(somecolumn)); Create a new type CREATE TYPE sometype AS (somecolumn integer,someothercolumn integer[]);
Create a new type CREATE TYPE sometype AS (somecolumn integer,someothercolumn integer[]);
Create a trigger

CREATE OR REPLACE FUNCTION sometrigger()
		RETURNS trigger AS
		$$
		BEGIN
		IF OLD.somecolumn <> NEW.somecolumn OR (OLD.
		somecolumn IS NULL AND NEW.somecolumn IS NOT NULL)
		THEN
		NEW.sometimestamp := CURRENT_TIMESTAMP;
		END IF;
		RETURN NEW;
		END;
		$$
		LANGUAGE ‘plpgsql’ VOLATILE;

Add trigger to table CREATE TRIGGER sometrigger BEFORE UPDATE ON sometable FOR EACH ROW EXECUTE PROCEDURE sometriggerupdate();
Suppress redundant updates [8.4] A built in trigger that prevents updates that would not change any data. CREATE TRIGGER trig_01_suppress_redundant BEFORE UPDATE ON sometable FOR EACH ROW EXECUTE PROCEDURE suppress_redundant_updates_trigger();

Hot Tip

A table can have multiple triggers, and each trigger for a particular event on a table is run in alphabetical order of the named trigger. So if order is important, name your triggers such that they are sorted in the order you need them to run

Query and Upda te (DML )

These are examples that showcase some of PostgreSQL query features.

Retrieving data

View running queries SELECT * FROM pg_stat_activity;
Selecting the first record of each distinct set of data

-- this example selects the store and product
-- where the given store has the lowest price
-- for the product. This uses PostgreSQL
-- DISTINCT ON and an order by to resort
-- results by product_name.
SELECT r.product_id, r.product_name, r.product_price
FROM (SELECT DISTINCT ON(p.product_id) p.product_
id, p.product_name, s.store_name, i.product_price
FROM products AS p INNER JOIN inventory As i
ON p.product_id = i.product_id
INNER JOIN store AS s ON i.store_id = s.store_id
ORDER BY p.product_id, i.product_price) As r;

Using window function to number records (row_num)* SELECT ROW_NUMBER() OVER(ORDER BY p.product_price) As row_num, p.product_name FROM products;
Using window function to number records by category and price *

--numbering restarts for each category
SELECT ROW_NUMBER() OVER(PARTITION BY
p.product_category ORDER BY p.product_price),
p.product_category, p.product_price
FROM products AS p
ORDER BY p.product_category, p.product_price

Non-Recursive CTE with 2 CTE expressions. Note a CTE expression has only one WITH, each subexpression is separated by a , and the final query follows.
Example returns the lowest priced car in each category
	
WITH c AS
( SELECT country_code, conv_us
FROM country
), prices AS
(SELECT p.car, p.category, p.price*c.conv_us As us_price
FROM cars As p
INNER JOIN c
ON p.country_code = c.country_code
)
SELECT DISTINCT ON(category)
category, car, us_price
FROM prices
ORDER BY category, us_price;
	
Recursive CTE * inventory, gives full name which includes parent tree name e.g Paper->Color->Red->20 lbs

WITH RECURSIVE tree AS
(SELECT id, parentid,
CAST(item As text) As fullname
FROM products
WHERE parentid IS NULL
UNION ALL
SELECT p.id,p.parentid,
CAST(t.fullname || ‘->’
|| p.item As text) As fullname
FROM products As p
INNER JOIN tree AS t
ON (p.parentid = t.id)
)
SELECT id, fullname
FROM tree;

Adding and Updating Data

Insert statement with multirows --requires 8.2+ INSERT INTO tableA(id,price)VALUES (1,5.00), (1,5.25)
Insert statement from select only load items not already in table

INSERT INTO tableA(id,price)
SELECT invnew.id,invnew.price
FROM tableB As invnew LEFT JOIN tableA As invold ON
(invnew.id = invold.id)
WHERE invold.price IS NULL;

Cross update only update items for a particular store where price has changed

UPDATE tableA
SET price = invnew.price
FROM tableB AS invnew
WHERE invnew.id = tableA.id
AND NOT (invnew.price = tableA.price);

Insert from a tab delimited file no header COPY products FROM “/tmp/productslist.txt” WITH DELIMITER ‘\t’ NULL As ‘NULL’;
Insert from a comma delimited file with header row --these copy from the server’s file system COPY products FROM “/tmp/productslist.csv” WITH CSV HEADER NULL As ‘NULL’
Copy data to comma delimited file and include header --this outputs to the server’s file system COPY (SELECT * FROM products WHERE product_rating = ‘A’) TO ‘/tmp/productsalist.csv’ WITH CSV HEADER NULL As ‘NULL’;

Procedural Lan guages

PostgreSQL stands out from other databases in its extensive and extendable support for different languages to write database stored functions. It allows you to call out to libraries native to that language. We will list the key as well as some esoteric ones. The ones with * are preinstalled with PostgreSQL and can be enabled. Some require additional installs in addition to the language handler.

You can create set returning functions, simple scalar functions, triggers, and aggregate functions with most of these languages. This allows for languages highly optimized for a particular task to work directly with data without having to always copy it out to process as you normally would need to with a simple database storage device. Language handlers can be of two flavors trusted and untrusted. An untrusted language can access the file system directly.


CREATE PROCEDURAL LANGUAGE ‘plpythonu’ HANDLER plpython_call_handler;
CREATE OR REPLACE somename(arg1 arg1type)
RETURNS result_argtype AS
$$
body goes here
$$
LANGUAGE ‘somelang’;

Language Description Req
sql*(trusted) Enabled in all dbs. Allows to write simple functions and set returning functions in just sql. The function internals are visible to the planner so in many cases performs better than other functions since the planner can strategize how to navigate based on the bigger query. It is simple, fast, but limited in functionality. CREATE OR REPLACE FUNCTION prod_state(prev numeric, e1 numeric, e2 numeric) RETURNS numeric AS $$ SELECT COALESCE($1,0) + COALESCE($2*$3,0); $$ LANGUAGE ‘sql’ IMMUTABLE; none
c* Built in and always enabled. Often used to extend PostgreSQL e.g. postgis, pgsphere, tablefunc or for example introduce new windowing functions (introduced in PostgreSQL 8.4). Functions are referenced from a .so or .dll file. CREATE OR REPLACE FUNCTION summary(geometry) RETURNS text AS ‘$libdir/postgis-1.4’, ‘LWGEOM_summary’ LANGUAGE ‘c’ IMMUTABLE STRICT; none
plpgsql*(trusted)

Not always enabled but packaged so can be installed.
CREATE FUNCTION cp_upd(p_key integer, p_value
varchar)
RETURNS void AS
$$
BEGIN
IF EXISTS(SELECT test_id FROM testtable WHERE test_id
= p_key) THEN
UPDATE testtable
SET test_stuff = p_value
WHERE test_id = p_key;
ELSE
INSERT INTO testtable (test_id,
test_stuff)
VALUES(p_key, p_value);
END IF;
RETURN;
END;
$$
LANGUAGE ‘plpgsql’ VOLATILE;

none
plperl (trusted),plperlu (untrusted)

CREATE OR REPLACE FUNCTION use_quote(TEXT)
RETURNS text AS $$
my $text_to_quote = shift;
my $qfunc = $_SHARED{myquote};
return &$qfunc($text_to_quote);
$$ LANGUAGE plperl;

perl
plpythonu (untrusted)

CREATE FUNCTION fnfileexists(IN fname text) RETURNS
boolean AS
$$
import os
return os.path.exists(fname)
$$
LANGUAGE ‘plpythonu’;

python
plr

Good for doing advanced stats and plotting using R
statistical language.
CREATE FUNCTION r_quantile(float8[])
RETURNS float8[] AS
$$
quantile(arg1, probs = seq(0, 1, 0.25),
names = FALSE)
$$ LANGUAGE ‘plr’;

R
plsh (untrusted)

Allows to write in shell script
CREATE FUNCTION callscript(id integer) RETURNS text AS
$$
#!/bin/sh
wget -q “http://somesite.com/somepage?id=$1” >/dev/
null 2>&1
echo “done”
$$
LANGUAGE ‘plsh’ VOLATILE;

R

Common Procedural Tasks

Create a trigger and use in table


CREATE FUNCTION mytable_ft_trigger() RETURNS trigger AS $$
BEGIN
NEW.tsv :=
setweight(to_tsvector(‘pg_catalog.english’,
coalesce(new.field1,’’)), ‘A’) ||
setweight(to_tsvector(‘pg_catalog.english’,
coalesce(NEW.field2,’’)), ‘B’);
RETURN NEW;
END
$$ LANGUAGE plpgsql;
CREATE TRIGGER mytable_trigiu BEFORE INSERT OR UPDATE
ON mytable FOR EACH ROW EXECUTE PROCEDURE mytable_ft_trigger();

Return sets and use of out params


CREATE OR REPLACE FUNCTION
fn_sqltestmulti(param_subject varchar,
OUT test_id integer,
OUT test_stuff text)
RETURNS SETOF record
AS
$$
SELECT test_id, test_stuff
FROM testtable
WHERE test_stuff LIKE $1;
$$
LANGUAGE ‘sql’ STABLE;
--example
SELECT * FROM fn_sqltestmulti(‘%stuff%’);

Create an aggregate function


CREATE AGGREGATE sum(text) (
SFUNC=textcat,
STYPE=text,
INITCOND=’’
);
SELECT item, sum(category || ‘ ‘) as categories
FROM item_cat GROUP BY item;

About The Author

Photo of Regina

The wife and husband team of Leo Hsu and Regina Obe founded Paragon Corporation in 1997 specializing in database technology. Paragon Corporation works with numerous organizations to design, develop and maintain database and web applications. In 2002, Leo and Regina started to dabble in the growing field of spatial analysis and have become active participants in the on-going development of PostGIS which is a spatial extension of PostgreSQL. Regina is a member of the PostGIS core development team and Project Steering Committee.

Through Paragon Corporation, Leo and Regina have helped many clients using PostgreSQL, SQL Server and MySQL. Paragon Corporation takes on database projects in a wide range of industries and advocates database-driven development.

Leo and Regina met at MIT and both graduated with engineering degrees. Leo went on to obtain a master’s degree from Stanford University in Engineering of Economic Systems.

They maintain two sites: http://www.postgresonline.com -- provides tips and tricks for using PostgreSQL and http://www.bostongis.com - provides tips and tricks for using PostGIS, SQL Server 2008 Spatial and other open source and open GIS tools.

Email contact: lr@pcorp.us

URL: http://www.paragoncorporation.com

Recommended Book

PostGIS in Action

PostGIS in Action is the first book devoted entirely to PostGIS. It will help both new and experienced users write spatial queries to solve real-world problems. For those with experience in more traditional relational databases, this book provides a background in vector-based GIS so you can quickly move to analyzing, viewing, and mapping data. Advanced users will learn how to optimize queries for maximum speed, simplify geometries for greater efficiency, and create custom functions suited specifically to their applications. It also discusses the new features available in PostgreSQL 8.4 and provides tutorials on using additional open source GIS tools in conjunction with PostGIS.


Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

Download Your Free Grails Cheat Sheet Now!

New to Grails? Click here to download DZone's latest Refcard: Getting Started with Grails.

1 replies - 7428 views - 06/29/09 by Lyndsey Clevesy in Announcements

Getting Started with Grails

By Dave Klein

16,024 Downloads · Refcard 60 of 151 (see them all)

Download
FREE PDF


The Essential Grails Cheat Sheet

Grails is a full-stack web application framework built on top of popular open source frameworks such as Spring, Hibernate, Ant, and JUnit. This DZone Refcard takes you through a brief introduction of Grails, and helps you to set up your first Grails application in minutes. It serves as a handy reference manual for working with Grails Controllers, Services, Views and GSP taglibs. You also may enjoy these other DZone Refcardz on related topics: Spring Configuration and JUnit and EasyMock.
HTML Preview
Getting Started with Grails

Getting started with grails

By Dave Klein

Getting started with grails

Grails is a full-stack web application framework built on top of such tried and true open source frameworks as Spring, Hibernate, Ant, JUnit and more. By applying principles such as Convention over Configuration and Don't Repeat Yourself, and taking advantage of the dynamic Groovy programming language, Grails makes it incredibly easy to use these powerful tools. Grails doesn't reinvent the wheel; Grails makes a wheel that inflates itself and rolls where you want it to!

In case you are new to Grails, we'll start with a brief introduction, which should be enough to get you hooked and turn you into a Grails developer. That's when this Refcard will come in handy; it is a cheat sheet for Grails developers, a quick source for those things you keep having to go back to the docs to look up. Controllers, Services and Views with a detailed GSP taglib reference.

Installing Grails

Download the Grails archive from http://grails.org/download and extract it to a local directory. Set a GRAILS_HOME environment variable to that directory and add GRAILS_ HOME/bin to your path. (You also need a valid JAVA_HOME environment variable.) Now you're ready to go!

A Web App in the Blink of an Eye

To create a new Grails application, type:


$ grails create-app AutoMart

Now change to the AutoMart directory and create a domain class:


$ grails create-domain-class Car

Open AutoMart/grails-app/domain/Car.groovy and edit it, like so:


class Car {
	String make
	String model
	Integer year
}

Save this file, and run:


$ grails generate-all Car

Create-app and create-domain-class are Grails scripts. To see what other scripts are provided by Grails, run grails help from the command line.

You now have a complete working web application, with pages for creating, displaying, editing and listing Car instances. You can launch it with:


$ grails run-app

Grails runs on port 8080 by default. You can easily run on a different port like this:


$ grails -Dserver.port=9090 run-app

Grails

Figure 1: Create Car

Navigate to http:// localhost:8080/AutoMart and look around. Figure 1 and Figure 2 show a couple of the views Grails gives us 'out of the box.' You can also leave it running while you continue to develop. Just save your changes and refresh your browser. This rapid feedback is one of the strengths of Grails.

Grails_2

Figure 2: Car List

Grails Conventions

In our example the Car.groovy file that Grails created for us was placed in a directory called grails-app/domain. This is one of the many conventions in Grails. Placing source files in certain directories and naming them in certain ways can make magical things happen in a Grails application.

Domain Classes

Placing a Groovy class file in the grails-app/domain directory will turn it into a persistent domain class. Several properties and methods will be added to the class dynamically, and Grails will create a table based on the name of the class, with fields for each property.

Refcards_subscribe

Hot Tip

Be careful about adding methods beginning with 'get' to a domain class. Grails will consider that a property and try to persist it. You can avoid this by not giving the method a return type (ie. def getSomething(){...} )

Controllers

Grails controllers are simple Groovy classes with names ending in 'Controller' and residing in the grails-app/controllers directory. Controllers also receive several methods and properties dynamically. Any closure defined as a property in a Controller will become an action reachable by a URL in the following form: application/controller/action.

Views

For each controller in your application there will be a directory under grails-app/views/ named after the controller class (ie. grails-app/views/car). This directory is where your views (.gsp files) go. When a controller action is completed it will automatically attempt to render a view with the name of the action. So, when you call http://localhost:8080/AutoMart/car/list the list action will execute and render the list.gsp page. You can, of course render specific pages but convention can be a huge time-saver.

Services

A Groovy class with a name ending in 'Service' and residing in the grails-app/services directory becomes a Grails service and has built-in transaction handling and more.

TagLibs

Creating custom tags in Grails is so easy it should be illegal. Just create a Groovy class ending with 'TagLib' and place it in the grails-app/TagLib directory. Then define a closure property and write to the OutputStream called 'out' that is already given to you. You can also use the tag's attributes and body by simply declaring them.


class YourTagLib{
def saySomething = {attrs, body ->
  if (attrs.tone == 'loud')
	out << body().toUpperCase()
  elseif (attrs.tone == 'quiet')
	out << body().toLowerCase()
  else
	out << body()
}
You can use this tag in a .gsp like this:
<g:saySomething tone='loud'>
  I'm shouting now!
</g:saySomething>

You can use this tag in a .gsp like this:


<g:saySomething tone='loud'>
   I'm shouting now!
</g:saySomething>


I'M SHOUTING NOW!

There are no extra classes to create, no interfaces to implement, no TLDs to create. You could probably take up a new hobby with the time you'll save!

Testing

Unit Tests

Grails encourages unit testing by automatically creating stubbed out unit tests when creating artifacts (domain classes, controllers, services, and taglibs), and by including a powerful testing framework based on JUnit.

In Grails 1.1 (and earlier via a plugin), there are several classes inheriting from JUnit's TestCase class. Figure 1 shows these classes.

Grails Test Classes

Figure 3: Grails Test Classes

Here are some of the methods that are available to us in the Grails testing framework.

mockDomain(class, list)- Mocks the domain class and stores instances in the list. Provides the dynamic GORM methods to class.
mockFor(class, loose) '" Returns a mock of the class. loose determines whether mock has loose expectations or strict.
mockForConstraintsTest(class, list) '" Similar to mockDomain but works for domains or command objects and adds a validate method to test constraints.
mockController(controllerClass) '" Mocks a controller, adding the usual dynamic properties and methods.
mockTagLib(tagLibClass) '" Mocks a TagLib, adding the usual dynamic properties and methods.
mockLogging(class, enableDebug) '" Adds a mock logger to the class.

Do main Class Magic

Dynamic methods

Grails adds several methods to our domain classes at runtime. Here are some of the most commonly used domain class methods:

Method Description
get(id) Retrieves an instance by id
getAll([id,id,id...]) Retrieves multiple instances by ids
list() Retrieves all instances
listOrderBy*() Retrieves list of instances sorted by expression
findBy*() Returns first instance matching expression
findAllBy*() Returns all instances matching expression
count() Returns total number of instances
countBy*() Returns count of instances matching expression
save() Attempts to persist instance
validate() Validates an instance based on constraints
delete() Attempts to permanently remove instance
withTransaction Executes a closure within a transaction
withCriteria Executes a hibernate criteria using the CriteriaBuilder
hasErrors Returns true if instance has validation errors

Methods such as countBy*() and findBy*() are synthesized methods, made up of the root method name and a combination of domain class properties and the following comparators:

  • Equal (Implied default comparator)
  • NotEqual
  • LessThan
  • LessThanEquals
  • GreaterThan
  • GreaterThanEquals
  • Like
  • like (case insensitive Like)
  • InList
  • Between
  • IsNull
  • IsNotNull

Up to two properties with optional comparators can be combined with a logical operator for example:


Car.countByMakeAndAgeLessThan('Dodge', 4)
Car.findAllByMakeAndModelInList('Ford', ['Mustang', 'Explorer'])


Pagination and Sorting Parameters

The domain method list() takes four parameters which are used for sorting and pagination. These same parameters can be passed in a Map as the last parameter to the findAllBy* methods.

Parameter Description
max The maximum number of instances to return
offset The position to begin retrieving from
sort Domain property to sort by
order Whether to sort asceding or decending

Example: Car.list(max:10, sort:'year', order:'desc')

Constraint Validation

Either the save() or validate() methods shown above will trigger Grails' powerful data validation. Grails provides 17 built-in constraints plus the validation constraint for custom validation.

Many of these constraints also influence database schema generation; those are in shown in bold.

blank (true/false) allow an empty string value
nullable (true/false) allow nulls
max Maximum value of any type that implements java.lang.Comparable
min Minimum value of any type that implements java.lang.Comparable
size Uses a range to determine the upper and lower limits of a collection or a String
maxSize The maximum size of a collection or String
minSize The minimum size of a collection or String
range Uses a range to determine the limits of a numeric value
scale Rounds value to specified number of decimal places - Does not generate an Error
notEqual No comment
inList Value must be contained in supplied list
Matches Value must match supplied regular expression
Unique (true/false) Verifies uniqueness in database
url (true/false) Must be valid URL
email (true/false) Must be valid email address
creditCard (true/false) Must be valid credit card number
password (true/false) Must be valid password
Validator Takes a closure for custom validation. First parameter is value, second (if supplied) is the instance being validated

Here's an example of how constraints are declared in a domain class:


static constraints = {
   make()
   model()
   year()
   desription()
}

If validation fails during a call to save(), no exception is thrown. The failure is quietly recorded and stored in the instances errors property. To see if a domain class instance has validation errors, use the hasErrors() method in conjuction with the errors property, like so:


if (carInsance.hasErrors()){
   carInstance.errors.allErrors.each{ println it }
}

Relationships

GORM also makes relationships between different domain classes easier. Here we'll show how to implement the basic domain relationships.

One-to-one

one-to-one-figure

Unidirectional one-to-one is the simplest type of relationship. One class has a Reference to another. This is declared by a property of the type of another domain class:


class Car {
   Engine engine
}

A bi-directional one-to-one is the same thing but with each class in the relationship having a reference to the other.


class Driver {
    Car car
}
class Car {
    Driver driver
}

Usually in a situation like this you want to show ownership and have cascading updates based on that ownership. With Grails that just takes single line of code:


class Car{
   Driver driver
   static belongsTo = Driver
}

Now a Car belongs to a Driver and when the Driver is saved the Car will be saved too. If the Driver is deleted then the Car goes with him. Makes sense. belongsTo also works with unidirectional relationships.

One-to-many

one-to-one-figure

Figure 5: One-to-many

To declare a uni-directional one-to-many relationship just include a static hasMany property in the owning class.


class Driver {
   static hasMany = [cars : Car]
}

Now a Driver will have a Collection of Car instances called cars.

To make this bi-directional add a belongsTo property to the many class.


class Car{
Driver driver
   static belongsTo = Driver
}

Many-to-many

many-to-many-figure

Figure 6: Many-to-many

A many-to-many relationship can be declared by adding a hasMany to both classes involved and a belongsTo property to one of them.


class Driver {
   static hasMany = [cars : Car]
}
class Car {
   static hasMany = [drivers : Driver]
   static belongsTo = Driver
}

The three r's of co ntrollers

There are three possible conclusions to a controller action in Grails. You can return map of data or nothing and Grails will attempt to display the .gsp view with the same name as the action. You can also redirect to another url or action. The third way is to render something. This can be a view, a template, JSON, XML or just about anything that can be written to the response.

Redirect Method Parameters
controller Controller to redirect to. If action is not present the default action of the controller will be used.
action Action to redirect to. If controller is not present current controller will be used.
id Id to be passed, as params.id, to the redirect.
url URL to redirect to ('http://grails.org').
uri URI to redirect to ('/car/edit/1').
params A map of parameters to be passed to the redirect.
Render Method Parameters
text Text to rendered to the resonse
view A GSP view. Can include pat
template A template (partial GSP view). Often used with AJAX actions.
model A map containing data to be used by the view or template.
bean A single bean to used as the model for the view or template.
collection A collection of objects to be used as the model for the view or template.
builder Builder object to be used to render markup.
contentType Sets the content type of the response.
encoding Sets the encoding of the response

Services

Grails service classes are powerful and easy to use. Here's a few important things to remember when using services.

Transactions

Grails service classes are transactional by default. If you don't want that behavior, for example, if you are going to handle transactions with the dynamic domain class method withTransaction, you can turn it off with a single line:


class CarService {
   static transactional = false
}

Service Injection

Services can be injected into controllers, domain classes, or other services simply by declaring a property with the same name as the service class type but with the first letter lowercase:


class CarController{
def carService
}

You can also inject Services into plain old groovy objects, but it takes a couple more lines of code. First declare the service in your POGO.


class MyPogo {
  def carService
}

Then when you create an instance of your POGO do this:


import org.codehaus.groovy.grails.commons.ApplicationHolder
def myPogo = new MyPogo()
def ctx = ApplicationHolder.application.mainContext
ctx.autowireCapableBeanFactory.autowireBeanProperties(
   myPogo,
   AutowireCapableBeanFactory.AUTOWIRE_BY_NAME,
   false)


This will tell Spring to inject any Spring beans that are declared in the MyPogo class into the myPogo instance. Since Grails service classes are Spring beans, the carService will be injected.

Services and Scope

Grails services are singletons by default. To change that you just need to declare a static scope variable.


static scope = 'request'

Here are the possible values for scope:

Scope Description
request A new instance is created for each request
session One instance is created for an entire session
flash An instance is created that will exist for this request and the next request
prototype A new instance is created for each objec that it's injected into
flow instance exists for the life of a flow (webflow only)
conversation Instance exists for the life of a flow and all sub-flows (webflow only)

GSP

Along with the ability to easily create custom GSP tags, Grails provides over 50 built-in tags. So many that it can be easy to miss one that could be just what you need. The following table shows some of the tags you don't want to miss:

Logical Tags

if- Conditionaly render GSP portions

Attribute Description
test* Expression to evaluate
env* Name of a Grails environment

<g:if env='development' test='${car.year > 2008}'>
   <p>This car is new.</p>
</g:if>

Else- The logical else tag


Else- The logical else tag
No Attributes


<g:if test='car.make == 'Honda''>
...
</g:if>
<g:else>
   <p>Sir, this car is not a Honda.</p>
</g:else>

Looping Tags

each- iterate over each element of the specified object

Attribute Description
in The object to iterate over
status Variable to stor the iteration index in
var The name of the item

<g:each var='car' in='${cars}'>
   <p>Make: ${car.make}</p>
   <p>Year: ${car.year}</p>
</g:each>

findAll- condtionally iterate over objects in a collection

Attribute Description
in The collection to iterate over
expr A Gpath expression

<h1>2003 Vehicles</h1>
<ul>
<g:findAll in='${cars}' expr='it.year == '2003''>
   <li>${it.make} ${it.model}</li>
</g:findAll>
</ul>

Form Tags

form- HTML Form with an action attribute based on controller/action/id

Attribute Description
action* Action to use in the Form Action
controller* Controller to use in the Form Action
id* ID to use in the Form Action
url* A map containing the action/controller/id

<g:form name='myForm' action='show' id='1'>
...
</g:form>
<g:form name='myOtherForm' url='[action:'list',controller:'car']'>
...
</g:form>


*if not specified, the current controller/action will be used

datePicker- Creates HTML selects for day/month/year/hour/second

Attribute Description
name Name of the date picker field set
value Current value of the date picker
default Default date. if '=none', the default is blank
Precision Date granularity: year, month, day, hour, minute
noSelection Map detailing the key and value to use for the 'no selection made' choice in the selected box
Years List/range of displayed years, in specific order

<g:datePicker name='myDate' value='${new Date()}'
noSelection='['':'-Choose-']'/>
<g:datePicker name='myDate' value='${new Date()}' precision='day'
years='${1930..1970}'/>

checkBox- Creates an HTML checkbox form field

Attribute Description
name Name of the checkBox
value Expression; if evaluates to true, 'checked = true'

<g:checkBox name='Used' value='${true}' />

radio- Creates an HTML radio button

Attribute Description
value value represented by radio button. Displayed as label
name Name of radio button
checked Boolean true/false for checked status

<g:radio name='myGroup' value='2' />

Select- Creates an HTML select

from a list or range to select from
value the current value of the property
optionKey property of the bean to use as the key
optionValue property of the bean to use as the value
noSelection single-entry Map with a default key to return and a default value to display.
valueMessage-Prefix Will be prepended to the option value with a '.' to create a key to lookup the value in the i18n message bundle.

<g:select name='car.year' from='${1903..2009}' value='${year}'
noSelection='['':'-Choose car year-']'/>



hiddenField-

Creates an HTML input of type 'hidden'

name the name of the input field
value the value of the text field

<g:hiddenField name='year' value='${carInstance.year}'/>

actionSubmit- Creates a submit button with the specified value

name Required; the title of the button. if 'action' is not specified, this will be the default action.
action the action to execute

<g:actionSubmit value='Button Label' action='Update'/>

Link Tags

link- Creates an HTML anchor tag based on parameters

action name of action to link to - if not specified, the default action will be used
controller name of the controller to link to '" if not specified, the current controller will be used
id the id to use in the link
params a Map containing request parameters
url a Map containing the controller, action, etc. to use in the link

<g:link controller='car' action='list'>List of Cars</g:link>

createLink- Creates a URL which can be used in anchor tags, etc.

action the action to be used in the link- if not specified the default action will be used
controller the controller to be used in the link '" if not specified the current controller will be used
id the id to be used in the link
url a Map containing the controller, action, id, etc.

<g:createLink controller='car' action='show' id='1'/>
will create
AutoMart/car/show/1
As a method call in a GSP:
<a href='${g.createLink(controller:car, action:show, id:1)}'>View
Car #1</a>
will create
<a href='/AutoMart/car/show/1'>View Car #1</a>

createLinkTo- Creates a link to a static resource

dir the directory in the application to link to
file the name of the file in the application to link to

<g:createLinkTo dir='css' file='main.css' />
creates a link to
/shop/css/main.css

Ajax Tags


In order for the Ajax tags to work properly, you need
to include the javascript tag, specifying the library to
use, in the <head> section of your page. For example:
<head>
   <javascript library='yui' />
</head>

javascript- For inclusion of JavaScript libraries and scripts; also a shorthand for inline JavaScript

library the library to include
src the name of the JavaScript file to import - will look for a file in /web-app/js/
base the full URL to prepend to the library name

<g:javascript src='/sites/all/modules/dzone/assets/refcardz/060/thisisascript.js' />
will import
/web-app/js/thisisascript.js
<g:javascript library='scriptaculous' />
will import necessary JavaScript for the Scriptaculous library
Inline JavaScript:
<g:javascript>alert('hello')</g:javascript>


About The Author

Photo of author Dave Klein

Dave Klein

Dave Klein is a developer with Contegix, a company specializing in delivering managed internet infrastructure based upon Linux, Mac OS X, JEE, and Grails. Dave has worked as a developer, architect, project manager, mentor, and trainer for the past 15 years, and has presented at user groups and national conferences. Dave's Groovy and Grails-related thoughts can be found at http://dave-klein.blogspot.com.

Recommended Book

WPF book

In Grails: A Quick-Start Guide,you'll see how to use Grails by iteratively building an unique,working application. By the time we're done, you'll have built and deployed a real,functioning website.

Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.

Core Mule

By Jos Dirksen

10,003 Downloads · Refcard 36 of 151 (see them all)

Download
FREE PDF


The Essential Mule Cheat Sheet

Mule is one of the most mature open source enterprise service busses (ESBs) out there. It provides an easy to use, lightweight ESB that can easily be integrated with a large amount of technologies. Mule also provides a rich set of routers, transformers, and filters which you can use in your own integration flows. This DZone Refcard will provide an overview of the architecture of Mule and show the different routers, transformers, and filters that are available, and will show how to use them by using example configurations.
HTML Preview
Core Mule

Core Mule

By Jos Dirksen

ABOUT MULE CONFIGURATION

Mule is one of the most mature open source enterprise service busses (ESBs) out there. It provides an easy to use, lightweight ESB that can easily be integrated with a large amount of technologies. Mule also provides a rich set of routers, transformers, and filters which you can use in your own integration flows. This reference card will provide an overview of the architecture of Mule and show the different routers, transformers, and filters that are available, and will show how to use them by using example configurations.

MULE ARCHITECTURE IN A NUTSHELL

To make it easier to understand Mule, let's first have a quick look at Mule's architecture.

Mule'sArchitecture

As you can see, the basic concepts of Mule are pretty straight forward. In Mule's architecture we have the following main parts:

Component Contains the business logic. For instance this could be a spring bean, a REST service, a POJO, etc.
Transport Handles connectivity with a specific technology or application (e.g. JMS, SAP, FTP, etc.).
Transformers Transforms the data to the format the next component expects and can work with.
Inbound Routers Determines what to do with the received message before it's sent to the service.
Outbound Routers Determines where a message needs to be sent to after it's been processed by the service.

Basically what happens is:

  1. A transport receives a message. For instance a message has been put on a JMS queue the transport is listening on.
  2. Before the message is sent to the inbound router, it's first transformed (if needed) to the required format.
  3. Then the message is processed by the inbound router. For instance we could have a "selective consumer" which only accepts messages that are sent by applications we trust.
  4. After the inbound router, the message is sent to the component, which applies its business logic to it.
  5. After the service is done processing, the message is sent to the outbound router. This router determines where to next send the message. We could for instance split this message into multiple parts and send those to different targets.
  6. And finally we can transform the message once again, and let the transport handle all the connectivity details.

CONFIGURING MULE

Mule's configuration is based on Spring and uses XML schemas to provide code completion. This makes it very easy to write your integration flows. Let's start with a very basic Mule configuration:


<mule xmlns="http://www.mulesource.org/
schema/mule/core/2.1" xmlns:xsi="http://www.
w3.org/2001/XMLSchema-instance" xmlns:vm="http://
www.mulesource.org/schema/mule/vm/2.1" xmlns:file="http://www.mulesource.org/schema/mule/file/2.1"
xsi:schemaLocation="


	http://www.mulesource.org/schema/mule/core/2.1
	http://www.mulesource.org/schema/mule/core/2.1/mule.xsd
	http://www.mulesource.org/schema/mule/vm/2.1
	http://www.mulesource.org/schema/mule/vm/2.1/mule-vm.xsd
	http://www.mulesource.org/schema/mule/file/2.1
	http://www.mulesource.org/schema/mule/file/2.1/mule-file.xsd">


<model name="refcheat-model">
 <service name="basic-service">
   <
     inbound>
      <file:inbound-endpoint name="example-in"
        path="work/example/in" />
  </inbound>
  <component>
      <singleton-object class="dzone.Reverser" />
  </component>
  <outbound>
     <pass-through-router>
       <file:outbound-endpoint name="example-out"

Configuring Mule, continued


           path="work/example/out" />
        </pass-through-router>
      </outbound>
    </service>
  </model>
</mule>

You can see that at the top of the file we've declared a number of Mule specific namespaces. Mule provides XML schemas for all its features. We'll focus on Mule core sheets, but also show you a couple of features from the vm and file schemas. After the namespaces declaration we define a <model> element. A model in Mule is a container element for a number of services. In the model element you can see that we've defined a single service where the parts we've discussed in the introduction appear again. In this case we've configured a file transport which will read messages from the file system, pass it on to a component which will reverse the content of the input file, and finally use an outbound router, with a single transport to write the now reversed string back to the file system. We didn't specify an inbound-router. If we don't specify one, all the messages are simply processed by the specified component.

In this example the inbound endpoint uses the File transport. Mule provides several standard transports you can use, as described in the next section. For details on a specific transport, see http://www.mulesource.org/display/MULE2USER/Available+Transports

We won't go into the details of all the endpoints. We'll just provide an overview of the transports available.

Mule Transports

Here is an overview of the transports Mule provides.

Namespace Description
file Provides endpoints which allow you to read and write to the file system
axis Allows you to consume and provide webservices using axis
jbpm Adds functionality to interact with jBPM
cxf Allows you to consume and provide webservices using CXF
ejb Using the endpoints from this transport you can connect to EJBs
email The email namespace provides functionality to connect to POP3, SMTP and IMAP servers
ftp Provides endpoints to read and write to ftp servers
http Allows you to receive and send information using HTTP
jdbc With the JDBC endpoints you can interact with databases using SQL
jms Provides endpoints to connect to JMS queues and topics
multicast Provides an UDP multicast endpoint
quartz Allows you to control the quartz job manager from Mule
rmi Provides inbound and outbound endpoints for RMI
stdio Allows you to send messages to mule using stdio
tcp Provides endpoints for tcp connectivity
udp Provides endpoints for udp connectivity
vm The vm endpoint can be used for internal communication
xmpp The XMPP endpoint can be used to connect to XMPP compliant instant messaging servers

Mule Expressions

With expressions Mule allows you to access certain properties from the message or from the payload, and based on these

Mule Expressions, continued

properties, execute certain actions. There are, for instance, routers, filters, and transformers that work based on these expressions. Below are a couple of examples of how these expressions can be configured. The first one shows how to use an expression on a filter.


<expression-filter evaluator="header" expression="priority=1"/>

This one shows how you can use expressions for routing.


<expression-recipient-list-router
evaluator="xpath"
           expression="/header/routing/recipient" />


Available Evaluators

To use an expression, you specify an evaluator (the expression type) and the expression itself.

Evaluator Description
attachment Allows you to access an attachment of a message
attachments Returns a java.util.Map of attachments
attachments-list Returns a java.util.List of attachments objects
bean With this property you can access the message using a javabean style
endpoint Allows you to access endpoint information
exception-type Allows you to match the type of an exception
function Performs a function: now, date, datestamp, systime, uuid, hostname, ip, or count. Not supported by expression filters.
groovy Evaluates the expression using the Groovy language
header Evaluates the specified part of the message header
headers Returns all the headers as a java.util.Map
headers-list Returns all the headers as a java.util.List of header values
jxpath Allows you to specify an XPath expression that works on XML and javabeans
map-payload Returns a single value from a Map
message Gives you access to various message properties: id, correlationId, correlationSequence, correlationGroupSize, replyTo, payload, encoding, exception
mule Allows access to certain Mule properties: serviceName, modelName, inboundEndpoint, serverId, clusterId, domainId, workingDir and homeDir
ognl Allows you to use OGNL to access the message
payload If expression is provided, it will be a class to be class loaded. The class will be the desired return type of the payload.
payload-type Allows you to check the payload-type of the message
regex Allows you to use a regular expression to access data
wildcard You can use a wildcard expression to determine whether a filter matches
xpath Allows you to use an XPath expression

Hot Tip

Most elements allow you to configure the expression using the evaluator and expression attributes. For properties, you can specify multiple expressions using #[<evaluator>:<expression>] in Mule 2.1 or ${<evaluator>:<expression>} in Mule 2.0. For example: <message-properties-transformer> <add-property name="GUID" value="#[xpath:/msg/header/ID]-#[xpath:/msg/body/@ref]"/> </message-properties-transformer> For more information on expressions you can look at http://www.mulesource.org/display/MULE2USER/Expressions+Configuration+Reference.

TRANSFORMERS

Mule provides a number of transformers which you can use in your own integration flows. Before we look at the transformers provided by Mule, let's first look at how you configure transformers. In the following listings you can see the different ways we can configure and reference a transformer:


<custom-transformer class="dzone.CustomTransformer"
		name="myCustomTransformer"/>
<xml:xslt-transformer name="xsltTransformer"
		xsl-file="resources/xslt/transform.xslt"/>
<file:file-to-string-transformer name="fileToString"/>
...
<file:inbound-endpoint name="example-in"
		path="work/example/in"
transformer-refs="fileToString
		myCustomTransformer xsltTransformer"/>

You can add transformers as a transformers-refs attribute to any endpoint. If you want to do this you first have to make sure you've already defined them. The transformers will be executed in the same sequence as they are listed in the attribute. Note that most of the transports have their own default transformer which is executed if you don't specify transformers yourself. If you do specify transformers yourself you have to make sure you also add the default one, which in this case is the fileToString transformer.


<file:inbound-endpoint name="example-in" path="work/
  example/in">
	<transformer ref="fileToString"/>
	<custom-transformer class="dzone.
     CustomTransformer"/>
	<transformer ref="xsltTransformer"/>
</file:inbound-endpoint>

In the previous listing we added the transformers as child elements of the endpoint. This has the same effect as the previous configuration, but now we don't have to define all the transformers before hand, but can define them inline.


<file:inbound-endpoint name="example-in" path="work/
 example/in">
  <transformers>
   <transformer ref="fileToString"/>
   <transformer ref="myCustomTransformer"/>
   <custom-transformer class="dzone.
     CustomTransformer"/>
  </transformers>
  <response-transformers>
   <base64-encoder-transformer/>
   <transformer ref="stringToFile"/>
 </response-transformers>
</file:inbound-endpoint>

<transformers> element. What you also see is that we've added a response-transformers element (which is also available as an attribute <response-transformers-refs>). A response-transformer does the same as a normal transformer, but is applied specifically on the response to a synchronous call.

Synchronous or asynchronous
Starting from Mule 2.1 you need to explicitly define whether a message is processed synchronously or asynchronously on both the inbound and outbound endpoints. You can do this by using the synchronous attribute on an endpoint. If you specify synchronous="true" Mule will return a result from the call. If you specify synchronous="false" no result will be returned. This value defaults to false. So by default Mule operates asynchronously. You can, however, override this by adding <configuration defaultSynchronousEndpoints="true"/> to your configuration file.

Available Transformers

The following table lists all the transformers from the Mule core and the Mule XML namespace. They can be used in the manner explained earlier.

Name Description
<append-string-transformer/> A transformer that appends a string to a string payload
><auto-transformer> A transformer that uses the transform discovery mechanism to convert the message payload
<custom-transformer> Allows you to create a custom transformer
<message-properties-transformer> A transformer that can add or delete message properties
<no-action-transformer> A transformer that does nothing
<base64-encoder-transformer> Transforms a string or byte array to base64
<base64-decoder-transformer> Transforms a base64 message to an array of bytes
<xml-entity-encoder-transformer> A transformer that encodes a string using XML entities
<xml-entity-decoder-transformer> A transformer that decodes a string containing XML entities
<gzip-compress-transformer> A transformer that compresses a byte array using gzip
<gzip-uncompress-transformer> A transformer that uncompresses a byte array using gzip
<byte-array-to-hex-string-transformer> A transformer that converts a byte array to a string of hexadecimal digits
<hex-string-to-byte-array-transformer> A transformer that converts a string of hexadecimal digits to a byte array
<byte-array-to-object-transformer> A transformer that converts a byte array to an object
<object-to-byte-array-transformer> A transformer that serializes all objects
<object-to-string-transformer> A transformer that gives a human-readable description of various types
<byte-array-to-serializable-transformer> A transformer that converts a byte array to an object (deserializing the object)
<serializable-to-byte-array-transformer> A transformer that converts an object to a byte array (serializing the object)
<byte-array-to-string-transformer> A transformer that converts a byte array to a string
<string-to-byte-array-transformer> A transformer that converts a string to a byte array
<encrypt-transformer> A transformer that encrypts a message
<decrypt-transformer> A transformer that decrypts a message
<expression-transformer> A transformer that evaluates one or more expressions on the current event
<xml:xml-to-dom-transformer> Transforms an XML message payload to an org.w3c.dom.Document
<xml:xml-to-object-transformer> Converts XML to Java bean graphs using Xstream
<xml:xslt-transformer> Transformer that uses XSLT to transform the message payload

MULE FILTERS

Mule provides a set of default filters you can use to determine whether a message should be sent to a destination or whether it's read from a destination. Defining a filter works in the same manner as defining a transformer. You can define them globally and reference them from an endpoint.


<regex-filter name="regex" pattern="(^my)(.*)(txt$
  )"/>
<custom-filter name="custom" class="dzone.
  CustomFilter"/>
....
<file:inbound-endpoint name="example-in"
  path="work/example/in-1">
	<filter ref="regex"/>
</file:inbound-endpoint>
<file:inbound-endpoint name="example-in-2"
  path="work/example/in-2">
	<payload-type-filter expectedType="java.
	 lang.String"/>
</file:inbound-endpoint>

In the previous example the message will only be received if it passes the filter. Mule also provides a set of logical filters which you can use to combine filters using NOT, AND and OR semantics.


<not-filter>
  <filter ref=”custom”/>
</not-filter>
<and-filter>
<payload-type-filter expectedType="java.lang.
   String"/>
  <filter ref="regex"/>
</and-filter>
<or-filter>
<payload-type-filter expectedType="java.lang.
    String"/>
  <payload-type-filter expectedType="java.lang.
    StringBuffer"/>
</or-filter>

Available Filters

The following table shows an overview of all the filters from the core and the XML schema. Note that certain transports have their own custom filters you can use.

Name Description
<not-filter> Invert the enclosed filter
<and-filter> Return true only if all the enclosed filters return true
<or-filter> Return true if any of the enclosed filters returns true
<wildcard-filter> Matches String messages against a number of wildcards. For example order.* would match order.line, order.total etc.
<expression-filter> A filter that evaluates whether a specific expression is valid
<regex-filter> A filter that matches the message against a regular expression
<exception-type-filter> A filter that matches the type of an exception
<payload-type-filter> A filter that matches whether the payload is of the correct class
<custom-filter> Allows you to implement your own custom filter
<xml:is-xml-filter> Checks whether the message is an XML message
<xml:jxpath-filter> Checks the message against an XPath expression using JXPath
<xml:jaxen-filter> Checks the message against an XPath expression using Jaxen

MULE ROUTERS

Routers are used in Mule to determine how messages are received by a component and to where they are sent after the component has processed them. Mule implements most of the patterns from the Enterprise Integration Patterns book (Addison-Wesley), and for most uses the same names. We have inbound routers and outbound routers. In this section, we'll first look at the inbound routers, how to configure them, and which ones are available. After that, we'll look at the outbound routers and do the same thing. First let's look at how to configure an inbound router.


<inbound>
   <file:inbound-endpoint path="work/test/in"/>
   <idempotent-secure-hash-receiver-router/>
</inbound>

You define the inbound router on the inbound element in the Mule service configuration. This means that every message that is received on any of the inbound endpoints is processed by the inbound router, before it's processed by the configured component.

Available Inbound Routers

Name Description
<collection-aggregator-router> Configures a Collection Response Router. This will return a MuleMessageCollection message type that will contain all messages received for each correlation group.
<custom-correlation-aggregator-router> Allows you to create a custom correlation implementation.
<custom-inbound-router> With this element you can configure your own custom router.
<forwarding-router> Forwards a message directly to the outbound router without invoking the component.
<idempotent-receiver-router> This router makes sure that only unique messages are received. This is done by checking the unique message ID of the message.
<idempotent-secure-hash-receiver-router> This router generates a hash of the message and uses that to determine whether a message has already been received.
<message-chunking-aggregator-router> Combines two or more messages into a single message by matching messages with a given Correlation ID
<selective-consumer-router> Applies one or more filters to the incoming message. If the filters match, the message is forwarded to the component.
<wire-tap-router> This router allows you to send a copy of a specific message to a certain destination.

Outbound routers are configured on the outbound element:


<outbound>
  <static-recipient-list-router>
    <file:outbound-endpoint path="work/example/
     out" />
    <vm:outbound-endpoint path="example.out" />
  </static-recipient-list-router>
</outbound>

In this example we define an outbound router on the outbound element, and defined a static-recipient-list-router which sends the message that is received from the component to all the specified endpoints.

Available Outbound Routers

Name Description
pass-through-router This router always matches and simply sends or dispatches the message via the endpoint that is configured.

Available Outbound Routers, continued

Name Description
filtering-router Uses filters to determine whether the message matches a particular criteria, and if so, will route the message to the endpoint configured on the router.
template-endpoint-router Allows endpoints to be altered at runtime based on properties set on the current message, or fallback values, set on the endpoint properties.
chaining-router Sends the message through multiple endpoints using the result of the first invocation as the input for the next.
exception-based-router Sends a message over an endpoint by selecting the first endpoint that can connect to the transport.
multicasting-router Sends the same message over multiple endpoints.
endpoint-selector-router Selects the outgoing endpoint based on the evaluation of an expression.
list-message-splitter-router Accepts a list of objects that will be routed to different endpoints. The actual endpoint used for each object in the list is determined by a filter configured on the endpoint itself.
expression-splitter-router Splits the message based on an expression. The expression must return one or more message parts in order to be effective.
message-chunking-router Allows you to split a single message into a number of fixed-length messages that will all be routed to the same endpoint.
static-recipient-list-router Sends the same message to multiple endpoints.
expression-recipient-list-router Sends the same message to multiple endpoints. The destination is determined based on the evaluation of an expression.
custom-outbound-router This router allows you to define your own custom outbound router.

So far we've seen all the various parts that make up a Mule service except the component which contains the business logic. For this, Mule provides a number of options.

Hot Tip

Reuse existing spring configurations
Since Mule is based on Spring it's very easy to reuse your existing spring beans. If you've already got an applicationcontext, and want to reuse those beans from Mule, you can very easily import them. All you have to do is declare the spring namespace, and add the following to your configuration: <spring:import resource="applicationContext.xml"/>. Now you can use all the beans defined in that file directly in Mule.

MULE COMPONENTS

There are a number of different ways to configure Mule components. Here, we'll show you, and also explain how Mule determines which method to call on your component. Mule provides two types of elements to use in your configuration to specify the component you want to use. The first one is the <component> element: <component class="dzone.Reverser"/>

If you use this configuration, Mule will create a new instance of this class for each request which is received. You can also configure Mule to create objects that can be pooled. For this, don't use the <component> element, but use the <pooled-component> element: <pooled-component class="dzone.Reverser"/>

In the previous examples we directly specified the class as an attribute on the elements. We can also use a different way to specify the implementation of the component. You can do this by using any of the following elements inside the <component> or the <pooled-component> element:

Mule Components, continued


<component class>
  <prototype-object class="dzone.Reverser"/>
</component>
<component class>
  <singleton-object class="dzone.Reverser"/>
</component>
<component class>
  <spring-object name="springBean"/>
</component>

The first two of these elements allow you to specify whether you want a new object for each message (the <propotype-object> element), or whether you want to create an object to be a singleton (<singleton-object> element), and reused for all the messages. The final option you can use to specify the implementation of the component is the <spring-object> element. Here you can directly reference a spring-bean from the application context.

MULE ENTRY POINT RESOLVING

One thing we haven't discussed yet is how Mule can determine which method to call on your component. Your component often is just a simple spring bean or POJO, which has multiple methods. The default configuration for Mule is to use a set of entry point resolvers to determine which method to call on your bean. Mule uses the following steps to determine which method to invoke on your POJO.

1. If a property with the name “method” is specified, the value of that property is used to determine the method to invoke on your component. So if you set this (message) property to helloWorld, Mule will look for a method with that name on your bean. This makes use of the MethodHeaderPropertyEntryPointResolver.
2. Mule provides an interface, org.mule.api.lifecycle.Callable, you can implement. If Mule finds this interface on your POJO it will invoke the onCall() method of this interface, when a message is received for this component. This uses the CallableEntryPointResolver.
3. If there is a transformer configured, Mule will use the return type of this transformer to try and determine if there is a method which accepts this type. If this is found Mule will invoke that method. This uses the ReflectionEntryPointResolver.
4. If there is still no unique match Mule will check the type of the payload to see if that matches any of the methods in the bean. This also uses the ReflectionEntryPointResolver.

If the previous steps don't result in a single method, Mule will throw an exception. Beside the ones already mentioned, you can configure your own set of entry point resolvers, should the default configuration be insufficient. The following example shows a custom configuration, which you can configure on the model or on a component.


<entry-point-resolver-set>
  <array-entry-point-resolver
      acceptVoidMethods="true" transformFirst="true"/>
  <callable-entry-point-resolver/>
  <method-entry-point-resolver
      acceptVoidMethods="true"/>
</entry-point-resolver-set>

If you create a custom entry-point resolver, you can easily add to this entry point resolver set.

Entry Point Resolvers

The following table shows an overview of the entry point resolvers which are provided by Mule.

Name Description
<callable-entry-point-resolver> An entry point resolver for components that implement the Callable interface.
<custom-entry-point-resolver> Use to create your own custom implementation.
<property-entry-point-resolver> Uses a property to determine which method on your component to invoke.
<method-entry-point-resolver> This uses the “method” property to determine which method to invoke.
<reflection-entry-point-resolver> Tries to determine the method to invoke based on the payload of the message.
<array-entry-point-resolver> Checks whether there is a method available which takes a single array as its parameter.
<no-arguments-entry-point-resolver> Calls a method which has no arguments.

Resources

Open Source Mule site http://www.mulesource.org
Commercial Mule site http://www.mulesource.com
Open Source ESB in action website http://www.esbinaction.com

About The Author

Photo of author Cay S. Horstmann

Jos Dirksen

Jos Dirksen is a software architect for Atos Origin, where he has been the architect for a number of large integration projects over the last couple of years. Jos has worked with various integration products, commercial and open source, for the last five years. He co-authored the book Open Source ESBs in Action, and regularly presents on topics ranging from enterprise integration patterns to JavaFX, at such conferences as Javapolis and JavaOne.

Publications

Open Source ESBs in Action, co-author with Tijs Rademakers, Manning Publications

Website

www.esbinaction.com

Recommended Book

Open Source ESB's

Open-Source ESBs in Action will help you to learn open-source integration technologies quickly and will provide you with knowledge that you can use to effectively work with Mule and ServiceMix.


Share this Refcard with
your friends & followers...

DZone greatly appreciates your support.


Your download should begin immediately.
If it doesn't, click here.