Continuous Integration: Patterns and Anti-Patterns

By Paul M. Duvall

ABOUT CONTINUOUS INTEGRATION

Continuous Integration (CI) is the process of building software with every change committed to a project's version control repository.

CI can be explained via patterns (i.e., a solution to a problem in a particular context) and anti-patterns (i.e., ineffective approaches sometimes used to "fix" the particular problem) associated with the process. Anti-patterns are solutions that appear to be beneficial, but, in the end, they tend to produce adverse effects. They are not necessarily bad practices, but can produce unintended results when compared to implementing the pattern.

Continuous Integration

While the conventional use of the term Continuous Integration generally refers to the "build and test" cycle, this Refcard expands on the notion of CI to include concepts such as Deployment and Provisioning. The end result is learning whether you are capable of delivering working software with every source change.

BUILD SOFTWARE AT EVERY CHANGE

A CI scenario starts with the developer committing source code to the repository. There are four features required for CI.

• A connection to a version control repository
• An automated build script
• Some sort of feedback mechanism (such as e-mail)
• A process for integrating the source code changes (manual or CI server)

PATTERNS AND ANTI-PATTERNS

Version Control

The patterns in this section were originally described in the book Software Configuration Management Patterns (Addison-Wesley, 2003, Berczuk and Appleton), except for 'Label Build':

Task-Level Commit

Label Build

<path id="svn.classpath">
  <fileset dir="${lib.dir}">
   <include name="**/*.jar" />
  </fileset>
</path>
<taskdef name="svn" classpahref="/sites/all/modules/dzone/assets/refcardz/084/svn.classpath" classname="org.tigris.subversion.svnant.SvnTask"/>
<target name="create-tag-from-trunk">
	
<svn username="jhancock" password="S!gnhere">
  <copy srcUrl="https://brewery-ci.googlecode.com/svn/trunk"
   <destUrl="https://brewery-ci.googlecode.com/svn/tags/brewery-1.0.0"
    <message="Tag created by jhancock on ${TODAY}" />
 </svn>
</target>

Build Management

Automated Build

<?xml version="1.0" encoding="iso-8859-1"?>
<project name="brewery" default="all" basedir=".">
  <target name="clean" />
  <target name="svn-update" />
  <target name="all" depends="clean,svn-update"/>
  <target name="compile-src" />
  <target name="compile-tests" />
  <target name="integrate-database" />
  <target name="run-tests" />
  <target name="run-inspections" />
  <target name="package" />
  <target name="deploy" />
</project>

Minimal Dependencies

Binary Integrity

Dependency Management

Tools such as Ivy and Maven can be used for managing dependencies.

Consistent Directories

Template Verifier

Staged Builds

Build Practices

Private Build

Integration Build

Continuous Feedback

Expeditious Fixes

Developer Documentation

Automating your documentation's generation will help you keep it up to date and thereby make it more useful for your software's users.

SchemaSpy

<property name="reports.dir" value="${basedir}"/>
<java jar="schemaSpy_3.1.1.jar" output="${reports.dir}/output.log"
error="${reports.dir}/error.log" fork="true">
  <arg line="-t mysql"/>
  <arg line="-host localhost"/>
  <arg line="-port 3306"/>
  <arg line="-db brewery"/>
  <arg line="-u root"/>
  <arg line="-p sa"/>
  <arg line="-cp mysql-connector-java-5.0.5-bin.jar"/> <arg line="-o
${reports.dir}"/>

Note: 'Private Build' and 'Integration Build' are also from Berczuk and Appleton's book Software Configuration Management Patterns (Addison-Wesley, 2003, Berczuk Staged Builds and Appleton)

Build Configuration

Independent Build

Single Command

Single-command deployment execution using Ant:

ant-Dproperties.file=$USERHOME/projects/petstore/properties/devinstall.
properties deploy:remote:install

Dedicated Machine

When creating an integration build machine consider the following:

Externalize Configuration

Example properties that are external to application-specific files:

authentication.type=db
application.url=http://${tomcat.server.hostname}:${tomcat.server.port}/brewery-webapp
database.type=mysql
database.server=localhost
database.port=3306
database.name=mydb
database.user=myuser!
database.password=mypa$$!
database.url=jdbc:mysql://${database.server}:${database.port}/${database.name}
tomcat.server.hostname=localhost
tomcat.server.name=default
tomcat.web.password=pa$$123!
tomcat.cobraorb.port=12748

Tokenize Configuration

Protected Configuration

Database

Script all DDL and DML so that database changes can be run from the command line. Use a version-control repository to manage all database-related changes. (i.e. refer to the pattern)

<target name="db:create" depends="filterSqlFiles" description="Create the database definition">
   <sql driver="com.mysql.jdbc.Driver"
   url="jdbc:mysql://localhost:3306/"
   userid="root"
   password="root"
   classpahref="/sites/all/modules/dzone/assets/refcardz/084/db.lib.path"
   src="/sites/all/modules/dzone/assets/refcardz/084/${filtered.sql.dir}/database-definition.sql" delimiter="//"/>
</target>

Database Sandbox

Give each developer, tester or test user a separate database instance. Install a lightweight database server in each user's test environment (e.g., MySQL, Personal Oracle), which can be installed on the user's private workstation, on a shared test server, or on a dedicated “virtual server” running on a shared server.

Database Upgrade

Running a custom SQL file from a LiquiBase change set:

build.xml
<updateDatabase changeLogFile="db.change.xml"
driver="org.apache.derby.jdbc.EmbeddedDriver" url="jdbc:derby:brewery" username="" password="" dropFirst="true" classpahref="/sites/all/modules/dzone/assets/refcardz/084/project.class.path"/>
db.change.xml
<changeSet id="1" author="phenry">
  <sqlFile path="insert-data.sql"/>
</changeSet>

Testing and Code Quality

Automated Tests

A Simple Unit Test
public void setUp() {
beerService = new BeerDaoStub();
}
public void testUnitGetBeer() {
Collection beers = beerService.findAll();
assertTrue(beers != null && beers.size() > 0);
}
Running a Unit Test in Ant
<junit fork="yes" haltonfailure="true" dir="${basedir}" printsummary="yes">
   <classpath refid="test.class.path" />
   <classpath refid="project.class.path"/>
   <formatter type="plain" usefile="true" />
   <formatter type="xml" usefile="true" />
   <batchtest fork="yes" todir="${logs.junit.dir}">
     <fileset dir="${test.unit.dir}">
       <patternset refid="test.sources.pattern"/>
     </fileset>
   </batchtest>
</junit>

Categorize Tests

Continuous Inspection

Examples:

CheckStyle

<taskdef resource="checkstyletask.properties" classpath="${checkstyle.jar}"/>
<checkstyle config="${basedir}/cs-rules.xml" failOnViolation="false">
  <formatter toFile="${checkstyle.data.file}" type="xml" />

    <fileset casesensitive="yes" dir="${src.dir}" includes="**/*.java" />
</checkstyle>
<xslt taskname="checkstyle"
in="${checkstyle.data.file}"
out="${checkstyle.report.file}"
style="${checkstyle.xsl.file}" />

Build Threshold

<module name="CyclomaticComplexity">
  <property name="max" value="10"/>
</module>

Deployment Test

The table below describes examples of the types of test that might be run as part of a Deployment Test smoke suite.

Deployment

Scripted Deployment

<available file="@{tomcat.home}/server/@{tomcat.server.name}/bin"
   property="tomcat.bin.exists"/>
<if>
   <isset property="tomcat.bin.exists"/>
<then>
   <echo message="Starting tomcat instance at @{tomcat.home} with start_tomcat" />
<exec executable="@{tomcat.home}/server/@{tomcat.server.name}/bin/start_tomcat"
   osfamily="unix" />
 </then>
 <else>
   <echo message="Starting tomcat instance at @{tomcat.home} with startup.sh" />
<exec osfamily="unix" executable="chmod" spawn="true">
   <arg value="+x" />
   <arg file="@{tomcat.home}/bin/startup.sh" />
   <arg file="@{tomcat.home}/bin/shutdown.sh" />
  </exec>
  <exec executable="sh" osfamily="unix" dir="@{tomcat.home}/bin" spawn="true">
<env key="NOPAUSE" value="true" />
<arg line="startup.sh" />
</exec>
    <exec osfamily="windows" executable="cmd" dir="@{tomcat.home}/bin" spawn="true" >
     <env key="NOPAUSE" value="true" />
      <arg line="/c startup.sh" />
    </exec>
    <sleep seconds="15" />
    </else>
</if>

Headless Execution

Unified Deployment

Disposable Container

Disposable Container

Remote Deployment

Environment Rollback

Continuous Deployment

Single-Command Provisioning

Decouple Installation

About The Author

Paul M. Duvall

Paul M. Duvall is the CEO of Stelligent, a firm that helps clients create production-ready software every day. A featured speaker at many leading software conferences, he has worked in virtually every role on software projects: developer, project manager, architect, and tester. He is the principal author of Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley, 2007) and a 2008 Jolt Award Winner. Paul contributed to the UML 2 Toolkit (Wiley, 2003), wrote a series for IBM developerWorks called "Automation for the People", and contributed a chapter to No Fluff Just Stuff Anthology: The 2007 Edition (Pragmatic Programmers, 2007). He is passionate about automating software development and release processes and actively blogs on IntegrateButton.com and TestEarly.com.

Some of the concepts and material in this Refcard were adapted from:

• Continuous Integration: Improving Software Quality and Reducing Risk, by Paul M. Duvall (Addison-Wesley, 2007) - http://www.amazon.com/gp/product0321336380/?tag=integratecom-20
• IBM developerWorks series Automation for the people, by Paul Duvall - http://www.ibm.com/developerworks/views/java/libraryview.jsp?search_by=automation+people

Recommended Book

For any software developer who has spent days in "integration hell," cobbling together myriad software components, Continuous Integration: Improving Software Quality and Reducing Risk illustrates how to transform integration from a necessary evil into an everyday part of the development process. The key, as the authors show, is to integrate regularly and often using continuous integration (CI) practices and techniques.

---

BUY NOW

books.dzone.com/books/continuous-integrations