Automate deployment with Jobs
From ControlTier
This example shows how to use CTL Center Jobs to automate a multi-step deployment process. The process is a simple yet typical example of a deployment cycle including shutdown, installation, configuration and then start up of the Apache Tomcat server and a webapp archive.
The example shows how to define the entire deployment cycle as a Job that in turn calls a set of subordinate jobs, each of which performs a specific step in the cycle. The example also shows how to use Job options to supply specific information like package location and installation directory. Using options in this way, helps keep the Jobs environment or release independent.
This example is setup to work on a single node but can be modified to work on multiple nodes.
Contents |
Dependencies
The example has these dependencies:
- ControlTier — 3.6.1
- Meet the prerequisites and install ControlTier according to the Installing ControlTier instructions.
- Unix
- This example is not compatible with Windows. It is compatible with Linux, Mac OS X and most unix variants.
- "demo" project
- Be sure the "demo" project has been setup on the nodes you will run this example
- Tomcat — 5.5.31
- Download Zip: http://tomcat.apache.org/download-55.cgi or here v5.5.31
- Copy the downloaded Zip to
$CTIER_ROOT/examples/job-appserver-war-deployment/pkgs/apache-tomcat-5.5.31.zip
Note: A WAR file is included in the $CTIER_ROOT/examples/job-appserver-war-deployment/pkgs directory so there is no external dependency for that.
Build the example
Execute these bootstrap steps:
- At the command line, navigate to the
examples/job-appserver-wardirectory under your$CTIER_ROOTdirectory.-
cd $CTIER_ROOT/examples/job-appserver-war
-
- This configures the job definitions for your environment
-
$ANT_HOME/bin/ant -f build.xml
-
- Run ctl-jobs to load the job definitions into CTL Center
-
for job in jobs/*.xml ; do ctl-jobs load -f $job ;done
-
You are now ready to run the examples.
Run the example
You can run jobs from the command line using ctl-run or via the graphical interface in CTL Center.
Run from CTL Center
Login to CTL Center using your credentials. Go to the "Jobs" page and view the listing of jobs. The bootstrap steps will have created a set of jobs under the "examples" folder.
Run Deploy
1. Navigate to the "examples/simpleapp" job group.
2. Select the "Deploy" job and open it to show the tool bar.
3. Press the Run button
4. Select the default values for the catalinahome, tomcaturl and simpleurl options. These choices will appear in their respective menus:
5. Press "Run Job Now" and follow the output:
After the job completes the output will show "Successful".
You should be able to access the deployed Tomcat and "simple" application. Visit the link like so (eg, http://localhost:28080/simple/):
Run from the shell
Via the shell, use ctl-run to start the Deploy job. It requires three arguments:
ctl-run -j "examples/simpleapp/Deploy" -- -catalinahome <dir> -- \ -tomcaturl <url> -simpleurl <url>
What the example shows
- Job workflow
- The "Deploy" does nothing except call other jobs. "Deploy" illustrates a routine process modeled as a sequence of workflow steps carried out by lower level Jobs — stop, install, configure, start.
- Dispatch options that filter nodes using tags
- Jobs are dispatched to the Node tagged with the "admin" and "simpleapp" tags. The node tagged "admin" plays the role of central control point. The Node(s) tagged "simpleapp" run the Jobs that execute the actual deploy scripts.
- Scripts driven by the Job's option data
- The scripts embodied by the Jobs are called with an arguments exposed as Job options.
It is also worth discussing the design of these Jobs. One might ask, why not just define "Deploy" to carry out all the steps? Why break each step into a separate Job which must be called by a another one? Generally, speaking a procedure can be broken down into a set of jobs if the author believes each job can be useful in its own right. The author might also envision reusing these jobs in new workflows.
How it works
The following sections describe the definitions used through this example. The setup procedure you followed early on took care of generating these definitions but they are included here so you can understand how they work.
The Option definitions
A Job can be given options that are configured to present and accept input in a variety of ways. This example shows how to present a menu of choices using data that comes from an external file containing a JSON map.
All five jobs define a common option used to specify what directory the tomcat server should be installed:
- catalinahome
The "Deploy" and "install" jobs define two options that pull data from JSON files:
- simpleurl
- tomcaturl
The boostrap procedure generated these JSON files replacing the
${basedir} token with a value reflecting your install
path. The build.xml generated these files to the directory,
$CTIER_ROOT/examples/job-appserver-war-deployment/jobs/options.
The Job references the JSON data via a URL specified with
valuesUrl.
This means the data could be accessed from a web server but also from
a local file using the "file://" scheme. This example uses two local
files with json data: simpleurl.json and tomcaturl.json.
The simpleurl.json file contains URLs to the WAR file used by the "Deploy" and "install" jobs. The file listing shows there is one WAR file in the map.
File listing: simpleurl.json
{"simple-1.2.3.war":"file://${basedir}/pkgs/simple-1.2.3.war"}
The tomcaturl.json file contains Tomcat ZIP choices. This references the apache-tomcat Zip file you downloaded earlier.
File listing: tomcaturl.json
{"apache-tomcat-5.5.31.zip":"file://${basedir}/pkgs/apache-tomcat-5.5.31.zip"}
Let's imagine that there were multiple versions of the WAR to present
to the user. Just add more entries to the map.
{ "simple-1.2.1.war":"file://${basedir}/pkgs/simple-1.2.1.war", "simple-1.2.2.war":"file://${basedir}/pkgs/simple-1.2.2.war", "simple-1.2.3.war":"file://${basedir}/pkgs/simple-1.2.3.war" }
The option choices are stored in a local file but imagine generating the choices via a CGI script. Such a CGI script could play the role of exposing packaged artifacts from a build server to the Job.
The Job definitions
Jobs can be defined using the CTL Center graphical interface but can also be defined in an XML file we refer to as job.xml. A single document can contain mulltiple job definitions or just one. Your preferences dictate that convention. The Jobs defined for this example are defined in their own file. The filename does not need to correspond to the name of the job itself.
Each job definition is shown below in their original template form.
The build.xml generated the working job defintions you ran using the
ctl-jobs command in the setup step earlier. You can find
them in directory, $CTIER_ROOT/examples/job-appserver-war-deployment/jobs.
Deploy job
The Deploy job drives the deployment cycle, doing nothing accept running the Jobs that carry out the actual steps. It defines the three options discussed earlier — catalinahome, simpleurl, tomcaturl.
The XML document shows how one job can call another job by using the
jobref element. Options can be specified to the
subordinate jobs using the <arg/> element. You can see in
every case the -catalinahome option is passed through to
each job. The install Job also gets the -simpleurl and
-tomcaturl options passed.
Finally, the Deploy job is unique in that it is only dispatched to the Node tagged "admin". This signfies that the Job runs locally on the CTL Center host since it really does nothing except call the other jobs.
File listing: Deploy.xml
<?xml version="1.0" encoding="UTF-8"?> <joblist> <job> <name>Deploy</name> <description>deploy the simple app</description> <additional /> <loglevel>INFO</loglevel> <group>examples/simpleapp</group> <context> <project>${project}</project> <options> <option name='catalinahome' enforcedvalues='false' required='true' values="${basedir}/simpleapp/apache-tomcat" description='the tomcat install directory' /> <option name='simpleurl' enforcedvalues='false' required='true' valuesUrl='file://${basedir}/options/simpleurl.json description='the simple war url' /> <option name='tomcaturl' enforcedvalues='false' required='true' valuesUrl='file://${basedir}/options/tomcaturl.json description='tomcat zip url' /> </options> </context> <sequence threadcount='1' keepgoing='false' strategy='node-first'> <command> <jobref name='stop' group='examples/simpleapp'> <arg line='-catalinahome ${option.catalinahome}' /> </jobref> </command> <command> <jobref name='install' group='examples/simpleapp'> <arg line='-catalinahome ${option.catalinahome} -simpleurl ${option.simpleurl} -tomcaturl ${option.tomcaturl}' /> </jobref> </command> <command> <jobref name='configure' group='examples/simpleapp'> <arg line='-catalinahome ${option.catalinahome}' /> </jobref> </command> <command> <jobref name='start' group='examples/simpleapp'> <arg line='-catalinahome ${option.catalinahome}' /> </jobref> </command> </sequence> <nodefilters excludeprecedence="true"> <include> <tags>admin</tags> </include> </nodefilters> <dispatch> <threadcount>1</threadcount> <keepgoing>false</keepgoing> </dispatch> </job> </joblist>
configure job
The "configure" job's sole purpose is to customize the tomcat installation after the files have been extracted from their archive. The bulk of the job definition is a simple shell script that makes the tomcat start/stop scripts executable and use a non-standard set of ports.
The script is parameterized by reading from its argument vector (eg.,
$1). Jobs can pass arguments to their scripts using the
<scriptargs tag (eg,
<scriptargs>${option.catalinahome}</scriptargs>).
This job will execute on nodes tagged "simpleapp".
File listing: configure.xml
<?xml version="1.0" encoding="UTF-8"?> <joblist> <job> <name>configure</name> <description>configure the simple app</description> <additional/> <loglevel>INFO</loglevel> <group>examples/simpleapp</group> <context> <project>${project}</project> <options> <option name='catalinahome' enforcedvalues='false' required='true' default="${basedir}/simpleapp" description='the tomcat install directory' /> </options> </context> <sequence threadcount="1" keepgoing="false" strategy="node-first"> <command> <script><![CDATA[#!/bin/sh # File: configure.sh USAGE="$0 configure.sh <catalina-home>" die() { echo "ERROR: $@" ; exit 1 ; } [ $# = 1 ] || { die "$USAGE" ; } CATALINA_HOME=$1 [ -d "$CATALINA_HOME" ] || { die "CATALINA_HOME not found: $1" ; } echo "Chmod'ing files in $CATALINA_HOME/bin/*.sh" chmod +x $CATALINA_HOME/bin/*.sh || die "chmod step failed" echo "Customizing $CATALINA_HOME/conf/server.xml ..." server_xml_tmp=/tmp/server.xml.$$ ;# define a temporary file sed 's/8080/28080/g' $CATALINA_HOME/conf/server.xml > ${server_xml_tmp} mv ${server_xml_tmp} $CATALINA_HOME/conf/server.xml || { die "Text replace failed: $CATALINA_HOME/conf/server.xml" } sed 's/8005/28005/g' $CATALINA_HOME/conf/server.xml > ${server_xml_tmp} mv ${server_xml_tmp} $CATALINA_HOME/conf/server.xml || { die "Text replace failed: $CATALINA_HOME/conf/server.xml" } sed 's/8009/28009/g' $CATALINA_HOME/conf/server.xml > ${server_xml_tmp} mv ${server_xml_tmp} $CATALINA_HOME/conf/server.xml || { die "Text replace failed: $CATALINA_HOME/conf/server.xml" } echo "$CATALINA_HOME/conf/server.xml customized" ]]></script> <scriptargs>${option.catalinahome}</scriptargs> </command> </sequence> <nodefilters excludeprecedence="true"> <include> <tags>simpleapp</tags> </include> </nodefilters> <dispatch> <threadcount>1</threadcount> <keepgoing>false</keepgoing> </dispatch> </job> </joblist>
install job
The "install" job uses the curl command to obtain the Zip
and War archives chosen through the Job option menus. It also extracts
the archives into the specified installation directory.
Like the Deploy job, install defines three options — catalinahome, simpleurl, tomcaturl.
File listing: install.xml
<?xml version="1.0" encoding="UTF-8"?> <joblist> <job> <name>install</name> <description>installs the simple app server and war</description> <additional/> <loglevel>INFO</loglevel> <group>examples/simpleapp</group> <context> <project>${project}</project> <options> <option name='catalinahome' enforcedvalues='false' required='true' values="${basedir}/simpleapp/apache-tomcat" description='the tomcat install directory' /> <option name='simpleurl' enforcedvalues='false' required='true' valuesUrl='file://${basedir}/options/simpleurl.json description='the simple war url' /> <option name='tomcaturl' enforcedvalues='false' required='true' valuesUrl='file://${basedir}/options/tomcaturl.json description='tomcat zip url' /> </options> </context> <sequence threadcount="1" keepgoing="false" strategy="node-first"> <command> <script><![CDATA[#!/bin/bash # File: install.sh USAGE="$0 <catalinahome> <tomcat-pkg-url> <simple-pkg-url>" die() { echo "ERROR: $@" ; exit 1 ; } [ $# = 3 ] || { die "$USAGE" ; } CATALINA_HOME=$1; TOMCAT_PKG_URL=$2; SIMPLE_PKG_URL=$3; catalina_parentdir=`dirname $CATALINA_HOME` ;# parent dir mkdir -p $catalina_parentdir || {die "failed creating dir: ${catalina_parentdir}"} tomcat_filename=${TOMCAT_PKG_URL##*/} ;# parse file names tomcat_base=${tomcat_filename%.zip} ;# simple_filename=${SIMPLE_PKG_URL##*/} ;# simple_base=${simple_filename%-*} ;# [ -d ${CATALINA_HOME} ] && { rm -r ${CATALINA_HOME} || die "failed cleaning existing tomcat deployment" } # retrieve the ZIP curl -s -S --user default:default ${TOMCAT_PKG_URL} \ -o $catalina_parentdir/${tomcat_filename} || { die "Download failed: ${TOMCAT_PKG_URL}" } echo "Downloaded ${TOMCAT_PKG_URL}" # extract the ZIP unzip -q $catalina_parentdir/${tomcat_filename} -d $catalina_parentdir || { die "Extract failed: $catalina_parentdir/${tomcat_filename}" } # move the extracted dir to the catalina_home basename mv $catalina_parentdir/${tomcat_base} ${CATALINA_HOME} || die "failed to create ${CATALINA_HOME}" echo "Extracted ${tomcat_filename} to $CATALINA_HOME" # retrieve the WAR curl -s -S --user default:default ${SIMPLE_PKG_URL} \ -o $catalina_parentdir/${simple_filename} || { die "Download failed: ${SIMPLE_PKG_URL##*}" } echo "Downloaded ${SIMPLE_PKG_URL}" # extract the WAR mkdir -p ${CATALINA_HOME}/webapps/${simple_base} || { die "failed making subdirectory directory: ${simple_base}" } unzip -q $catalina_parentdir/${simple_filename} -d ${CATALINA_HOME}/webapps/${simple_base} || { die "Extract failed: $catalina_parentdir/$simple_filename" } echo "extracted ${simple_filename} to ${CATALINA_HOME}/webapps/${simple_base}" echo "install done" ]]></script> <scriptargs>${option.catalinahome} ${option.tomcaturl} ${option.simpleurl}</scriptargs> </command> </sequence> <nodefilters excludeprecedence="true"> <include> <tags>simpleapp</tags> </include> </nodefilters> <dispatch> <threadcount>1</threadcount> <keepgoing>false</keepgoing> </dispatch> </job> </joblist>
start job
The "start" job is a simple wrapper around Tomcat's own "startup.sh" script. It defines one option to specify the location of the tomcat install directory, "catalinahome".
File listing: start.xml
<?xml version="1.0" encoding="UTF-8"?> <joblist> <job> <name>start</name> <description>start the simple app</description> <additional/> <loglevel>INFO</loglevel> <group>examples/simpleapp</group> <context> <project>${project}</project> <options> <option name='catalinahome' enforcedvalues='false' required='true' values="${basedir}/simpleapp/apache-tomcat" description='the tomcat install directory' /> </options> </context> <sequence threadcount="1" keepgoing="false" strategy="node-first"> <command> <script><![CDATA[#!/bin/bash # File: start.sh USAGE="$0 <catalinahome>" die() { echo "ERROR: $@" ; exit 1 ; } [ $# = 1 ] || { die "Usage: start.sh <catalinahome>" ; } export CATALINA_HOME=$1 [ -d "$CATALINA_HOME" ] || { die "CATALINA_HOME not found: $CATALINA_HOME" ; } export CATALINA_BASE=$CATALINA_HOME; echo "Invoking $CATALINA_HOME/bin/startup.sh" $CATALINA_HOME/bin/startup.sh; exit $?]]></script> <scriptargs>${option.catalinahome}</scriptargs> </command> </sequence> <nodefilters excludeprecedence="true"> <include> <tags>simpleapp</tags> </include> </nodefilters> <dispatch> <threadcount>1</threadcount> <keepgoing>false</keepgoing> </dispatch> </job> </joblist>
stop job
The "stop" job, like "start" is a wrapper around one of Tomcat's scripts, "shutdown.sh". It also adds a bit of logic to only run that script if the Tomcat server is listening on its port.
File listing: stop.xml
<?xml version="1.0" encoding="UTF-8"?> <joblist> <job> <name>stop</name> <description>stop the simple app</description> <additional/> <loglevel>INFO</loglevel> <group>examples/simpleapp</group> <context> <project>${project}</project> <options> <option name='catalinahome' enforcedvalues='false' required='true' values="${basedir}/simpleapp/apache-tomcat" description='the tomcat install directory' /> </options> </context> <sequence threadcount="1" keepgoing="false" strategy="node-first"> <command> <script><![CDATA[#!/bin/bash # File: stop.sh <catalina-home> <port> die() { echo "ERROR: $@" ; exit 1 ; } [ $# = 2 ] || { die "Usage: stop.sh <catalina-home> <port>" ; } export CATALINA_HOME=$1 export PORT=28080 [ ! -d "$1" ] && { echo "CATALINA_HOME not found: $1" ; } export CATALINA_BASE=$CATALINA_HOME; netstat -an|grep ${PORT}|grep -q LISTEN && { echo "Tomcat listening (port=$PORT). Running stop script ..." $CATALINA_HOME/bin/shutdown.sh || die "Script failed: stop.sh" exit $? } echo "Tomcat stopped" exit 0 ]]></script> <scriptargs>${option.catalinahome}</scriptargs> </command> </sequence> <nodefilters excludeprecedence="true"> <include> <tags>simpleapp</tags> </include> </nodefilters> <dispatch> <threadcount>1</threadcount> <keepgoing>false</keepgoing> </dispatch> </job> </joblist>
The setup build.xml
Driven mostly for the convenience for a quick start at this example, an Ant build.xml does some of the setup work to make the example work for your installation. The build file is divided into a few targets to ensure the needed archives are in the right location as well as generate working job and option definitions. The build.xml also adds the tags "admin,simpleapp" to the CTL Center Node.s
File listing: build.xml
<project default="all" name="setup"> <target name="all" depends="intro,pkgs,jobs,tags"> <echo> Setup steps completed.</echo> <echo>------------------------------------------------------------</echo> </target> <target name="intro"> <property name="project" value="demo"/> <property environment="env"/> <fail message="CTL_HOME not set" unless="env.CTL_HOME"/> <echo>------------------------------------------------------------</echo> <echo> Running automated setup steps for this example. </echo> </target> <target name="jobs"> <echo>------------------------------------------------------------</echo> <echo> Generating jobs from template definitions </echo> <echo>------------------------------------------------------------</echo> <mkdir dir="${basedir}/jobs"/> <copy todir="${basedir}/jobs" verbose="true"> <fileset dir="${basedir}/templates/jobs" includes="**/*.xml"/> <filterchain> <expandproperties/> </filterchain> </copy> <echo>------------------------------------------------------------</echo> <echo> Generating options from template definitions </echo> <echo>------------------------------------------------------------</echo> <mkdir dir="${basedir}/jobs/options"/> <copy todir="${basedir}/jobs/options" verbose="true"> <fileset dir="${basedir}/templates/jobs/options" includes="**/*.options"/> <filterchain> <expandproperties/> </filterchain> </copy> </target> <target name="tags"> <echo>------------------------------------------------------------</echo> <echo> Adding tags admin,simpleapp to this client Node </echo> <echo>------------------------------------------------------------</echo> <property file="${env.CTL_BASE}/etc/framework.properties"/> <fail unless="framework.node.name" message="Failed reading framework.node.name property from ${env.CTL_BASE}/etc/framework.properties"/> <exec executable="${env.CTL_HOME}/bin/ctl"> <arg line="-p demo -m modelutil -c tag-add -- -type ${framework.node.type} -name ${framework.node.name} -tags admin,simpleapp"/> </exec> </target> <target name="pkgs"> <echo>------------------------------------------------------------</echo> <echo> Verifying necessary zip and war pkgs are present </echo> <echo>------------------------------------------------------------</echo> <fail message="file not found: simple-1.2.3.war"> <condition> <not> <available file="${basedir}/pkgs/simple-1.2.3.war"/> </not> </condition> </fail> <fail message="file not found: apache-tomcat-5.5.31.zip. Download it to ${basedir}/pkgs"> <condition> <not> <available file="${basedir}/pkgs/apache-tomcat-5.5.31.zip"/> </not> </condition> </fail> </target> </project>
Extending this example on your own
The example bootstrap process configured the example to run on the CTL Center Node. If you want to emulate a scenario where the CTL Center host dispatches the Deploy jobs to remote hosts then perform the following:
1. Remove the "simpleapp" tag from the server Node
ctl -p demo -m modelutil -c tag-remove -- -type Node -name serverNode --\ -tags simpleapp
2. Replace "yourNode" with the resource name for the remote Node you wish to dispatch the Deploy commands.
ctl -p demo -m modelutil -c tag-add -- -type Node -name yourNode --\ -tags simpleapp
Run the Deploy Job as earlier. You should notice the workflow steps execute on the remote hosts.
Here are some more ideas on how to extend this example.
- Create a "Restart" job that calls the "stop" and "start" job
- Reconfigure the Job option definitions so they all declare
enforcedvalues="true". Run the Job and see what effect it has on user input. - Write a CGI script that returns option data containing choices for the simpleurl option.
Releated topics
| ||||||||||||||




