Drupal Continuous Integration with Jenkins - Deploy

Drupal Continuous Integration
With Jenkins – Deploy

Classic Graphics – Charlotte, NC

By: John W Smith
License: Creative Commons CC0

Description / Purpose

1. In the following slides, we will be describing simple Jenkins jobs that can
be used to automate the deployment your git repository based code (not
specific to Drupal, although that is our purpose) to your Demo, Live,
Staging and/or Testing environments.
First, we will configure a Jenkins job that has the simple task of deploying code to
a single server / environment. Although this is not optimal, as it requires a
separate Jenkins job for each site / server / environment, it is simple to setup and
requires only basic scripting.
Second, we will introduce a couple of scripts that will allow deploying multiple
repositories to their corresponding servers based on parameters passed to the
Jenkins job. This goes back to our original recommendation to name your
repositories, servers, Jenkins jobs, etc using a common simple format.
2. We assume you have already followed the previously released document
“Drupal Continuous Integration with Jenkins” describing the Basic install,
setup and configuration of Jenkins for Drupal Continuous Integration.

Disclaimer

This document does NOT go into the complexities of
securing your system from internal or external attackers,
please ensure your system is secure whether or not it is
accessible from the internet.
Also, please consult with your organizations security
and system administrators before making any changes to
existing systems configuration or authentication models,
and before introducing new systems into the development
and / or production server environments.

System Requirements

2GHz+ Multi-Core CPU
4GB+ RAM
250GB or more depending on the number of concurrent builds,
and build history, artifacts, etc that you will keeping around for
review.
Software needed:
 Jenkins 1.xxx (I used 1.433.x)
 Java JDK/JRE 1.x
 MySQL 5.x
 PHP 5.x
 Drush (latest version)
 Drupal 6.x Core SimpleTest patch file (included with
SimpleTest Drupal Module)

References

1. Points of Contact
The Points of Contact (POC) for this documentation are:
Technical Lead: David Norman
(davidn@knowclassic.com)
Project Lead: John Smith
(johns@knowclassic.com)
or
(JSmith@i1Technologies.com)

Welcome!

Jenkins is here to help. Specifically, Jenkins is an "extensible continuous integration (CI)
server". From a 10,000ft view, Jenkins is a tool for offloading the deployment and
automated testing of a software application. You write your code, push it into version
control, and Jenkins will take over the task of grabbing that code, running an installation /
deployment process, testing the application (if you provide it with a test environment),
and reporting back to you those test results.
~ (http://thinkshout.com/blog/2010/09/sean/beginners-guide-using-hudson-continuous-
integration-drupal)

There are a number of different CI tools out there, but we (Classic Graphics) have chosen
Jenkins, as it seems to be the leader in the Drupal world. Jenkins, for us, provides a web-
based user interface, easy integration via LDAP to our Windows AD Domain, a friendly
way to run scripts, integration with CVS / Subversion / Git repositories, and of course it
works seamlessly with multiple projects / builds simultaneously. Also, as an added bonus,
Jenkins has a Debian repository that we can use to install it on our Ubuntu servers
without worrying about clobbering other libraries and avoid any dependency nightmares.

Create a Server Specific Deployment Job

Here, we will be creating a simple server specific deployment job for our Drupal
environment, this job could then be used as the base for other deployment sites / servers. Since
every shops environment, testing and verification requirements will differ, we leave the
advanced deployment scripting to the user of this document.

The Jenkins job setup in this tutorial makes a few assumptions:
1. You have setup pub-key based auth over ssh to your git repository and possibly the
destination web servers, depending on your configuration this type of setup may or
may not work for you.
2. You have Drupal core, all required modules, and all custom modules located within
your git repository. However, the script could be easily modified to retrieve the latest
released version of Drupal and any other required modules using Drush if this is not
the case in your organization.
3. Access to deployment account private and public keys (if required) and / or the
accounts password. Deployment account MUST have SSH access to the destination
web server(s) AND write permission to each sites “DocumentRoot” directory.
Obviously this type of setup has security implications, please consult with your
systems security officer / administrator.

1. First, open up your favorite browser
and point it to the Jenkins server
(http://jenkins.ourdomain.com:8080).
Click the “log in” link in the upper
right-hand corner of the browser
window.
2. Enter your login credentials and click
“log in”.

3. For this job, we will need an
additional plugin installed and
enabled. Click the “Manage Jenkins”
link on the left.
4. Click the “Manage Plugins” link.
5. Scroll down to find the “SSH Plugin”,
click the check box to enable it.
Scroll to the bottom of the page and
click “Install without restart”.
6. Click the “Return to Dashboard” link
in the left menu to return to the
Jenkins Dashboard once the plugin
has been installed.

7. Click the “Manage Jenkins” link in the
menu.
8. Click the “Configure System” link.
9. Scroll down to the SSH section, and add
the appropriate information for your
destination server. We have the SSH
Server advanced view shown in this
screen shot which allows entry of
server login / password information for
the SSH account to be used.
10. Add as many entries as needed. You
will need one for each server you plan
to use as a destination.
11. Click “Save” when done.

12. Click the “New Job” link in the menu.
Enter “Deployment Test” and select
“Build a free-style software project”.
Click “Ok” when done.

NOTE: When creating these jobs, in order to
simplify the creation of the “git” post-receive bash
script to launch a build remotely, you should
consider naming your jobs the same as the name of
your repository/repo-directory name. So, if you
have your repos on your git server in the
/var/git/reponame.git directory,then this job would
be called reponame. As we progress through this
tutorial, it will become apparent why this makes
things easier.

13. Now let’s configure this simple job.
14. Scroll down to the Build Environment and we’ll
enable the “Send files or execute commands over
SSH after the build runs” option.
15. Select the server you will be deploying to from the
Drop down list.
16. Enter the following text in the “Exec Command”
text area without the quotes. “git-update.sh
user.account branch” where:
• git-update.sh – is a script defined later in this
document
• user.account – is the user the script will use to
retrieve the git repository code (should be the
same account as the SSH account in a perfect
world)
• branch – the repository branch you want to
deploy with this script (most likely master in all
instances except the test server environment)
17. Enable the “Exec in pty” option.
18. Click “Save”.

Create a Generic Deployment Job

Here, we will be creating a generic deployment job for our deployment environment. Since every
shops environment, testing and verification requirements will differ, we leave the advanced
deployment scripting to the user of this document.

The Jenkins job setup in this tutorial makes a few assumptions:
1. You have setup pub-key based auth over ssh to your git repository and the destination web
servers, depending on your configuration this type of setup may or may not work for you.
Alternately, you could use an expect script to enter the login credentials for you, however
this is somewhat insecure as the password would be available in the expect script.
2. You have code (Drupal core, all required modules, and all custom modules) located within
your git repository. However, the script could be easily modified to retrieve the latest
released version of Drupal and any other required modules using Drush if this is not the case
in your organization.
3. Access to the deployment account private and public keys (if required) and / or the accounts
password. Deployment account MUST have SSH access to the destination web server(s)
AND write permission to each sites “DocumentRoot” directory. Obviously this type of setup
has security implications, please consult with your systems security officer / administrator.

3. Click the “New Job” link in the menu.
Enter “Deployment Test” and select
“Build a free-style software project”.
Click “Ok” when done.

NOTE: When creating these jobs, in order to
simplify the creation of the “git” post-receive bash
script to launch a build remotely, you should
consider naming your jobs the same as the name of
your repository/repo-directory name. So, if you
have your repos on your git server in the
/var/git/reponame.git directory,then this job would
be called reponame. As we progress through this
tutorial, it will become apparent why this makes
things easier.

4. Now we need to fill in some
basic configuration for our
newly created Deploy-Test job
as shown in the following
screenshots.
a. Enter a description for our job (not
required)
b. If you would like to setup automatic
deletion of old builds, check the
“Discard Old Builds” checkbox (not
needed for this build, as nothing will
be downloaded or created in this
deployment scenario).
c. You will probably want to leave
project based security enabled, add
any additional users and set
permissions as required.

d. We will configure 3 build parameters for
this build.
e. The first parameter is “REPO” and will
be used to select the git repository to
deploy.
f. The second parameter is “BRANCH”
and will determine the branch of the
given repository to deploy. The
“BRANCH” parameter should default to
master in most cases.
g. The third parameter is “SRVENV” and
will contain the server enviroment we
want to deploy the git repository to (in
our shop we use [test, demo, staging,
and live environments] ).

h. Check the “Execute concurrent builds if
necessary” option.
i. No Source Code Management needed
for deployment jobs.
j. Check “Trigger Builds remotely” option,
and enter a token that can be used to
deploy the sites. For security reasons,
only persons designated to deploy code
to the production environment should
be given this token.
k. Check the “Color ANSI Console Output”
option. (not required)

#!/bin/bash -x
######
## This script is an example ONLY and should be customized for the site/modules you are working on...
######

######
# This is pretty straight forward actually, and can be accomplished a couple of different ways. This
# script is designed to be used inside the Jenkins environment, with a job that can deploy a repository
# to a matching server environment.
#
# Requirements:
# SSH Keys setup for the Jenkins user account
# SSH authorized_keys on destination servers must contain the jenkins account pub key in the
# destination servers deployment account
# The generic version of git-update.sh script included with this job script.
l. Now we’ll add a BASH script to our job, # The deployment account on each destination server needs to have SSH keys setup to allow read-
# only access to the git repository server. This will enable the deployment account to retrieve the
shown to the right here. This will get # appropriate repository and branch to deploy without the need for human interaction.
#
you started, and can be expanded # Jenkins job parameters:
# REPO = GIT Repository to deploy
upon depending on your # BRANCH = GIT Repostiry branch to deploy
# SRVENV = Should be one of the following for CG's environment / configuration. Your setup may
requirements. # differ.
# test - test server environment
m. That’s it for Jenkins using this #
#
staging - staging environment for final QA
demo - demonstration server environment
deployment method. Scroll down to #
######
live - production server environment

the bottom of the job creation screen ### Configuration Variables ###
and click “Save”. # Your deployment user account
DEPLOY_ACCOUNT="deploy_user"

n. We’ll now move on to the server side # The base domain for all your servers in your test/production environment this could also be moved
# up to the Jenkins job parameters section if you need to deploy to several different domains
script that will be used with both of DOMAIN="exampledomain.com"

the above Jenkins deployment jobs. # set the deployment server based on supplied information
if [ "x"${SRVENV%live} = "x" ]; then
# deploying to the live / production environment
DEPLOY_SERVER="${REPO,,*}.${DOMAIN,,*}"
else
# deploying to an environment other than live / production
DEPLOY_SERVER="${REPO,,*}-${SRVENV,,*}.${DOMAIN,,*}"
fi

# now we'll simply execute the git-update.sh script (installed on the deployment
# destination server) with the appropriate parameters
ssh ${DEPLOY_ACCOUNT}@${DEPLOY_SERVER} git-update.sh ${DEPLOY_ACCOUNT} ${BRANCH}

Generic git Deployment Script
#!/bin/bash

script=`basename $0`;
usage=$(cat <<EOF_USAGE
USAGE: $script <cgnum> [branch]
EOF_USAGE);

if [[ $1 == '' ]]; then
echo $usage >&2;
exit 1;
fi
1. The following script will need to gitconfig=$(cat <<EOF_USAGE
You need to configure your git identity and try again.n
be placed on each server that will This information is required once for identifying who isn
responsible for each tag this script creates during a deploy.n
be used as a destination for one of n
Example:n
our Jenkins deployment jobs, and $ git config --global user.name “Git User"n
$ git config --global user.email “guser@ourdomain.com"n
should be placed within the path n
Templates to copy/paste:n
n
and accessible / executable by the git config --global user.namen
git config --global user.emailn
deployment account. EOF_USAGE);

if [ ! -e "$HOME/.git/config" ]; then
echo -e $gitconfig >&2;
exit 1;
Fi

HOSTNAME=`hostname`

if [ ${HOSTNAME:(-8)} == "-staging" ]; then
REPO=${HOSTNAME%-staging}
ENVIRON="staging“
elif [ ${HOSTNAME:(-5)} == "-test" ]; then
REPO=${HOSTNAME%-test}
ENVIRON="test"

elif [ ${HOSTNAME:(-5)} == "-demo" ]; then
REPO=${HOSTNAME%-demo}
ENVIRON="demo“
else
REPO=${HOSTNAME}
ENVIRON="live“
fi

if [[ $2 == '' ]]; then
BRANCH=“master”
else
BRANCH=$2
fi

if [ -d "/tmp/clone/" ]; then
echo "Removing clone directory...“
rm -rf "/tmp/clone/“
fi

echo "Cloning $BRANCH...“
mkdir "/tmp/clone/“

git clone -b "$BRANCH" ssh://$1@code.cgraphics.com/var/git/$REPO.git/ /tmp/clone/

cd "/tmp/clone/“

TAG=`date +%Y-%m-%d-%H-%M`
HASH=`git log "$BRANCH" -n 1 --pretty=format:%h`

echo "Tagging local clone...“
git tag -a -m "$BRANCH $HASH to $ENVIRON by $1" "$ENVIRON-$TAG“

echo "Pushing tag to origin...“
git push origin "$ENVIRON-$TAG“

echo "Syncing the $BRANCH clone to public_html...“
rsync -rl --exclude=private --exclude=".git" --exclude=".gitignore" /tmp/clone/ /var/www/

echo "Removing clone directory...“
rm -rf /tmp/clone/

echo "Launch complete!“
echo "Template for pasting to a related issue:“
echo "Deployed $HASH with tag $ENVIRON-$TAG."

2. That’s it, just place this script in the path, setup one of the previous Jenkins jobs modified
appropriately for your environment, and you will be on your way to automated
deployment of your code via your Jenkins CI Server.

Drupal Continuous Integration with
Jenkins - Deploy

Enjoy, and feel free to contact us with any suggestions,
corrections or mistakes we may have made or that
inadvertently slipped between the cracks.

John W Smith (johns@knowclassic.com) or (Jsmith@i1Technologies.com)

Drupal Continuous Integration with Jenkins - Deploy

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Drupal Continuous Integration with Jenkins - Deploy (20)

Recently uploaded (20)

Drupal Continuous Integration with Jenkins - Deploy