Tuesday, December 7, 2010

Continuous Integration - Drupal way. Part 2 - Build.

Why should you read this article?

  • Developer, Team Leader, Admin:
    • You will find out how to use build script that will save your time, spend on configuration of your project on dev, test, stage and production servers, on local computers of managers and testers. 
    • Build script will allow you to make CI system and get all profits it gaves you.
  • Manager:
    • You have a lot of project in same time, and want to install in on local comp easily.
    • You will get standart way to install projects on some machine in one click. To install new project you have to run $ant in console. Do this on local machine.
    • You can demo machine with Husdon and build script. It will allow you allways have actual version of project to test and to show to client.
  • Tester, QA manager:
    • You have a lot of project in same time, and want to install in on local comp easily.
    • You will get standart way to install projects on some machine in one click. To install new project you have to run $ant in console. Do this on local machine.
    • You can make CI, add there tests and have regression testing easily.

Project page: ci-drupal



  • $ cd {project_dir}
  • $ svn export http://ci-drupal.googlecode.com/svn/trunk/_build/ ./_build/
  • $ cd ./_build/libs
  • $ wget http://citylan.dl.sourceforge.net/project/ant-contrib/ant-contrib/1.0b3/ant-contrib-1.0b3-bin.zip
  • $ unzip ant-contrib-1.0b3-bin.zip
  • $ rm -f ant-contrib-1.0b3-bin.zip
  • download  mysql j connector from  http://dev.mysql.com/downloads/connector/j/ and extract into current dir
  • $ cd ../../
  • $ mkdir _db
  • download db dump to _db directory
  • $ nano _build/_properties/project.prop
  • set values
    • project.code.dir=/
    • project.sql.dir=/_db
    • project.sql.mask=*.sql
    • project.build.dir=/_build
  • $ nano _build/_properties/default.prop
  • set values
    • project.root={project_dir}
    • env.db.name={db_name}
    • env.db.user={db_user}
    • env.db.password={db_user_pass}
    • env.host.name={host name}
  • $ svn add _build
  • $ svn ci -m "add build script"
  • sudo ant
if you are interesting in more details  you can read futher.

Build targets

CI practice consists of several steps:
Trigger, Update and basic Report stages will be provided by CI system like Hudson. We need to focus now on build step. We are going to use Ant as the tool for make build script.
    Wethever we deal with php and web development, build steps should differs from ones we have for compiled language and non web projects. For example we have no need in compile step, but we should have virtual host installattion step. Realy I have doubts that creating virtual host is related to build step, but because we are going to use selenium I decide to make it as part of build step. Also it is very usefull to run in console $ant and to have new host installed.
     Drupal is DB driven system, so we can not skip DB installation. The other drupal specific thing is drupal settings.php and multisite system.
Project is avaliable on google code here ci-drupal.
    Summing up, we have such targets in our build script:
  1. Init (include libraries we need) // init
  2. Init Database (drop old DB, create new, import new one) // init_db
  3. Add virtual host // create_virtual_host
  4. Create drupal settings.php // create_settings

Using of build script

First of all you have to copy dir with build script to your project dir. How to do this you can find here. Also do not forget that you have to add third-party libraries to /_build/libs. You will need ANT-contrib and mysql J connector.

Project dirs structure

Build script is flex to different dir sturctures. But you have fill build config properties to make it work with it. Wethever dir structure is single for the same project, we move that settings to project structure properties file. You can find it there /_build/_properties/project.prop . You have to specify 3 dirs: dir that contains code, dir that contains db dump, dir that contains build script. Path to that dirs should be related to some workspace dir. How to set it up we will find out later.

For example you have such structure:


|____ code
|____ _build
|____ db

then your properties will be such

If you have structure like this

workspace/code (in same dir)
|____ _build
|____ _db

then your properties will be such

Other project specific properties

In /_build/_properties/project.prop you can also config
  1. project.sql.mask - mask to files that contains db dump. Script will look for db dump files with this mask in project.sql.dir . If you doing backup with backup and migrate you mask will be *.mysql. If some other tool it usually *.sql. Feel free to set db.sql if you sure that there will be allways actual db in this file
  2. project.build.templates.vhost - file name of template that will be used for creation virtual host. That template should contains @ant|vhostname@ and @ant|vhostdir@ that will be replaced with property values.
  3. project.build.templates.settings - file name of template that will be used for creation drupal settings.php. That template should contains @ant|db_user@ @ant|db_password@ @ant|db_host@ @ant|db_name@ that will be replaced with property values
  4. project.build.templates.dir - dir contains template files. Dir related to _build dir.

User specific properties

There are a lot of different professionals in the team and we need to move user specific settings into separated property files. You can find them in /_build/_properties/ dir. By default we use default.prop. Feel free to create your own for every team member. Do not forget that you have to pass file name as ant parameter.
This way $ ant -Duser=user_name .
Let's check out what properties we have.
  • Project root directory  project.root=/var/www/site.com
    • This is directory that contains all structure we describe before.Like code, db, build dirs. You can determinate not only absolute path, but also related. In that case path will be related to dir that we will have in env.WORKSPACE variable. This thing was done to allow Hudson use build script easy, because it can create several instances of project. Hudson always pass env.WORKSPACE variable to ant, but if you are running script manually you have to pass this parameter your self.
  • Use default settings flag project.settings.use_default_dir=true
    • Weatherer drupal support multi-site we have determinate will we run this instance as default or not. If you set true then settings.php will be create in sites/default dir else in sites/site.com dir.
  • Next few options is db server configures. I suppose they clear enough  to skip detailed description, you can find in comments in settings.
    • env.db.host=db-host
    • env.db.name= db-name
    • env.db.user= user
    • env.db.password= password
  • Path to host file. env.host.file=/etc/hosts It is useful for resolve ip of just create host. But because we are changing local hosts file we would not resolve host from other machines. Really, in future, we need some way to register just created domain in DNS server to have network access from it.
  • Host name env.host.name=site.com With this value you will have access to drupal instance. Developer responsible to have unique host name.
  • Host ip. env.host.ip= Drupal instance's host will be resolved with this ip address.
  • Next settings are related to apache instance. Because script will create virtual host it have to "know" where sites-available directory situated. Usually it is env.host.vhost_dir.avaliable=/etc/apache2/sites-available . Other parameter is env.host.vhost_dir.enabled=/etc/apache2/sites-enabled used to enable site. env.host.apache.init=/etc/init.d/apache2 and env.host.apache.command.restart=restart params is used to restart apache after creation new virtual host. In next versions we will remove it, because will us a2ensite command for this.
  • Use sudo parameter env.host.apache.use_sudo=true. In linux only root can bind port 80, that is why we need to restart apache from root. The other problem is that we have to run sudo with out password. Use visudo command to change this option. For user "hudson" add 
    #hudson  ALL=(ALL) NOPASSWD:/etc/init.d  
Build script is described. In next article we will find out inspection tools and extend this script to make inspections.

    1 comment:

    1. This comment has been removed by a blog administrator.