Thursday, March 16, 2006

Pragmatic NAnt scripting

Why am I talking about NAnt when the rest of the world is in love with Ruby and Rake? Well, I still think NAnt has its value. Rake is still pretty raw (it doesn't even have an NUnit task yet), and until it has matured there is still a huge group of developers out there not wanting to spend a huge deal of time learning Ruby/Rake. Plus, most companies out there who have already adopted NAnt is stuck with their NAnt code. This article provides some insights on how to provide such companies business value by making NAnt scripts more maintainable.

Based on my experiences with NAnt, I am concluding the followings are true:

  1. NAnt scripts are very hard to maintain, due to reasons such as:

    • Target dependencies are all intermingled into a big unmanageable web.
    • No consistencies across NAnt scripts. There is no coding convention in NAnt scripts.
  2. The scripts have to handle a lot if you think about it. It has to handle...
    • Pre-Build (like build artifact directory creations)
    • The CI (Continuous Integration) cycle like GetLatest, Compile, UnitTests, OtherTests.
    • Post-Build (like package, stage, distribute, deploy, blah...)
  3. Build scripts are nasty and no one wants to inherit, and that's why names like "build monkey" (and even more provocative ones) are created and stick to people who initially start the script and tend to stick to those people until they leave the project. It is a thankless job.
  4. Build scripts are, nevertheless, crucial to any projects. If you have a good set of build scripts, they can help developers writing higher quality code, BAs to verify story functionalities quicker, QA to mitigate environment nightmare, and streamlines deployment process by deployment team.

And here are some tips that I have found to be useful to extend the maintainability and lifespan of your build scripts.

Have multiple build script files
I would break down what I see in many projects one monolithic projectname.build file into many smaller build files named after their build function. For example, the file below is called Sesame.build. It has one and only one purpose: to build a project, and test it. I am putting these two build functions together into one build file because the targets involving testing the build output is usually small, and usually go hand-in-hand together. Conveniently co-locating them into one file I think is better than creating a small and isolated test.build file. But other build disciplines, such as package & deploy, by itself could be a big deal, so if complexity warrants I will have a separate file for them. The only encapsulation we can apply to build scripts is really separating them into files.

<?xml version="1.0" ?>
<project name="cruise" default="all">

    <target name="all" depends="get, build, tag" description="Target executed by CCNet." />

    <target name="get" depends="get.get_latest" description="Gets the latest source code from Subversion." />
    <target name="build" depends="build.build" description="Compile and build the source code." />
    <target name="tag" depends="tag.tag" description="Tag the successful build by the naming convention tags\\CRUISE-B999" />

    <target name="get.get_latest">
        
    </target>

    <target name="build.build">
        <nant buildfile="Sesame.build" target="all" />
    </target>

    <target name="tag.tag">
        
    </target>

</project>


<?xml version="1.0" ?>
<project name="Sesame" default="all">

    <target name="all" depends="build, test" description="Compile, build, and test the Sesame project." />

    <target name="build" depends="build.compile, build.database" description="Compiles the .NET source code and setup local database instance." />
    <target name="test" depends="test.unit_test, test.other_test" description="Runs unit tests and functional tests." />

    <target name="build.compile">
            ...
    </target>

    <target name="build.database">
            ...
    </target>

    <target name="test.unit_test">
            ...
    </target>

    <target name="test.other_test">
            ...
    </target>

</project>


NAnt target categorizations and naming convention
The ultimate sin of unmaintainable build scripts more or less attribute to the over-profileration of target dependencies. When targets start having a complex web of dependencies, build scripts will take an enormous amount of courage and time to repair.

By breaking down all targets in a single NAnt build file into three categories, and through coding consistency and naming conventions, build scripts can last for a very long time. I find the following categorizations of all NAnt targets in your build scripts useful.

<?xml version="1.0" ?>
<project name="Sesame" default="all">

    <!-- Level 1 -->
    <target name="all" depends="build, test" description="Compile, build, and test the Sesame project." />

    <!-- Level 2 -->
    <target name="build" depends="build.compile, build.database" description="Compiles the .NET source code and setup local database instance." />
    <target name="test" depends="test.unit_test, test.other_test" description="Runs unit tests and functional tests." />

    <!-- Level 3 -->
    <target name="build.compile">
        // Compile using the build.solution_configuration property value...
    </target>

    <target name="build.database">
        // Rebuild database using the database_server property value...
    </target>

    <target name="test.unit_test">
            
    </target>

    <target name="test.other_test">
            
    </target>

</project>


Points of interests:

  • Level 1: These are targets that orchestrates the order of executions of various Level 2 targets and not Level 3 targets. They will only contain depends, but never have a target body. They are the common entry points into the build function the script file represents (eg. target "all" in cruise.build will be called by CruiseControl.NET to kick off the CI build process). They must have descriptions. I prefer names to be underscore-delimited.

  • Level 2: These are targets that group Level 3 targets together to make a cohesive unit of work of function. For example, a "clean" target might do altogether a few things to clean a build: build artifacts directories, build results directories, VS.NET bin/obj/VSWebCache, etc. They again will never have a target body. These targets will also have descriptions. I again prefer their names to be underscored.

  • Level 3: These targets will *never* contain depends. They will *only* do one very refined detail piece of work. In addition, their names should be namespaced by a Level 2 target name using a period, and then their function, so that they are easily distinguishable from other targets. This helps newcomers to recognize them and start treating them differently. They never have descriptions (think of these as private methods in a class that does one very specific function for you).
The namespacing naming convention can also be expanded to properties as well.

Use properties to consolidate paths
Have a file (eg. common_paths.build) that extensively use NAnt properties to consolidate your source's folder structure. This will save a lot of code duplication in your build scripts in the longer run if you intend to keep your build scripts for a while.



<?xml version="1.0" ?>
<project name="common_paths.nant" default="all">
    <property name="trunk.dir" value="." />

    <property name="build_output.dir" value="${trunk.dir}\build_output" />
    <property name="build_results.dir" value="${trunk.dir}\build_results" />
    <property name="source.dir" value="${trunk.dir}\source" />
    <property name="tools.dir" value="${trunk.dir}\tools" />

    <property name="dotnet.dir" value="${source.dir}\dotnet" />
    <property name="dts.dir" value="${source.dir}\dts" />
    <property name="sql.dir" value="${source.dir}\sql" />

    <property name="nunit.dir" value="${tools.dir}\nunit" />
    <property name="nant.dir" value="${tools.dir}\nant" />
    <property name="nantcontrib.dir" value="${tools.dir}\nantcontrib" />

</project>


Use the target description attribute
NAnt provides a -projecthelp command line switch to list all of the targets in a given build file. When you give targets a description these targets gets first-class recognition in the listing as "Main Targets":



Combining this tip with the naming convention of the Level 3 targets can be a very powerful technique in improving the readability of your build scripts. As a bonus tip, consider also implementing a target named "help" to display all this -projecthelp target lists.

Use depends or call, but not both
If you start using both, you will be impairing the readability of your build scripts. They more or less do the same thing anyways. I would use depends over call because newcomers to NAnt would be much more likely to learn about depends ahead of call.

Know your property inheritance
NAnt's property inheritance convenience is commonly overlooked and under-valued. Many people chose using <call> task to preserve locally declared property values, rather than experimenting property inheritance of various kinds. I am publishing my findings here.

<?xml version="1.0" ?>
<project name="example" default="example">

    <include buildfile="common_paths.nant" unless="${property::exists('trunk.dir')}" />

    <target name="example" depends="calling_script.properties">
        <nant buildfile="example_two.build" inheritall="true">
            <properties>
                <property name="solution.configuration" value="DEBUG" />
                <property name="build_output.dir" value="c:\modified_build_output" />
            </properties>
        </nant>
    </target>

    <target name="calling_script.properties">
        <property name="calling_script.depends_inherited" value="Inherited from depends clause." />        
    </target>

</project>


<?xml version="1.0" ?>
<project name="example_two">

    <include buildfile="common_paths.nant" unless="${property::exists('trunk.dir')}" />

    <echo message="1) Explicitly inheriting property from nant task tag: solution.configuration='${solution.configuration}'" />
    <echo message="2) Explicitly overriding property in calling script's nant task tag: build_output.dir='${build_output.dir}'" />
    <echo message="3) Implicitly inheriting property from calling depends clause: calling_script.depends_inherited='${calling_script.depends_inherited}'" />

    <echo message="Is (1) readonly? ${property::is-readonly('solution.configuration')}" />
    <echo message="Is (2) readonly? ${property::is-readonly('build_output.dir')}" />
    <echo message="Is (3) readonly? ${property::is-readonly('calling_script.depends_inherited')}" />

</project>




1 & 2) Inherit properties through <nant> task
You can call the <nant> task and include a <properties> set within the tags, while setting the inheritall attribute to true to pass those properties to the build script that is getting invoked. In addition, as shown in (2), you can also override the loaded property "build_output.dir" in the properties section prior to passing it into the callee build script.

If you use this style of properties passing, it is a very good idea to pass them as readonly=true to avoid your customization of script behaviors to be reset again by the callee script.

3) It's important to know that properties declared earlier in the depends clause can be used further downstream in the later depends targets, even into another callee build script. This technique is useful when you do not want to load all properties for all targets up front (with the side-effect that the last target in your execution will have access to all properties declared in every single previously executed target, but I am okay with it because I rarely run into properties nightmare compare to build scripts nightmare). Therefore, the following readable script can be achieved:

<?xml version="1.0" ?>
<project name="Sesame" default="all">

    <target name="all" depends="build, test" description="Compile, build, and test the Sesame project." />
    
    <target name="build" depends="build.properties, build.compile, build.database" description="Compiles the .NET source code and setup local database instance." />

    <target name="build.properties">
        <property name="build.solution_configuration" value="DEBUG" unless="${property::exists('build.solution_configuration'}" />
        <property name="build.database_server" value="localhost" unless="${property::exists('build.database_server'}" />
    </target>

    <target name="build.compile">
        // Compile using the build.solution_configuration property value...
    </target>

    <target name="build.database">
        // Rebuild database using the database_server property value...
    </target>


I am sure there are lots of other tips as well, but that's it for this post. Comments, feedback welcome.

3 comments:

Anonymous said...

Thanks from Spain.

Anonymous said...

Excellent, thanks.

Anonymous said...

Steve! Long time no talk. Thanks for the post.

Would you happen to know of tools or techniques for testing nant scripts? Ours are getting pretty big and unwieldy, and consequently getting more error prone. Any advice would be greatly appreciated. Hope things are going well with you.