Feb 9, 2014

Build Stage in CI Jenkins and Psake

The build stage is the first job in the delivery pipeline.
Please, refer the general article for setting up the delivery pipeline with Jenkins and Psake here.
The primary job of build stage is to update the source code repository till the latest revision, to build the artifacts and to archive them so they are available to the downstream projects.
Building the artifacts is pretty much building the Visual Studio solutions in your project. Yet this can be not intuitive job to do.
If projects in the solutions that are to be built have post-build or pre-build actions that refer 3rd party software, building job can become very complex.
If you integrate existing solution in CI pipeline, review all related projects’ post and pre-build actions and check relations with 3rd party software. Make sure referred 3rd party software supports command-line invocations.
If you build your CI delivery pipeline for new project from its beginning, choose wisely your 3rd party software that is going to be integrated with the project. It must support command-line invocations.
The first step for setting up the build stage is to define the repository type and to setup its details.
In my case, as you can see from the screenshot below I use “Subversion”. Once you provide the “Repository URL”, authentication link will be displayed so you can enter repository credentials. After they are saved, they can be changed any time by clicking the question mark button next to the repo url field.
defineSVNcredentials
The “buildstage.ps1” can be elaborated, so it can start doing “real” job.
Let’s assume that source content is:
  • Mvc.net project (regardless the version)
Let’s name it WebProject and consider it is part of the WebProject.sln. It will be located in “C:\CI\Source\Project\WebProject\” folder
  • Installer vdproj that outputs deployment package *.zip
Let’s name it ProjectPackager.vdproj
Deployment web package will be outputted in “C:\CI\ Source\Setup\Output\”
  • SSDT DB project (solution)
Let’s name the solution ProjectDB.sln. It is located in “C:\CI\ Source\ProjectDB\”
  • Unit Tests solution
Let’s name the solution WebProject.Tests.sln
The IDE is Visual Studio 2010.
This scenario covers pretty much everything needed to have a web based software solution, which can be rolled out to production environment via installer.
So, the job that build stage does can be split in 3 major actions:
1. Cleanup the solutions and blank folders that keep previous version of deployment packages
Here is the body of the “Clean” Psake task:

task Clean { 


      Write-Host -ForegroundColor "Green" -BackgroundColor "White" "Start Clean task..."
      If ($revision)
    {


        Write-Host $revision
    }
    else
    {
        Write-Host "Revision is EMPTY"
    }


      Write-Host -ForegroundColor "Green" -BackgroundColor "White" "Start Clean task..."


      Write-Host "Clean WebProject.sln..."
      exec { msbuild  "$code_directory\WebProject.sln" /t:Clean  /v:n /nologo}


      Write-Host "Clean ProjectDB.sln..."


      exec { msbuild  "$code_directory\ProjectDB\ProjectDB.sln" /t:Clean  /v:n /nologo}
    
      Write-Host "Clean WebProject bin and obj folders..."


      Remove-IfExists $code_directory\Project\WebProject\bin


      Remove-IfExists $code_directory\Project\WebProject\obj


      #delete packages folder
      Write-Host "Clean Output folder along with packages in it(if exists)..."


      Remove-IfExists $code_directory\Setup\Output


      Write-Host -ForegroundColor "Green" -BackgroundColor "White" "Complete Clean task..."
}


#helper methods
function Create-IfNotExists([string]$name) {
    if (!(Test-Path -path $name)) 
    {
        New-Item -Path $name -ItemType "directory"
    }
}
function Remove-ThenAddFolder([string]$name) {


    Remove-IfExists $name
    New-Item -Path $name -ItemType "directory"
}


function Remove-IfExists([string]$name) {
    if ((Test-Path -path $name)) {
        dir $name -recurse | where {!@(dir -force $_.fullname)} | rm
        Remove-Item $name -Recurse
    }
}

As you can see from the “Clean” task code, solutions can be cleaned command line using msbuild command.

If msbuild is not recognized, please check your environment path variable and make sure it contains the path to the command.

Below are some errors you can face, while trying to accommodate psake scripts

  • error MSB4019:
The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

If you spend some time googling for solution, you will find out there are 2 options:
a) Copying the following folder from your development machine to your build server fixes this if it's just web applications
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\WebApplication
b) Download the "Microsoft Visual Studio 2010 Shell (Integrated) Redistributable Package"

This in my option is preferable resolution.

Download and install chocolatey (http://chocolatey.org/) and install the VS2010 redist from http://chocolatey.org/packages/VS2010.ShellIntegratedRedist

After chocolately in installed, install the VS redistributable package with command:

cinst VS2010.ShellIntegratedRedist

After installing the redistributable package:

after-installing VS2010 Redist

2. Build the solutions you need

Here is the body of the “Build” Psake task:
task Build { 
    Write-Host -ForegroundColor "Green" -BackgroundColor "White" "Start Build task..."
 
    Write-Host "Build WebProject.sln in " $config "configuration..."
    exec { msbuild "$code_directory\WebProject.sln" /t:Build /p:Configuration=$config /v:m /nologo}

 
    Write-Host "Build ProjectDB.sln in Debug configuration..."
    exec {msbuild "$code_directory\ProjectDB\ProjectDB.sln" /t:Build /p:Configuration=Debug /v:m /nologo}

    Write-Host "Build WebProject.Tests.sln"
    exec {msbuild "$code_directory\WebProject.UnitTests\WebProject.Tests.sln" /t:Build /p:Configuration=Debug /v:m /nologo}

    Write-Host "Build WebProject web application package..."
    exec {devenv.com "$code_directory\WebProject.sln" /Build "Release|Any CPU" /Project "$code_directory\Setup\ProjectPackager.vdproj" /$config }

    Write-Host -ForegroundColor "Green" -BackgroundColor "White" "Complete Build task..."
}

Msbuild is used for building the solutions and devenv.com is used to build the vdproj project types.

Make sure that the path to devenv.com is part of the global environment PATH variable.

Unfortunately there is no way to build vdproj without installing Visual Studio 2010 on the build environment. Regardless how bad practices this is, there is no way to overcome the restrictions.

Alternative is to use Wix (http://wix.codeplex.com/ ) and InstallShield, which are the new installer projects since VS2012.

Make sure you have all 3rd parties that are referenced from the GAC installed on the build machine. That might be various reporting tools, loggers, etc…

Below is a list of common build errors that you may face during working the “BUILD STAGE” job in Jenkins:
  • Windows SDK required
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Microsoft.Common.targets(2342,9): error MSB3086: Task could not find "AL.exe" using the SdkToolsPath "" or the registry key

"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0A". Make sure the SdkToolsPath is set and the tool exists in the correct processor specific location under the SdkToolsPath and that the Microsoft Windows SDK is installed

Resolution:
http://www.microsoft.com/en-us/download/confirmation.aspx?id=8279
http://stackoverflow.com/questions/2731365/running-msbuild-fails-to-read-sdktoolspath

  • Solution file error MSB5009
Solution file error MSB5009: Error parsing the nested project section in solution file.

Resolution:
Regenerate your solution – create empty folder, delete it, rebuild.

Bear in mind that if you have to install SVN client on the build machine for a reason (it might be required for example by the usage of SubWCRev in post-build event), the version of the subversion client depends the version of the SVNkit that Jenkins plugin supports.

“Subversion Workspace Version” in “Manage Jenkins” -> “System Configuration” correspond to SVNkit version in the installed subversion plugin. (1.7 in this case)

subversionplugin-svnclient

3. Define what the artifacts are and archive them

That is definitely the most important part of the build stage in your delivery pipeline. It’s crucial all actions in the delivery pipeline instance to be executed against the very same artifacts if we don’t want to compromise the output of the pipeline.

Once they are built (only once for the instance), they have to be shared with Jenkin’s jobs. There are various artifact repositories over the network like Nexus, Artifactory, etc. They all turned out to be hard to implement with .Net project (at least to me).

That’s why I chose the built in capability for archiving artifacts that Jenkins has. It stores the artifacts on the file system. All paths on the screenshot are relative to the job’s workspace path (“C:\CI”).

ArchiveArtifacts


The archived artifacts are available for preview for each of the executed/completed job’s instances.

BuildStageInstance


At this stage, a build can be triggered through commit change in the source control management. Project’s solutions are cleaned and rebuilt, artifacts are identified and archived. They are accessible for each of the instances of “BUILD STAGE” job history. Email is sent on success or failure of the build job.

Related links:
https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin (Email notifications plugin)
http://msdn.microsoft.com/en-us/data/tools.aspx (SSDT for Visual Studio)
http://wix.codeplex.com/

4 comments:

  1. this is really too useful and have more ideas from yours. keep sharing many techniques. eagerly waiting for your new blog and useful information. keep doing more.
    PHP Training in Chennai

    ReplyDelete

  2. This blog having the details of Processes running. The way of running is explained clearly. The content quality is really great. The full document is entirely amazing. Thank you very much for this blog.
    Seo Company in Chennai

    ReplyDelete
  3. I just see the post i am so happy to the communication science post of information's.So I have really enjoyed and reading your blogs for these posts.Any way I’ll be replay for your great thinks and I hope you post again soon...
    Web Development Company in India

    ReplyDelete
  4. Its a wonderful post and very helpful, thanks for all this information. You are including better information regarding this topic in an effective way.Thank you so much

    Personal Installment Loans
    Title Car loan
    Cash Advance Loan

    ReplyDelete