Feb 9, 2014

Commit Stage in CI Jenkins and Psake

Related posts:
The “COMMIT STAGE” is the second job in my delivery pipeline.
Please, refer the general article for setting up the delivery pipeline with Jenkins and Psake here.
You can take a look on the article dedicated to preceding job in my pipeline “BUILD STAGE” here.
The primary job of “Commit stage” is to quickly validate the artifacts that are archived in the preceding job of the pipeline instance. Its job can be summarized with 2 major activities:
  • Getting the artifacts
  • Testing the artifacts
1. Get the artifacts generated from “BUILD STAGE” job
In order to extract artifacts, “Copy Artifact Plugin” must be installed.
copyArtifactPlugin
In order execution of the tests to work, “COMMIT STAGE” is the first place where the artifacts have to be “extracted” and used.
As first step of build actions in the job definitions, you have to set up the “Copy artifacts from another project” action.

commitstage-buildStep1
“Target directory” will be created if it doesn’t exist. The path can contain tokens and it will be in my case the number of the revision that triggered the pipeline instance. So, within Artifacts, a folder named with the revision name will be created and all artifacts will be copied from Jenkins home folder to it.

Artifact-copied

2. Testing the artifacts
Next actions that have to be executed are:

  • Execute unit tests with NUnit and fail the job if one of the unit tests fail
  • Generate NCover report, regardless if all unit tests execute successful
  • Check code coverage thresholds and fail the job if it is below given value
This screenshot gives understanding of what the actions for covering the above actions are:
commitStage-BuildSteps

  • Ncover.cmd
ECHO %REVISION%

"C:\Program Files (x86)\NCover\NCover.Console.exe" "%ProgramFiles(x86)%\NUnit 2.6.3\bin\nunit-console-x86.exe" "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\WebProjectTests.nunit" /xml="C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\TestResult.xml" /noshadow  //w "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests" //ias "Project1.dll" //at  "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.trend" //onlywithsource //x "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.nccov"

"C:\Program Files\NCover\NCover.Reporting.exe" "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.nccov" //lt C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.trend  //or FullCoverageReport:Html:C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\report_output 

The script executes the unit tests and creates coverage report based on the output.
FullCoverageReport has been generated for Project1.dll.
For details about command syntax refer NCover web site at http://www.ncover.com/
Bear in mind NCover is not a license-free tool.
  • Build.cmd
It’s the same bat file that is already discussed in the previous articles.

The “commitstage.ps1” Psake task copies the NCover report from “WebProject.UnitTests\report_output” to C:\CI\NCoverReports, because that folder is used from Ncover plugin.

Also the report is archived with 7z in PowerShell and the archive is moved to the Artifacts\$revision folder. You can omit this build step, or leave the Psake task empty. It won’t affect the job’s behavior.
  • Nunit.cmd
set OLDDIR=%CD%

IF not EXIST "C:\CI\NunitResult" (
 cd "c:\CI\"
 md NunitResult
)

cd %OLDDIR%

"%ProgramFiles(x86)%\NUnit 2.6.3\bin\nunit-console.exe" C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\WebProjectTests.nunit /xml:C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\output.xml

if defined ERRORLEVEL if %ERRORLEVEL% == 0 goto success else goto fail_build

: fail_build
copy C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\output.xml C:\CI\NunitResult /y
echo 'there are unit tests that failed.'
exit 1

: success
echo 'all unit tests succeeded'
copy C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\output.xml C:\CI\NunitResult /y
exit 0

The script executes the unit tests and copies the result to C:\CI\NunitResult. This folder path is used by NUnit plugin.

The problem is NUnit plugin for Jenkins doesn’t work with relatives to workspace paths, but only with absolute paths.
NUnit report is available for the job’s instance regardless tests succeed or fail.
  • Ncover-threshold.cmd

"C:\Program Files\NCover\NCover.Reporting.exe" "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.nccov" //lt C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.trend  //or MethodModule:Xml:C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\%REVISION%.xml  /e //mc BranchCoverage:95:Module

if defined ERRORLEVEL if %ERRORLEVEL% == 0 goto success else goto fail_build
: fail_build
echo 'Tested modules have branch coverage below the given threshold 95%.'
exit %ERRORLEVEL%
: success
echo 'Tested modules are above the given threshold 95%.'
exit 0
Code coverage 95% has been set up as threshold. If the coverage is less than this value, NCover.Reporting fails with exit code '3'. The job instance fails, which automatically invalidates the pipeline

The post-build actions leverage NCover and NUnit command executions to generate reports available for each of the job’s instance executions


commitStage-PostBuildActions


NCover and Nunit plugins generate reports that are available for each of the run instances


commitStageInstance


At last, the “Commit Stage” triggers next job in the pipeline called “Acceptance Stage”, by post-build step “Trigger parametrized build on other projects”. The context $REVISION is passed as parameter.

CommitStage-TriggersAcceptance


Related Links:

https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/NUnit+Plugin
http://wiki.hudson-ci.org/display/HUDSON/NCover+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
http://www.ncover.com/

3 comments:

  1. nice blog too informative. looking and reading your points its so impressive. doing more blog like this. i really appreciated doing like this.
    SEO Company in India

    ReplyDelete
  2. Thank you for taking the time to provide us with your valuable information. We strive to provide our candidates
    with excellent care and we take your comments to heart.As always, we appreciate your confidence and trust in us
    Jobs in Bangalore
    Jobs in Chennai

    ReplyDelete
  3. Thank you for taking the time to provide us with your valuable information. We strive to provide our candidates with excellent care and we take your comments to heart.As always, we appreciate your confidence and trust in us
    Digital Marketing Company in India

    ReplyDelete