Commit Stage in CI Jenkins and Psake

Related posts:

• set up the delivery pipeline
• build stage
• acceptance stage

The “COMMIT STAGE” is the second job in my delivery pipeline.
Please, refer the general article for setting up the delivery pipeline with Jenkins and Psake here.
You can take a look on the article dedicated to preceding job in my pipeline “BUILD STAGE” here.
The primary job of “Commit stage” is to quickly validate the artifacts that are archived in the preceding job of the pipeline instance. Its job can be summarized with 2 major activities:

• Getting the artifacts
• Testing the artifacts

1. Get the artifacts generated from “BUILD STAGE” job
In order to extract artifacts, “Copy Artifact Plugin” must be installed.

In order execution of the tests to work, “COMMIT STAGE” is the first place where the artifacts have to be “extracted” and used.
As first step of build actions in the job definitions, you have to set up the “Copy artifacts from another project” action.


“Target directory” will be created if it doesn’t exist. The path can contain tokens and it will be in my case the number of the revision that triggered the pipeline instance. So, within Artifacts, a folder named with the revision name will be created and all artifacts will be copied from Jenkins home folder to it.



2. Testing the artifacts
Next actions that have to be executed are:

• Execute unit tests with NUnit and fail the job if one of the unit tests fail
• Generate NCover report, regardless if all unit tests execute successful
• Check code coverage thresholds and fail the job if it is below given value

This screenshot gives understanding of what the actions for covering the above actions are:

• Ncover.cmd

ECHO %REVISION%

"C:\Program Files (x86)\NCover\NCover.Console.exe" "%ProgramFiles(x86)%\NUnit 2.6.3\bin\nunit-console-x86.exe" "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\WebProjectTests.nunit" /xml="C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\TestResult.xml" /noshadow  //w "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests" //ias "Project1.dll" //at  "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.trend" //onlywithsource //x "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.nccov"

"C:\Program Files\NCover\NCover.Reporting.exe" "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.nccov" //lt C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.trend  //or FullCoverageReport:Html:C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\report_output 

The script executes the unit tests and creates coverage report based on the output.
FullCoverageReport has been generated for Project1.dll.
For details about command syntax refer NCover web site at http://www.ncover.com/
Bear in mind NCover is not a license-free tool.

• Build.cmd

It’s the same bat file that is already discussed in the previous articles.

The “commitstage.ps1” Psake task copies the NCover report from “WebProject.UnitTests\report_output” to C:\CI\NCoverReports, because that folder is used from Ncover plugin.

Also the report is archived with 7z in PowerShell and the archive is moved to the Artifacts\$revision folder. You can omit this build step, or leave the Psake task empty. It won’t affect the job’s behavior.

• Nunit.cmd

set OLDDIR=%CD%

IF not EXIST "C:\CI\NunitResult" (

 cd "c:\CI\"

 md NunitResult

)

cd %OLDDIR%

"%ProgramFiles(x86)%\NUnit 2.6.3\bin\nunit-console.exe" C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\WebProjectTests.nunit /xml:C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\output.xml

if defined ERRORLEVEL if %ERRORLEVEL% == 0 goto success else goto fail_build

: fail_build

copy C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\output.xml C:\CI\NunitResult /y

echo 'there are unit tests that failed.'

exit 1

: success

echo 'all unit tests succeeded'

copy C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\output.xml C:\CI\NunitResult /y

exit 0

The script executes the unit tests and copies the result to C:\CI\NunitResult. This folder path is used by NUnit plugin.

The problem is NUnit plugin for Jenkins doesn’t work with relatives to workspace paths, but only with absolute paths.
NUnit report is available for the job’s instance regardless tests succeed or fail.

• Ncover-threshold.cmd

"C:\Program Files\NCover\NCover.Reporting.exe" "C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.nccov" //lt C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\coverage.trend  //or MethodModule:Xml:C:\CI\Artifacts\%REVISION%\Source\WebProject.UnitTests\%REVISION%.xml  /e //mc BranchCoverage:95:Module

if defined ERRORLEVEL if %ERRORLEVEL% == 0 goto success else goto fail_build

: fail_build

echo 'Tested modules have branch coverage below the given threshold 95%.'

exit %ERRORLEVEL%

: success

echo 'Tested modules are above the given threshold 95%.'
exit 0

Code coverage 95% has been set up as threshold. If the coverage is less than this value, NCover.Reporting fails with exit code '3'. The job instance fails, which automatically invalidates the pipeline

The post-build actions leverage NCover and NUnit command executions to generate reports available for each of the job’s instance executions





NCover and Nunit plugins generate reports that are available for each of the run instances





At last, the “Commit Stage” triggers next job in the pipeline called “Acceptance Stage”, by post-build step “Trigger parametrized build on other projects”. The context $REVISION is passed as parameter.




Related Links:

https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/NUnit+Plugin
http://wiki.hudson-ci.org/display/HUDSON/NCover+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
http://www.ncover.com/