When I first started running my Tycho/Maven build of my visual editor in Jenkins the build would always fail if a single test failed. That was mainly because the Maven build would run the surefire test automatically but cancel the build if there were any failures in them. In turn, this would not keep artifacts from being generated and thus not create a new snapshot release. To make matters worse, my tests are not that stable yet – running them on Linux sometimes fails for no reason whatsoever, leading to yet another broken build.
Now, this is obviously not what you (or Jenkins, for that matter) would expect to happen. What I wanted and what I guess you would normally want is for the build to pass but for Jenkins to mark the build as “unstable”.
To acomplish this on the command line, you simply need to tell Maven to ignore test failures
mvn clean install -Dmaven.test.failure.ignore=true
With Jenkins, this is equally straight forward:
Now Jenkins acts nicely and reports unstable builds while still producing all artifacts.
One of the most visited blog entries on this blog probably is my article on transforming CUnit results to JUnit results: CUnit Tests in Hudson. There I write about how to use xsltproc to transform the output of CUnit to be processed by Hudson/Jenkins.
Now, I decided to put the file on BitBucket to maybe consolidate the contributions or changes others are making. I welcome everybody to make contributions to the transform there.
The link to the BitBucket project is: https://bitbucket.org/mcdeck/cunit-to-junit
Since I needed to parse the results of some CUnit tests in Hudson in a recent project, I came up with the following transformation which I apply to the cunit xml result file.
Hudson can then evaluate the results as if they were generated by JUnit. The same approach is basically used in the CppUnit Plugin for Hudson – which is where I got the idea.
Since I’m too lazy at the moment to write my own plugin for Hudson, I figured running xsltproc after running my tests will be good enough:
xsltproc-win32xsltproc.exe --stringparam suitename testall -o testall_results.xml cunit-to-junit.xsl cunit_testall-Results.xml
To make CUnit produce XML results, you have to do something like this:
CU_set_output_filename( "cunit_testall" );
You can download the cunit-to-junit.xsl here.