Use Hudson
I have just hooked my project onto Hudson, a continuous integration build system. The process is amazingly easy that I don't even need to read any guideline before finish it. Just click the new job, field out the configuration form, save and I am done. The configuration form is good self-documented, and every entry is followed by a small question mark where you can click on it for more help information.
In my project's job, I use subversion for source code management and invoke ant task for build process. It is pretty straight forward that I have already these tools for quite a while.
Write Ant Task for Hudson
After hooked my project onto Hudson, I become more anxious about the used way my project being verified: run junit task for unit testing and emma task for coverage separately because emma does not fail even though the unit test fail and it does not send junit sensor data. Though I already feel it is not a good way to do it for a long time but I just did not brother to look into the ant file to fix it. But now as the Hudson is looking for new commit every minute and the Hudson's server is hooking up over 20 project in the same time, I am become kind of worried about wasting the machine's power for redundant work. Also, for one of the purpose of using Hudson is to let it generate useful project data for my AmbientDevice project, it is better to get the data sooner so that the triggers can react earlier. So I tried to look into the junit.build.xml and emma.build.xml to look for some solution.
What I found is, what makes the build fail is not directly because of the junit tests fail, but because a variable is set to false when a test fails. The variable is set to the failureproperty attribute in junit task, and it is not set in the junit task that inside the emma task. So I just add it back to emma one and make the build fail when tests fail. It make more sense because coverage of fail tests will not be accurate. Also I add the junit sensor to the task dependence to send data of unit tests. But then I noticed that even though the build fails when test fails, the coverage data is still computed and emma sensordata is sent. The horrible thing is this incorrect coverage data will corrupt the project data. For example a false alarm of coverage drop by 20% may be raised because of the junit tests fail. That will be quite misleading and confusing. The solution for this is to add unless="junit.failed" in emma.report and emma.sensor to stop it from running. But the unless attribute is actually occupied by a variable called emma.disable already. However, it does not have any usability about this variable so far. So I just removed it. I am also wondering if it is possible and how to put an AND operation in the attribute verification such as if and unless.
But anyway, the project is now verified in the way that I want: just run the unit test once and get both unit test results and coverage result if test pass. And emma will fail when test fails so that it will not generate some misleading coverage data. I just remember that I have met this kind of matters before that I run the emma after some small changes and suddenly found the coverage is much lower than I thought. I was thinking that I might have forget to put some test on some method then I looked into the test cases. But I found everything are there. It took me quite a while before noticing that it is because a test fail just when it start so all the rest of the test is not executed. That is quite a terrible experience and I am glad that I have fixed it by the way.
No comments:
Post a Comment