Sunday, July 24, 2011

OpenCover Performance Impact

So how does OpenCover's profiling impact your testing. The best way is to get some figures so that you can judge for yourself.

I decided to use OpenCover's own tests and use the timing value produced by Nunit itself; just like I'd expect any user who is trying to determine impact I suppose. I've also added the results from PartCover for comparison. Before I took any numbers I (warmed) the code by running the code several times beforehand.

Nunit32Nunit32 (OpenCover)Nunit32 (PartCover)Nunit64Nunit64 (OpenCover)
2.6432.6912.6394.5443.807
2.6292.692.6114.4263.753
2.6422.6382.6124.464.036
Average2.6382.6732.6214.4773.865

I don't know how to interpret these results as they don't make much sense, OpenCover seemed to add on average 1.3% to the total time (which I'd expect), whereas PartCover appears to make the code go faster by 0.64%. I can't explain why the results for 64 bit seem to show that OpenCover improves performance by 13.6%.

I tried to come up with a number of reasons for the above but the results I keep getting are reasonably consistent, so I decided to post them anyway and perhaps someone else will be able to tell me what is happening.

No comments:

Post a Comment