When Doing More Stuff Takes Less Time

I have a performance test in my memcached client wherein I do 300,000 fixed-size sets and measure how long it takes me to completely process all of them. I added a lot of code last night to deal with more error conditions and expected it to have some impact on performance.

I found that on my macbook pro (2GHz, etc...), with five tests, I could complete the test in about 10,534ms on average. That's 28,479 sets completed per second over ten seconds. Minimum time was 9,922ms and max was 11,770ms (30,235 sps and 25,488 sps respectively).

I consider these numbers good, as it seems to be about twice as long as it takes me to pipe pre-computed protocol data through netcat (obviously doing little more than just network work). It's also right around where I was before all this work.

Later, however, I realized I had left assertions enabled. I've got 73 assertions in this code base, and some of them do a lot of work (such as introduce loops for inspecting selector state on every select loop). I thought it'd be neat to run the tests again to see how it is without all those checks.

It was consistently slower.

Now, it wasn't dramatically slower, but it was consistently so. The slowest rate I got in my -ea test was 139ms slower than the fastest rate I got in my test without assertions. The slowest rate I got in my test without assertions was 2,928ms slower than the fastest rate in my -ea test. The average without assertions was 1,223ms slower.

I had been running my tests under 1.5, so I figured I'd try 1.6. The behavior there was consistent with my expectations. Using -ea is slower than not using -ea. The difference between the slowest -ea and the fastest test without assertions is 1,132 ms. -ea adds an average of 162ms to my test.

I can't say that I understand why -ea would take time away from my test in 1.5, but in neither case does is it affecting my results enough to offset the huge benefit I've got from the assertions.