Cheap and Effective Object Pooling with BlockingQueue

Last night I was brushing my teeth when suddenly the thought occurred to me: BlockingQueue is pretty much exactly what you want in a simple object pool. This seemed like a good thing to investigate.

The basic concept is to populate a BlockingQueue implementation with your object, and then have code that wants to access it pull it from the pool. It basically looks like this:

  // Create the pool
  BlockingQueue<SimpleDateFormat> pool=
    new ArrayBlockingQueue<SimpleDateFormat>(POOL_SIZE);

  // Populate the pool with SimpleDateFormat objects.
  SimpleDateFormat sdf=new SimpleDateFormat(FORMAT);
  for(int i=0; i<POOL_SIZE; i++) {
      pool.add((SimpleDateFormat)sdf.clone());
  }

  [...]

  // Hypothetical MT code accessing the pool
  SimpleDateFormat sdf=pool.take();
  try {
    doSomethingWith(sdf);
  } finally {
    pool.add(sdf);
  }

I selected java.text.SimpleDateFormat as the object to pool because it does not work properly if used by two threads at the same time. I've got code that needs to do something similar, so I currently use one of two strategies:

1. Construct a new parser for each use.
2. Synchronize on a single parser instance when used concurrently.

I named those strategies NEW and LOCKED respectively within my test case, and created a new one called POOL. You can download the source code to the test if you want to see how it works for you. It does two passes on the tests. The first I consider a warm up and the second are the results I present below.

Note that the Apple and FreeBSD machines are not similar in hardware, but it does give two different sort of views. In the better case, ArrayBlockingQueue has little to no synchronization and is quite fast. In the worst case, it's approximately the same as synchronization.

Apple 1.5.0_06 in Client mode

5 threads, 10 in pool, mode: POOL
Processing time:  46ms
15 threads, 10 in pool, mode: POOL
Processing time:  155ms
5 threads, 10 in pool, mode: NEW
Processing time:  86ms
15 threads, 10 in pool, mode: NEW
Processing time:  279ms
5 threads, 10 in pool, mode: LOCKED
Processing time:  87ms
15 threads, 10 in pool, mode: LOCKED
Processing time:  261ms

Apple 1.5.0_06 in Server Mode

5 threads, 10 in pool, mode: POOL
Processing time:  34ms
15 threads, 10 in pool, mode: POOL
Processing time:  98ms
5 threads, 10 in pool, mode: NEW
Processing time:  264ms
15 threads, 10 in pool, mode: NEW
Processing time:  849ms
5 threads, 10 in pool, mode: LOCKED
Processing time:  60ms
15 threads, 10 in pool, mode: LOCKED
Processing time:  189ms

FreeBSD 1.5.0-p1 in Client Mode

5 threads, 10 in pool, mode: POOL
Processing time:  126ms
15 threads, 10 in pool, mode: POOL
Processing time:  404ms
5 threads, 10 in pool, mode: NEW
Processing time:  381ms
15 threads, 10 in pool, mode: NEW
Processing time:  1108ms
5 threads, 10 in pool, mode: LOCKED
Processing time:  126ms
15 threads, 10 in pool, mode: LOCKED
Processing time:  372ms

FreeBSD 1.5.0-p1 in Server Mode

5 threads, 10 in pool, mode: POOL
Processing time:  113ms
15 threads, 10 in pool, mode: POOL
Processing time:  299ms
5 threads, 10 in pool, mode: NEW
Processing time:  775ms
15 threads, 10 in pool, mode: NEW
Processing time:  2325ms
5 threads, 10 in pool, mode: LOCKED
Processing time:  100ms
15 threads, 10 in pool, mode: LOCKED
Processing time:  299ms