Future Computations: Benchmarks

This is a second part of the original post where I’m introducing real benchmarks using Java Microbenchmark Harness.

As I said before, in the first version the benchmarks were very basic using just System.currentTimeInMillis and println because the idea was to give you a basic point of view about the alternatives to the standard Scala Futures by the creation of a simple scenario (and because I didn’t have so much time to create the benchmarks properly). And I have to say that the basic benchmarks are not so far from the real ones…

This simple scenario consists in the sum of three different square roots of a double by using Futures and measure the average time taken by the three different ways to do it: Future for-comprehension, Future.sequence and async-await.

THE NUMBERS

So let’s move to the interesting part. I created a branch called microbenchmark-harness where you can see the benchmarks created using sbt-jmh. Here are the results:

Running 3 iterations, 10 warmup iterations, 3 forks and 1 thread

The async-await is the fastest approach performing every operation in just 2026.783 nanoseconds followed by the future for-comprehension approach taking 7116.502 nanoseconds per operation.

Benchmark Mode Cnt Score Error Units
asyncAwaitResult avgt 9 2026.783 ±1278.719 ns/op
futureResult avgt 9 7116.502 ±1457.952 ns/op
futureSequenceResult avgt 9 11455.322 ±394.988 ns/op

Running 20 iterations, 20 warmup iterations, 10 forks and 1 thread

The async-await is again the fastest approach performing every operation in just 1999.398 nanoseconds.

Benchmark Mode Cnt Score Error Units
asyncAwaitResult avgt 200 1999.398 ±132.615 ns/op
futureResult avgt 200 6762.131 ±210.806 ns/op
futureSequenceResult avgt 200 10805.773 ±226.244 ns/op

To see the full results take a look at the README file of the project in the microbenchmark-harness branch.

CONCLUSION

I had this pending task after a few people were interested in have the results of a reliable benchmark like JMH. So here I came back with some results. I hope you like it!

Until next post!
Gabriel.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s