Connect a dropwizard application to elasticsearch the right way

Here at Dhatim, we use the elastic stack to gather and analyze our production logs. We also started to use Dropwizard, and the dropwizard people developed a wonderful metrics library, so naturally we want to grab those metrics and send them to our elasticsearch+kibana setup.

Googling around, I could find the dropwizard terse docs about its metrics integration. I could find metrics docs about how to add custom metrics. I could find elastic’s docs and library about how to send metrics to elasticsearch. But I couldn’t find how to put all the pieces together, much less in an elegant way.

Where’s my MetricRegistry?

Elasticsearch Metrics Reporter seemed like a good place to start: It’s maintained by the core Elastic team, so I guess they know how to feed dropwizard metrics to elasticsearch the right way. The doc says:

final MetricRegistry registry = new MetricRegistry();
ElasticsearchReporter reporter = ElasticsearchReporter.forRegistry(registry)
    .hosts("localhost:9200", "localhost:9201")
    .build();
reporter.start(60, TimeUnit.SECONDS);

However, nowhere in my dropwizard application code do I see that new MetricRegistry() call. Where is it? It turns out that Dropwizard takes care of that for me. Poking around, it appears that instance is available through either Bootstrap.getMetricRegistry() in your dropwizard application initialize(..) method, or through the Environment.metrics() method, which is available in your run(..) method.

The second option seems more promising, since I get to play with my app configuration in the run(..) method, so I can set up metrics-to-elasticsearch depending on the application configuration.

Still, this ain’t very dropwizardy, is it?

Let’s step back for a minute. Had I wanted to send dropwizard metrics to a ganglia server, I’d add dropwizard-metrics-ganglia to my application dependencies and then I’d write:

metrics:
  reporters:
    - type: ganglia
      ...

in my app configuration file. Say I want to use a graphite server instead, I’d depend on dropwizard-metrics-ganglia and I’d write:

metrics:
  reporters:
    - type: graphite
      ...

I’d choose console or csv-based reporting the same way. But for elasticsearch, I’d have to write custom some code? That doesn’t seem right. Enter Dropwizard Metrics Support for Elastic Search.

Dropwizard Metrics Support for Elastic Search

This library is modeled right after the core dropwizard metrics reporters, e.g. dropwizard-metrics-ganglia or dropwizard-metrics-graphite. It’s released under the Apache license v2.0.

Add this dependency:

<dependency>
    <groupId>com.dhatim.io.dropwizard</groupId>
    <artifactId>dropwizard-metrics-elasticsearch</artifactId>
    <version>1.0.2</version>
</dependency>

to your dropwizard application dependencies, and you gain the ability to send metrics to elasticsearch via the app configuration file:

metrics:
  reporters:
    - type: elasticsearch
      ...

Thanks to Elasticsearch Metrics Reporter, which does the heavy lifting of formatting the metrics correctly for elasticsearch, you now have access to all the dropwizard metrics – Happy kibana dashboarding!

comments powered by Disqus