appsignal

Monitor MongoDB (query) load with Custom Metrics

Robert Beekman

Robert Beekman on

Monitor MongoDB (query) load with Custom Metrics

Sometimes standard instrumentation isn't enough to track the root cause of a performance issue. Custom metrics help us track the missing pieces.

Thanks to AppSignal's performance graphs we know exactly what part of our codebase causes performance issues. In the screenshot below you can see we had a severe slowdown because of MongoDB:

custom_metrics

What it doesn't tell us is which of the numerous databases running on different ReplicaSets caused this slowdown. Using our Custom Metrics platform we can answer this question at a glance.

Mongo::Monitoring

With the new 2.x Ruby driver, the mongo gem exposes a monitoring API. We use this to track every query sent to the database.

Here's a subscriber class that implements the three required methods (started, succeeded and failed) and sends the data to AppSignal:

1# config/initializers/mongo_command_subscriber.rb
2class MongoComandSubscriber
3  VALID_DATABASES = Mongoid.clients.map { |k,v| v['database']}
4
5  def started(event)
6  end
7
8  def failed(event)
9    finished(event)
10  end
11
12  def succeeded(event)
13    finished(event)
14  end
15
16  def finished(event)
17    database = event.database_name
18    duration = event.duration
19    return unless VALID_DATABASES.include?(database)
20
21    Appsignal.increment_counter("query_count.#{database}", 1)
22    Appsignal.add_distribution_value("query_duration.#{database}", duration)
23  end
24end
25
26# Subscribe to all COMMAND queries with our subscriber class
27Mongo::Monitoring::Global.subscribe(
28  Mongo::Monitoring::COMMAND,
29  MongoComandSubscriber.new
30)

Custom Metrics dashboard

Now that we're sending these metrics to AppSignal, we have to create a dashboard to visualize the metrics. Let's create two graphs, one for the query count and one for the average query duration:

1- title: "MongoDB Query Load"
2  graphs:
3    - title: "Database Query count"
4      kind: count
5      filter: "query_count/*"
6      format: number
7    - title: "Database average query duration"
8      kind: measurement
9      filter: "query_duration/*"
10      format: duration

We use the filter key to provide a Regex that matches any query duration we have sent to AppSignal.

The result

Now we track the individual query count/duration and have a dashboard in place, and can see which of the databases caused the spike in our performance graph:

custom_metrics

This is one of the many examples where using Custom Metrics helps gaining more insights into the overall performance of our (and your!) application.

If you'd like to give Custom Metrics a try, or need help in identifying and tracking valuable metrics, just let us know.

Share this article

RSS
Robert Beekman

Robert Beekman

As a co-founder, Robert wrote our very first commit. He's also our support role-model and knows all about the tiny details in code. Travels and photographs (at the same time).

All articles by Robert Beekman

AppSignal monitors your apps

AppSignal provides insights for Ruby, Rails, Elixir, Phoenix, Node.js, Express and many other frameworks and libraries. We are located in beautiful Amsterdam. We love stroopwafels. If you do too, let us know. We might send you some!

Discover AppSignal
AppSignal monitors your apps