Logging and Notifications with Rogger, Graylog2 and Twilio

Rogger 0.1.2 was released yesterday, so check it out if you need a way to log your Rails applications / Rake tasks on Graylog2!

Anyway, this post is a showcase of how we’re using Rogger in production to log exceptions raised in Rake tasks and notify us by SMS using Twilio.

We have a few Rake tasks that run daily. Some of them are pretty sizable, and we want to know if any of them fails and throws an exception. Using Rogger’s log_exceptions, we can wrap the task like so:

namespace :important_tasks do
  task :first_task => :environment do
    Rogger.log_exceptions do
      some_function_that_might_fail_and_throw
    end
  end
 
  task :second_task => :environment do
    Rogger.log_exceptions do
      Rogger.info "Starting task important_tasks:second_task..."
      bad_function_that_might_fail
    end
  end
end

If the task does throw, Rogger will log it in Graylog2:

Notice the level. Level 4 corresponds to the :error level (level 7 for :debug), which is what log_exceptions sends exception messages with.

#Graylog2 Streams

We then create a stream in Graylog2 with the following rules:

source must match exactly MyImportantApp
level must be smaller than 5

The source name corresponds to the app_name setting as configured in rogger.yml, and defaults to <%= Rails.application.class.parent if defined?(Rails) %>.

#Twilio Notifications

We install the official Graylog2 Twilio SMS plugin on all our Graylog2 nodes. The instructions are in the README but it’s really straightforward - just wget the .jar file into your Graylog2 plugins folder and restart the Graylog2 server process (./graylogctl restart).

Once it’s successfully installed on your Graylog2 nodes, you can add the alert as a callback, and add your Twilio account details in:

We can test if the alert is working by using “Send Test Alert”, and even do a full integration test by writing a sure-fail test (something like x = 1/0 will do nicely):

#Conclusion

Now, everytime any of our aforementioned daily Rake tasks throw an exception, we will be notified by SMS - thankfully we set it up to run daily at a sane time (no, not in the middle of the night).