Sidekiq | Closer to Code

If you've set up Errbit/Airbrake and you use Sidekiq, by default you would expect, that Errbit tracks things that happen in Sidekiq workers as well. Unfortunately it doesn't.

In order to make Sidekiq retry failed jobs in needs to catch and handle exceptions on its own. And that's the reason why you need a bit of "magic" to make it work with Errbit. You need to add an custom error handler that will notify Errbit app about errors that occured in Sidekiq workers.

To do so, just create an initializer like this:

# Errbit error catching for Sidekiq workers
Sidekiq.configure_server do |config|
  config.error_handlers << Proc.new { |ex,ctx_hash| Airbrake.notify_or_ignore(ex, ctx_hash) }
end

And that's all!

Sidekiq is a great background processing tool that can be used with many applications deployed on the same server. In order to make it work without any issues or collisions, you need to define namespace for each Sidekiq process group within single application. To do so, you need to create an initializer that will contain both: server connection details and a namespace name:

Sidekiq.configure_server do |config|
  config.redis = {
    url: 'redis://localhost:6379',
    namespace: 'my_app_name_production'
  }
end

Sidekiq.configure_client do |config|
  config.redis = {
    url: 'redis://localhost:6379',
    namespace: 'my_app_name_production'
  }
end

Keep in mind, that you need to provide both url and namespace for client and server. I like to hook it up with SettingsLogic gem:

Sidekiq.configure_server do |config|
  config.redis = System::Settings.sidekiq.redis
end

Sidekiq.configure_client do |config|
  config.redis = System::Settings.sidekiq.redis
end

After you deploy all the apps, you can also easily check if everything is working, by simply executing redis console:

$: redis-cli

and running following command:

redis 127.0.0.1:6379> keys *NAMESPACE*

For example:

redis 127.0.0.1:6379> keys *api*
 1) "youtube_api_v2_production:stat:failed:2014-07-04"
 2) "youtube_api_v2_production:MIA-VPS-VM0974:770"
 3) "facebook_api_production:processes"
 4) "facebook_api_production:queues"
 5) "twitter_api_production:MIA-VPS-VM0974:32655"
 6) "youtube_api_v2_production:queues"
 7) "facebook_api_production:stat:processed"
 8) "youtube_api_v2_production:stat:processed:2014-07-04"
 9) "youtube_api_v2_production:stat:processed"
10) "youtube_api_v2_production:processes"
11) "facebook_api_production:MIA-VPS-VM0974:459"
12) "facebook_api_production:stat:processed:2014-07-04"
13) "twitter_api_production:stat:processed:2014-07-04"
14) "twitter_api_production:stat:processed"
15) "twitter_api_production:processes"
16) "twitter_api_production:queues"
17) "youtube_api_v2_production:stat:failed"

Above you can see multiple queues for multiple namespaced apps.

Tag: Sidekiq

Tracking Sidekiq workers exceptions with Errbit/Airbrake

Multiple Sidekiq processes for multiple Rails/Sinatra applications – namespacing