Multiple Sidekiq processes for multiple Rails/Sinatra applications – namespacing

Sidekiq is a great background processing tool that can be used with many applications deployed on the same server. In order to make it work without any issues or collisions, you need to define namespace for each Sidekiq process group within single application. To do so, you need to create an initializer that will contain both: server connection details and a namespace name:

Sidekiq.configure_server do |config|
  config.redis = {
    url: 'redis://localhost:6379',
    namespace: 'my_app_name_production'
  }
end

Sidekiq.configure_client do |config|
  config.redis = {
    url: 'redis://localhost:6379',
    namespace: 'my_app_name_production'
  }
end

Keep in mind, that you need to provide both url and namespace for client and server. I like to hook it up with SettingsLogic gem:

Sidekiq.configure_server do |config|
  config.redis = System::Settings.sidekiq.redis
end

Sidekiq.configure_client do |config|
  config.redis = System::Settings.sidekiq.redis
end

After you deploy all the apps, you can also easily check if everything is working, by simply executing redis console:

$: redis-cli

and running following command:

redis 127.0.0.1:6379> keys *NAMESPACE*

For example:

redis 127.0.0.1:6379> keys *api*
 1) "youtube_api_v2_production:stat:failed:2014-07-04"
 2) "youtube_api_v2_production:MIA-VPS-VM0974:770"
 3) "facebook_api_production:processes"
 4) "facebook_api_production:queues"
 5) "twitter_api_production:MIA-VPS-VM0974:32655"
 6) "youtube_api_v2_production:queues"
 7) "facebook_api_production:stat:processed"
 8) "youtube_api_v2_production:stat:processed:2014-07-04"
 9) "youtube_api_v2_production:stat:processed"
10) "youtube_api_v2_production:processes"
11) "facebook_api_production:MIA-VPS-VM0974:459"
12) "facebook_api_production:stat:processed:2014-07-04"
13) "twitter_api_production:stat:processed:2014-07-04"
14) "twitter_api_production:stat:processed"
15) "twitter_api_production:processes"
16) "twitter_api_production:queues"
17) "youtube_api_v2_production:stat:failed"

Above you can see multiple queues for multiple namespaced apps.

Categories: Default

1 Comment

  1. You already have distinct applications, why don’t you just specify a different database number for each one?

    http://www.rediscookbook.org/multiple_databases.html

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Copyright © 2024 Closer to Code

Theme by Anders NorenUp ↑