Running with Ruby

Tag: karafka (page 2 of 4)

Apache Zookeeper + Apache Kafka start / restart script

During my  work on Karafka framework I have to start/stop/restart Apache Kafka and Zookeeper quite often. Here’s a short script that will pull the most recent version of Kafka and Zookeeper, will run them and print their IPs.

Note, that it will also stop Kafka and Zookeeper if they were running.


docker stop zookeeper
docker stop kafka
docker rm zookeeper
docker rm kafka

docker pull jplock/zookeeper
docker pull ches/kafka

docker run \
  -d \
  --name zookeeper \

docker run \
  -d \
  --name kafka \
  --link zookeeper:zookeeper \

ZOOKEEPER_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' zookeeper)
KAFKA_IP=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' kafka)

echo "Zookeeper chroot: $ZOOKEEPER_CHROOT"
echo "Zookeeper: $ZOOKEEPER_IP"
echo "Kafka: $KAFKA_IP"

Dry-Configurable lazy evaluated default settings

Dry-configurable is  a simple mixin to make Ruby classes configurable. It is quite nice, uses Struct and allows you to have default values. Unfortunately, it does not support (yet hopefully) default values that are evaluated upon each setting key request.

When would that be useful? Let’s take a simple example: Kafka + Zookeeper. Zookeeper stores informations about Kafka cluster. They may vary in time, so unless you take them in real time, you might have problem if the cluster were reconfigured. The best option is to set a proc that will be evaluated on each setting request. This is how to obtain such a behavior (note that this will evaluate only default values – if you assign proc in configure block, it will be returned):

# Patch that will allow to use proc based lazy evaluated settings with Dry Configurable
class Dry::Configurable::Config
  # @param [Array] All arguments that a Struct accepts
  def initialize(*args)


  # Method that sets up all the proc based lazy evaluated dynamic config values
  def setup_dynamics
    self.to_h.each do |key, value|
      next unless value.is_a?(Proc)


  # Method that rebuilds a given accessor, so when it consists a proc value, it will
  # evaluate it upon return
  # @param method_name [Symbol] name of an accessor that we want to rebuild
  def rebuild(method_name)
    metaclass = class << self; self; end

    metaclass.send(:define_method, method_name) do
      super().is_a?(Proc) ? super().call : super()

Example usage:

class Application
  extend Dry::Configurable

  setting :standard_setting
  setting :lazy_assigned
  setting :lazy_default, -> { "lazy_default-#{rand}" }

  # Note that this works for nested values as well
  setting :namespaced do
    setting :lazy, -> { "namespaced.lazy-#{rand}" }

Application.configure do |config|
  config.standard_setting = 1
  config.lazy_assigned = -> { "lazy_assigned-#{rand}" }

Application.config.standard_setting #=> 1
Application.config.standard_setting #=> 1
Application.config.lazy_assigned #=> <Proc>
Application.config.lazy_default #=> "lazy_default-0.9601194173446863"
Application.config.lazy_default #=> "lazy_default-0.9450471949405372"
Application.config.namespaced.lazy #=> "namespaced.lazy-0.6938534114443213"
Application.config.namespaced.lazy #=> "namespaced.lazy-0.747101587617097"
Olderposts Newerposts

Copyright © 2017 Running with Ruby

Theme by Anders NorenUp ↑