Running with Ruby

Tag: deploy (page 1 of 3)

Gitlab, Ruby private repository, Docker container and safe way to add SSH key for build process only

Note

This is not a tutorial on how to use Docker with SSH keys and private repositories. This is a solution to a particular safety issue when building Docker containers.

I won’t get into details on how to create a Dockerfile with a SSH key, etc – there’s enough about that in the Internet.

Problem – container stored SSH key

During the Docker build process, we need a SSH key for bundle install. We need it, because we have a Gitlab with private repositories that we want to pull. Since Docker caches this key, it will go to a production environment together with the whole container. It is unsafe. If someone got somehow into the container, he could use this key to get into your repositories.

Solution – during build only, one-time SSH key pair

This issue can be easily solved. To do so, we need to:

  1. Generate SSH key pair
  2. Add it to our Gitlab instance using Gitlab API
  3. Add private key to Docker
  4. Docker b uild (and bundle install)
  5. Remove/revoke our key from Gitlab
  6. Deploy container

There’s still going to be a SSH key in the container, but it won’t be valid. Since it will be valid only during the build process (which happens on the CI), there’s no risk. The key gets revoked before we deploy container.

Code

For Gitlab API communication we will be using Gitlab API gem.

require 'gitlab'
require 'fileutils'

# I assume you have this script in the same place where your Dockerfile

# You can find your private token on the account page of your gitlab user
Gitlab.configure do |config|
  config.endpoint = 'http://your-gitlab/api/v3'
  config.private_token  = 'deploy/docker gitlab user private token'
end

# Add this if you use self signed SSL cert with Gitlab
OpenSSL::SSL::VERIFY_PEER = OpenSSL::SSL::VERIFY_NONE

'ssh-keygen -q -t rsa -f ./id_rsa -N ""'

key = Gitlab.create_ssh_key("docker-#{Time.now.to_s}", File.read('./id_rsa.pub'))

# Note that you should provide some sort of failover to remove key if build fails
system('docker build .')

Gitlab.delete_ssh_key(key.id)

FileUtils.rm('./id_rsa')
FileUtils.rm('./id_rsa.pub')

Clear memcached without restart with Ruby and Capistrano (Rake task)

After successful Capistrano update (cap deploy) if we use Memcached – we should clear it. How can we do it without root access to the server? There are two ways to clear memcached without restarting it:

  • Memcached command: flush_all
  • Rails.cache.clear

One flush to rule them all – flush_all

If we have a dedicated server with one application on it – we can clear whole memcached memory, to do so, we create a rake task (/lib/tasks/memcached.rake):

require 'socket'

namespace :memcached do
  desc 'Flushes whole memcached local instance'
  task :flush do
    server  = '127.0.0.1'
    port    = 11211
    command = "flush_all\r\n"

    socket = TCPSocket.new(server, port)
    socket.write(command)
    result = socket.recv(2)

    if result != 'OK'
      STDERR.puts "Error flushing memcached: #{result}"
    end

    socket.close
  end
end

Usage:

bundle exec rake memcached:flush

It is worth mentioning, that this task doesn’t require the Rails environment to be loaded. If it goes about the server address and port – you can always modify it so it will accept the env settings instead of hardcoding it.

Be aware, that this command will clear out all the data that is stored in Memcached instance, even the data that was used by other applications (other than our). If you want to clear out data used by one of many apps that are using same Memcached server, see the solution presented below.

Clearing single Rails app memcached data – Rails.cache.clear

Apart from flushing all the data that is in Memcached, we can always clear only the Rails cache by creating a really simple rake task (/lib/tasks/memcached.rake):

namespace :memcached do
  desc 'Clears the Rails cache'
  task :flush => :environment do
    Rails.cache.clear
  end
end

The execution process is exactly like in the previous case:

bundle exec rake memcached:flush

In this Rake task we do load the Rails environment (because we want to use Rails.cache instance). In multi application environment, this Memcached cleaning method seems way better because we work with our application scope only.

Capistrano task for clearing Memcached

So we have our rake task, but it would mean nothing without a Capistrano hookup:

namespace :memcached do

  desc "Flushes memcached local instance"
  task :flush, :roles => [:app] do
    run("cd #{current_path} && rake memcached:flush")
  end

end

Now we can use it like this:

bundle exec cap memcached:flush

Or we can hookup it to update process:

after 'deploy:update' do
  memcached.flush
end
Olderposts

Copyright © 2018 Running with Ruby

Theme by Anders NorenUp ↑