Tag: bash script

Introduction

As some of you might know, around one year ago I've developed small (around 1k PHP and bash code lines) integration software, to test Rails based apps "outside" of my own computer. I hate to wait when CPU is around 100% and it slows down my dev environment. Moving tests (for example tests of whole application after bigger re-factoring) to a different machine is really convenient:

1. Don't need to wait when developing
2. On a "non-gui" machine tests will probably run faster
3. Everything can be "clicked" via web interface so it is fast
4. Tests history and code coverage history available when needed

However my software had some disadvantages:

1. PHP based
2. Bad GUI (not working properly all the time)
3. No Ruby versions management
4. No gemsets (everything stored in one gemset per Ruby version)
5.  No locks on applications - could start same tests more then one time simultaneously
6. Messy output storage (needed to refer directly to files with output (not in GUI))
7. Other annoying bugs ;)

Despite the disadvantages of this software it served me quite well. However I've decided to rewrite it to Rails to speed up development. I know about Hudson and other continuous integration servers, however I like to do stuff on my own when I can. That's why I've decided to write my own small ruby integration server. I'll be posting here tips and source codes so you can build your own dedicated server software. I am writing this at the same time as I create this software, so probably sometimes it will be inconsistent - sorry ;)

One Ruby to rule them all!

Ok - lets start! The basic purpose of this software is to test applications with different Ruby versions (REE, 1.9.2, 1.8.7, etc). But how to manage Rubies from Ruby? We need to (well maybe we don't need to, but it is idea I came up with) create some bash scripts and then we will enclose them with Ruby code.

We need to be able to do following things with our Ruby code:

1. List Rubies
2. List available Rubies
3. Install Rubies
4. Uninstall Rubies

Listing RVM Rubies

We will be creating some bash scripts and then execute them from our application. Lets start from listing Rubies:

#!/bin/bash       

# Returns list of RVM Rubies installed

init_rvm(){
  if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then
    source "$HOME/.rvm/scripts/rvm"
  elif [[ -s "/usr/local/rvm/scripts/rvm" ]] ; then
    source "/usr/local/rvm/scripts/rvm"
  else
    printf "ERROR: An RVM installation was not found.\n"
  fi
}

perform_task(){
  init_rvm
  rvm list
}
perform_task

Nothing special - now something more fancy.

Installing RVM Rubies

#!/bin/bash       

# Install selected Ruby version
# 3 parameters required:
#   1) RVM ruby version to be installed
#   2) Absolute path to place (file) where save output
#   3) Unique id for lock file name

die () {
    echo >&2 "$@"
    exit 1
}

[ "$#" -gt 2 ] || die "Minimum 3 parameter required! $# provided"

ruby_version="$1"
output_path="$2"
lock_file="/tmp/$3_rvm_install.lock"

init_rvm(){
  if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then
    source "$HOME/.rvm/scripts/rvm"
  elif [[ -s "/usr/local/rvm/scripts/rvm" ]] ; then
    source "/usr/local/rvm/scripts/rvm"
  else
    printf "ERROR: An RVM installation was not found.\n"
  fi
}

perform_task(){
  init_rvm
  touch $lock_file
  rvm install $ruby_version
  rm $lock_file
}

perform_task | tee $output_path

Why use tee instead of redirecting stream? Well I wanted to save output into a file but also I would like to be able to fetch it directly in Ruby. Of course this task can take a lot of time, so it will be ignited in background (how to do so - later ;) ). All the scripts parameters will come directly from application. We put a lock when we start to install and we will remove it after it's done, so we will know when everything started and ended.

Uninstalling Ruby versions

This script is similar to the one above:

#!/bin/bash       

# Uninstall selected Ruby version
# Parameters:
#  1) Ruby version to be uninstalled
#  2) Output file path

die () {
    echo >&2 "$@"
    exit 1
}

[ "$#" -gt 1 ] || die "Minimum 2 parameter required! $# provided"

ruby_version="$1"
output_path="$2"

init_rvm(){
  if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then
    source "$HOME/.rvm/scripts/rvm"
  elif [[ -s "/usr/local/rvm/scripts/rvm" ]] ; then
    source "/usr/local/rvm/scripts/rvm"
  else
    printf "ERROR: An RVM installation was not found.\n"
  fi
}

perform_task(){
  init_rvm
  rvm uninstall $ruby_version
}

perform_task | tee $output_path

Uninstalling is quite fast so we don't need to add lock mechanism. We will run script and wait till it's finished (will not be executed in background).

List available Rubies

#!/bin/bash       

# Returns list of RVM Rubies installed

init_rvm(){
  if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then
    source "$HOME/.rvm/scripts/rvm"
  elif [[ -s "/usr/local/rvm/scripts/rvm" ]] ; then
    source "/usr/local/rvm/scripts/rvm"
  else
    printf "ERROR: An RVM installation was not found.\n"
  fi
}


perform_task(){
  init_rvm
  rvm list known
}

perform_task

Executing bash scripts in Ruby

Store those scripts somewhere in your app (for example in script/tasks/rvm) and make them all executable (chmod 755 ./scriptname). You can also test them however they work fine :)

Executing system commands in Ruby is easy (if you're not familiar with this topic read this and this).

I will be using two commands:

• system - will run command and wait till it's done
• `command` - will run command in a sub shell so we don't need to wait till it's done

Let's create a class to manage commands executing:

# Will execute system command as a user
# If use_sudo set to true - will use sudo su - user -c syntax
class Command
  attr_reader :command

  def initialize(command, *p)
    @parameters = p
    @prefix = prefix
    @command = "#{@prefix} #{exec_command(command)}"
  end

  # Ignites command (will start it but will not wait till its ended
  def ignite
    IO.popen(@command+parameters)
  end

  # Will run command and return result
  def run
    `#{@command+parameters}`
  end

  private

  def parameters
    out = ''
    @parameters.each {|p| out += " #{p}" }
    out
  end

  def exec_command(command)
    command
  end

  def prefix
    user = 'user'
    use_sudo = false
    "#{"sudo su - #{user} -c " if use_sudo}"
  end

end

It just wraps executing into a nice small class. This piece of code is easy - the only "weird" stuff can be this:

sudo su - #{user} -c 

This syntax can be helpful (but it is not safe at all!!!!) when we want to execute code as a different user than a current one.

Lets test it:

c1 = Command.new('ls')
c1.run => "command.rb\nrvm.rb\nscript.rb\n"

# We can add some parameters
c2 = Command.new('du', '-sh')
c2.run => "132K\t.\n"

# Start application
c3 = Command.new('gedit')
c3.ignite => #<IO:fd 3>

If you still don't understand difference between run and ignite try running following stuff and watch console:

c1 = Command.new('gedit')
c1.run
# Don't close Gedit and see script console
c1.ignite

So now we can execute our scripts like this:

c = Command.new(File.join(Rails.root, path, script.sh), *params)
c.run / c.ignite

But writing this all the time can be inconvenient so let's wrap it with an another class:

class Script < Command

  private

  def exec_command(command)
    File.join(Rails.root, 'script', 'tasks', "#{command}.sh")
  end

end

Now we can execute our RVM scripts much easier:

s = Script.new('ruby_install', '1.9.2-p0', '/path/rvm.log', 12345)
s.ignite

Still it is "low level" and we need to make it better, but we will do this in a next part of this tutorial.

In next part we will create gemsets management stuff:

RVM::Ruby.create_gemset('name')
RVM::Ruby.delete_gemset('name')

Tutorial parts

1. RVM Ruby version management directly from... Ruby
2. Handling Rubies and gemsets management easier (from Ruby)