Running with Ruby

Category: Software (page 1 of 63)

Dry-Validation as a schema validation layer for Ruby on Rails API

29-03-2017 / / 11 Comments

Legacy code is never the way we would like it to be

There are days, when you don’t get to write your new shiny application using Grape and JSON-API. Life would be much easier, if we could always start from the beginning. Unfortunately we can’t and one of the things that make some developers more valuable that others is their ability to adapt. The app might be outdated, it might have abandoned gems, but if you’re able to introduce new concepts to it (without breaking it and having to rewrite everything), you should be able to overcome any other difficulties.

ActiveRecord::Validations is not the way to go

If you use Rails, then probably you use ActiveRecord::Validations as the main validation layer. Model validations weren’t designed as a protection layer for ensuring incoming data schema consistency (and there are many more reasons why you should stop using them).

External world and its data won’t always resemble your internal application structure. Instead of trying to adapt the code-base and fit it into the world, try making the world as much constant and predictable as it can be.

One of the best things in that matter is to ensure that any endpoint input data is as strict as it can be. You can do quite a lot, before it gets to your ActiveRecord models or hits any business logic.

Dry-Validation – different approach to solve different problems

Unlike other, well known, validation solutions in Ruby, dry-validation takes a different approach and focuses a lot on explicitness, clarity and precision of validation logic. It is designed to work with any data input, whether it’s a simple hash, an array or a complex object with deeply nested data.

Dry-validation is a library that can help you protect your typical Rails API endpoints better. You will never control what you will receive, but you can always make sure that it won’t reach your business.

Here’s a basic example of how it works (assuming we validate a single hash):

BeaconSchema = Dry::Validation.Schema do
  UUID_REGEXP = /\A([a-z|0-9]){32}\z/
  PROXIMITY_ZONES = %w(
    immediate
    near
    far
    unknown
  )

  required(:uuid).filled(:str?, size?: 32, format?: UUID_REGEXP)
  required(:rssi).filled(:int?, lteq?: -26, gteq?: -100)
  required(:major).filled(:int?, gteq?: 0, lteq?: 65535)
  required(:minor).filled(:int?, gteq?: 0, lteq?: 65535)
  required(:proximity_zone).filled(:str?, included_in?: PROXIMITY_ZONES)
  required(:distance) { filled? & ( int? | float? ) & gteq?(0) & lteq?(100)  }
end

data = {}
validation_result = BeaconSchema.call(data)
validation_result.errors #=> {:uuid=>["is missing"], :rssi=>["is missing"], ... }

The validation result object is really similar to the validation result of ActiveRecord::Validation in terms of usability.

Dry Ruby on Rails API

Integrating dry-validation with your Ruby on Rails API endpoints can give you multiple benefits, including:

  • No more strong parameters (they will become useless when you cover all the endpoints with dry-validation)
  • Schemas testing in isolation (not in the controller request – response flow)
  • Model validations that focus only on the core details (without the “this data is from form and this from the api” dumb cases)
  • Less coupling in your business logic
  • Safer extending and replacing of endpoints
  • More DRY
  • Nested structures validation

Non-obstructive data schema for a controller layer

There are several approaches you can take when implementing schema validation for your API endpoints. One that I find useful especially in legacy projects is the #before_action approach. Regardless whether #before_action is (or is not) an anti-pattern, in this case, it can be used to provide non-obstructive schema and data validation that does not interact with other layers (like services, models, etc.).

To implement such protection, you only need a couple lines of code):

class Api::BaseController < ApplicationController
  # Make sure that request data meets schema requirements
  before_action :ensure_schema!

  # Accessor for validation results hash
  attr_reader :validation_result
  # Schema assigned on a class level
  class_attribute :schema

  private

  def ensure_schema!
    @validation_result = self.class.schema.call(params)
    return if @validation_result.success?

    render json: @validation_result.errors.to_json, status: :unprocessable_entity
  end
end

and in your final API endpoint that inherits from the Api::BaseController:

class Api::BeaconsController < Api::BaseController
  self.schema = BeaconSchema

  def create
    # Of course you can still have validations on a beacon level
    Beacon.create!(validation_result.to_h)
    head :no_content
  end
end

Summary

Dry-validation is one of my “must use” gems when working with any type of incoming data (even when it comes from my other systems). As presented above, it can be useful in many situations and can help reduce the number of responsibilities imposed on models.

This type of validation layer can also serve as a great starting point for any bigger refactoring process. Ensuring incoming data and schema consistency without having to refer to any business models allows to block the API without blocking the business and models behind it.

Cover photo by: brlnpics123 on Creative Commons 2.0 license. Changes made: added an overlay layer with an article title on top of the original picture.

Sharing models between Rails apps – Keeping Rails engine migrations in the engine

31-01-2017 / / 1 Comment

Note 1: If you have an option to use micro services and/or event-sourcing, go for it! Rails solutions based on shared models and a single shared database, can bring you in a longer perspective more harm than good.

Note 2: This approach is great when your data is tightly coupled and you can’t easily switch from a single app to a distributed model.

Sharing models between Rails apps – basics

The concept of shared models is really well described by Fabio Akita in following articles:

However, his approach towards migrations comes from his really specific use-case (two DBs in a single app – one shared and one private).

Managing migrations – standard Rails engine approach

Standard Rails engine approach assumes that your migrations will be copied from the engine into the application when you run following command:

bundle exec rake engine_name_engine:install:migrations

This is great when:

  • You have a single “master” application that you want to decompose with engines
  • you have multiple applications with separate databases and you want to use business logic from the engine from each of them
  • If you want to to have a single “master” application that is supposed to run all the migrations from the engine

However with some benefits, you get a huge (in my case) drawback – when you copy migrations, their timestamp is being changed. It means that if you share same database across multiple applications that also share the same engine, you will end up with a single migration being executed (assuming you install the migrations) from each of the separate applications.

Single database and no master application

This won’t do if your case is similar to mine, that is:

  • Single database
  • Multiple applications that need to share same models and scopes
  • Migrations should be executed in the first application that is being deployed after the model engine change (not from the “master” app)
  • There should not be any patching / adding  code into any of the apps that will use shared models gem

Keeping your migrations inside your model engine

Solution for such a case (in which all the models are being kept inside the gem) is pretty simple: you just need to append migrations into your apps migrations path without:

  • Copying them from the model engine gem
  • Changing the timestamp
  • Executing the same migrations multiple times

To achieve such a behavior, we will take advantage of how Rails config paths, migrations and initializers work:

  • Config paths aren’t bound to the Rails.root directory (which means that they can use files from gems and other locations)
  • Config paths are appendable (which means we can add our gem migrations into the app migration list without changing timestamps and copying files)
  • Engine initializer allow us to bind this process from the model gem, keeping the apps untouched (they will think that those migrations are theirs)
  • Rails migrations execution details are stored in schema_migrations table, so unless executed exactly the same moment (so transactions overlap) a single gem migration will not be executed twice

All of this comes down few lines of Ruby inside Rails engine engine class (engine_path/lib/engine_name/engine.rb):

initializer :append_migrations do |app|
  # This prevents migrations from being loaded twice from the inside of the
  # gem itself (dummy test app)
  if app.root.to_s !~ /#{root}/
    config.paths['db/migrate'].expanded.each do |migration_path|
      app.config.paths['db/migrate'] << migration_path
    end
  end
end

TL;DR – Final solution

engine_path/lib/engine_name/engine.rb:

module ModelEngine
  class Engine < ::Rails::Engine
    initializer :append_migrations do |app|
      # This prevents migrations from being loaded twice from the inside of the
      # gem itself (dummy test app)
      if app.root.to_s !~ /#{root}/
        config.paths['db/migrate'].expanded.each do |migration_path|
          app.config.paths['db/migrate'] << migration_path
        end
      end
    end
  end
end

Summary

Most of the time sharing models is bad, but there are some cases app data is really tightly coupled together and exposing API with building microservices around it would mean a huge overhead. For such cases model gem with internal migrations might be a great solution.

Warning: If you decide to go that road, please make sure, that:

  • Your models are stable
  • Your models are slim and without any business logic
  • Your models don’t have any callbacks or external dependencies
  • If your models have external dependencies, make them model gem dependencies
  • Your models are loosely coupled (if you follow Akitas approach with concerns it won’t be hard)
  • Your applications are well tested
  • Your model gem is well tested
  • You don’t use model validations – instead you can use Reform, Dry-Validations or any other solution that allows you to move validations logic out of models
  • All the model related things and migrations are inside the model gem
  • Migrations from external gems like Devise or FriendlyId are also inside the gem

With all of this in mind, you should be fine :-)

Cover photo by: Unsplash on Creative Commons 0 license.

Olderposts

Other stuff

Social media

Tags

ActiveModel ActiveRecord backup bash bash script chat deprecation warning gem HTTP Javascript juggernaut karafka Linux migration Mongo Mongoid Mysql paperclip passenger Performance PHP Rails Rails 2.3.8 Rails3 Rails3.1 Rails 4 Rails4 Rails 4.0 RoR routes routing error Ruby Ruby 1.8.7 Ruby 1.9 Ruby 1.9.2 Ruby 1.9.3 Ruby on Rails RVM safety server shell Sidekiq Sinatra SVN ubuntu

Archives

Recent Posts

Copyright © 2017 Running with Ruby

Theme by Anders Noren — Up ↑