Table of Contents
Legacy code is never the way we would like it to be
There are days, when you don't get to write your new shiny application using Grape and JSON-API. Life would be much easier, if we could always start from the beginning. Unfortunately we can't and one of the things that make some developers more valuable that others is their ability to adapt. The app might be outdated, it might have abandoned gems, but if you're able to introduce new concepts to it (without breaking it and having to rewrite everything), you should be able to overcome any other difficulties.
ActiveRecord::Validations is not the way to go
If you use Rails, then probably you use ActiveRecord::Validations as the main validation layer. Model validations weren't designed as a protection layer for ensuring incoming data schema consistency (and there are many more reasons why you should stop using them).
External world and its data won't always resemble your internal application structure. Instead of trying to adapt the code-base and fit it into the world, try making the world as much constant and predictable as it can be.
One of the best things in that matter is to ensure that any endpoint input data is as strict as it can be. You can do quite a lot, before it gets to your ActiveRecord models or hits any business logic.
Dry-Validation - different approach to solve different problems
Unlike other, well known, validation solutions in Ruby, dry-validation takes a different approach and focuses a lot on explicitness, clarity and precision of validation logic. It is designed to work with any data input, whether it’s a simple hash, an array or a complex object with deeply nested data.
Dry-validation is a library that can help you protect your typical Rails API endpoints better. You will never control what you will receive, but you can always make sure that it won't reach your business.
Here's a basic example of how it works (assuming we validate a single hash):
BeaconSchema = Dry::Validation.Schema do UUID_REGEXP = /\A([a-z|0-9]){32}\z/ PROXIMITY_ZONES = %w( immediate near far unknown ) required(:uuid).filled(:str?, size?: 32, format?: UUID_REGEXP) required(:rssi).filled(:int?, lteq?: -26, gteq?: -100) required(:major).filled(:int?, gteq?: 0, lteq?: 65535) required(:minor).filled(:int?, gteq?: 0, lteq?: 65535) required(:proximity_zone).filled(:str?, included_in?: PROXIMITY_ZONES) required(:distance) { filled? & ( int? | float? ) & gteq?(0) & lteq?(100) } end data = {} validation_result = BeaconSchema.call(data) validation_result.errors #=> {:uuid=>["is missing"], :rssi=>["is missing"], ... }
The validation result object is really similar to the validation result of ActiveRecord::Validation in terms of usability.
Dry Ruby on Rails API
Integrating dry-validation with your Ruby on Rails API endpoints can give you multiple benefits, including:
- No more strong parameters (they will become useless when you cover all the endpoints with dry-validation)
- Schemas testing in isolation (not in the controller request - response flow)
- Model validations that focus only on the core details (without the "this data is from form and this from the api" dumb cases)
- Less coupling in your business logic
- Safer extending and replacing of endpoints
- More DRY
- Nested structures validation
Non-obstructive data schema for a controller layer
There are several approaches you can take when implementing schema validation for your API endpoints. One that I find useful especially in legacy projects is the #before_action approach. Regardless whether #before_action is (or is not) an anti-pattern, in this case, it can be used to provide non-obstructive schema and data validation that does not interact with other layers (like services, models, etc.).
To implement such protection, you only need a couple lines of code):
class Api::BaseController < ApplicationController # Make sure that request data meets schema requirements before_action :ensure_schema! # Accessor for validation results hash attr_reader :validation_result # Schema assigned on a class level class_attribute :schema private def ensure_schema! @validation_result = self.class.schema.call(params) return if @validation_result.success? render json: @validation_result.errors.to_json, status: :unprocessable_entity end end
and in your final API endpoint that inherits from the Api::BaseController:
class Api::BeaconsController < Api::BaseController self.schema = BeaconSchema def create # Of course you can still have validations on a beacon level Beacon.create!(validation_result.to_h) head :no_content end end
Summary
Dry-validation is one of my "must use" gems when working with any type of incoming data (even when it comes from my other systems). As presented above, it can be useful in many situations and can help reduce the number of responsibilities imposed on models.
This type of validation layer can also serve as a great starting point for any bigger refactoring process. Ensuring incoming data and schema consistency without having to refer to any business models allows to block the API without blocking the business and models behind it.
Cover photo by: brlnpics123 on Creative Commons 2.0 license. Changes made: added an overlay layer with an article title on top of the original picture.
March 30, 2017 — 08:13
Thanks!
I should try it in my project.
March 30, 2017 — 15:00
Don’t you think that solution is a little bit too ‘magic’? What do you think about doing it with composition? moving ‘schema’ from class attribute to method in every API class , call validation explicitly and then call some method like `fail!(validation_result, :unprocessable_entity)` when validation failed?
Your solution might be good and because I’m huge ‘composition over inheritance’ fan so it may be the issue with me, not your solution.
Great job on sharing that knowledge
April 10, 2017 — 07:21
Agreed.
But I guess there is only 1 example here to show how it could be done in one way
And other people can feel free to write represent their own ways too :)
April 19, 2017 — 11:51
Keep in mind, that this is a refactoring case. Introducing new concepts
(not always used by other developers in the team) to an existing code
base might not be the best. But I agree – you could make it with
composition as well :)
April 20, 2017 — 19:15
Using JSON schema will: 1) allow one to generate API docs and 2) allow non-Ruby clients to conform to your data requirements.
Why did you choose not use it?
April 20, 2017 — 21:04
Could you give some reasons as to why ActiveRecord::Validations as the main validation layer is not the way to go? I understand the argument for validating at the controller level, but whether you validate them in the controller level or the model level, doesn’t it end up having the same result?
April 25, 2017 — 09:47
The json-schema ruby gem has horrible error messages output, especially when it comes to allOf, oneOf, anyOf options. So schema validation becomes a pretty much hell of a thing. Dry-validation gives more validation flexibility, but I don’t protect it also, because it has loads of drawbacks and limitations, and it is very hard to debug.
April 25, 2017 — 10:07
Just try to apply the dry-validation in more complicated cases, and you’ll miss the ActiveRecord::Validations :) It has poor documentation and you have to figure out the solution on your own. For example, create predicate and high-level rule to check if defaultGateway IP address is inside the allowed range of given IPs in addressing object. The solution involves 3 props here: address and netmask in addressing object, and address in defaultGateway object. And you need to check first if it has `custom` type and `enabled` is true.
https://uploads.disquscdn.com/images/30346e48618a72b9ee8e40c077737beb46cf8ffbc061d58e855b78751694011e.png
This was my last task, and it had been solved, but I’m still afraid of new cases to validate :))
April 25, 2017 — 12:34
FWIW we’re extracting schema part of dry-validation into dry-schema, and dry-validation will focus on more complex validations and address all the known bugs and limitations. You can read more about this effort here: https://discuss.dry-rb.org/t/plans-for-dry-validation-dry-schema-a-new-gem/215
April 25, 2017 — 12:45
AR::Validations are for validating database records, the input your application receives often doesn’t match what you store in the database. That’s the fundamental problem with AR::Validations. Another thing is that self-validating objects is a bad idea, conceptually it’s broken, as it mixes “data” with object’s state. In AR 5 an AR object has **146** methods (excluding predicate, bang and the ones starting with underscore), good luck validating data which includes a key which clashes with one of those names. That’s not all, self-validating objects means that it’s allowed for a domain object to be in an invalid state. This means that your object exposes an interface which may crash depending on the state in which an object is currently in.
Data validation is really one of the most complex things we need to do, having a dedicated library that handles just this is a good idea. Sure, not everybody will need it, a lot of people will continue to use AR::V and they’ll be happy with it, but it doesn’t change the fact we *need* something better, hence the effort behind dry-validation. For instance I started working on it after years of frustration with AR::V (and AM::V in general).
April 25, 2017 — 12:48
That is definitely good news :)