Running with Ruby

Tag: Ruby (page 2 of 80)

Exploring a critical Net::Protocol issue in Ruby 2.6.0p0 and how it can lead to a security problem


This bug has been fixed in 2.6.1. Please upgrade and all should be good.

If you do any HTTP communication (HTTP requests, Elasticsearch, etc) do not upgrade to 2.6.0p0 or apply the patch below as soon as possible.

Ruby is eating up characters when pushed over HTTP

Ruby 2.6.0 has been released not long ago. Not many are unfortunately aware of a major bug that was introduced with these release.

This bug can affect you in many ways, some of which you may not even be aware. All may run well up until you decide to send a particular type of payload and then, things will get interesting.

What am I talking about?

This. What does it even mean? Well in the best scenario it means, that you will end up having a critical error like so:''), 'あ'*100_000)
Traceback (most recent call last):
       16: from /net/http.rb:502:in `block in post'
       15: from /net/http.rb:1281:in `post'
       14: from /net/http.rb:1493:in `send_entity'
       13: from /net/http.rb:1479:in `request'
       12: from /net/http.rb:1506:in `transport_request'
       11: from /net/http.rb:1506:in `catch'
       10: from /net/http.rb:1507:in `block in transport_request'
        9: from /net/http/generic_request.rb:123:in `exec'
        8: from /net/http/generic_request.rb:189:in `send_request_with_body'
        7: from /net/protocol.rb:247:in `write'
        6: from /net/protocol.rb:265:in `writing'
        5: from /net/protocol.rb:248:in `block in write'
        4: from /net/protocol.rb:275:in `write0'
        3: from /net/protocol.rb:275:in `each_with_index'
        2: from /net/protocol.rb:275:in `each'
        1: from /net/protocol.rb:280:in `block in write0'

However, there’s a much more interesting case that you can encounter. You can end up sending data that will be trimmed in a way that will make your server receive incomplete yet valid information.

That is not a security issue per se but can be a massive problem if you use your format as a protocol between some internal services.

Sidenote: bjeanes reported on Github, that this bug can also corrupt JSON in a way that will make it parsable but incorrect regarding data it consists.

Set HTTP API as a POC of this bug

To illustrate how this bug can become problematic and hard to debug, let’s build an HTTP based API that implements basic set operations via the web.

Some assumptions for the sake of simplicity:

  • we always send data in the following format: DATA,COMMAND;
  • we have three commands: GET, ADD and DEL;
  • to save a couple of bytes, when no command provided as a second argument, we run an ADD command;

This is how our abstract API could work:

client =
client.get #=> []
client.add('12313131') #=> ['12313131']
client.add('msg') #=> ['12313131', 'msg']
client.del('msg') #=> ['12313131']

A set API server implementation

The implementation of such an API server will just take us a couple of lines in Ruby:

require 'webrick'
require 'set'

server = 3000)
set =

server.mount_proc '/' do |req, res|
  data, action = req.body.split(',')
  action ||= 'ADD'

  # Return set data for any command, no need to handle GET
  case action
  when 'ADD'
  when 'DEL'

  res.body = set.to_a.to_s

trap('INT') { server.shutdown }


You can start it by running:

ruby server.rb
[2019-01-09 22:38:58] INFO  WEBrick 1.4.2
[2019-01-09 22:38:58] INFO  ruby 2.6.0 (2018-12-25)
[2019-01-09 22:38:58] INFO  WEBrick::HTTPServer

A set API client implementation

The client is not much more complicated:

require 'net/http'

class Api
  HOST = 'localhost'
  PORT = 3000

  def initialize
    @http =, PORT)

  def get
    request nil, 'GET'

  def add(data)
    request data, 'ADD'

  def del(data)
    request data, 'DEL'


  def request(data, cmd)
      .new('/', 'Content-Type': 'application/json')
      .tap { |req| req.body = "#{data},#{cmd}" }

client =

When executed, you end up with exactly what we’ve wanted to achieve:

puts client.get #=> []
puts client.add('12313131') #=> ['12313131']
puts client.add('msg') #=> ['12313131', 'msg']
puts client.del('msg') #=> ['12313131']

Risk of an uncompleted payload

So far so good. We have an excellent API that we can use for storing anything we want. And here magic starts.

We decide to store some analytics results, that are used by other APIs to grant access to some super essential and expensive business information™.

It doesn’t matter what the results are. All we need to know from our perspective, is the fact, that it will fit into memory. So, we hand out our API client code to other developers; we run our server and… in the middle of the night the phone rings:

Data that is supposed to be deleted is still available. We constantly run the DEL command but nothing disappears! We need to revoke all the access ASAP!

How can it be!? This service has been running for months now, and everything was good. There was a recent update in Ruby, but even after that specs were passing and the service has been running for at least two weeks.

And this is the moment when this bug presents itself in all the glory. For big enough payload, Ruby is trimming data that is being sent, and unfortunately for us, it trims last three letters, that is the full DEL command. When we run an ADD and DEL on a given string, we expect it not to be in the results anymore, however…

Note: the dots from the payload below aren’t usual dots but Unicode middle dots – that is important.

PAYLOAD_SIZE = 8_301_500
data = 'a' * PAYLOAD_SIZE + '···'

client =
puts client.get #=> ["aaaaaaaaaaaa...\xC2\xB7\xC2\xB7\xC2\xB7"]

The data is still there! Because the data consists multibyte characters, the payload got trimmed, and we’ve ended up with a non-direct GET operation (DATA,) instead of a DEL. We had three multibyte characters in the data, and because of that, Ruby removed three last characters from the string before sending it to the server.

Patching things up

As a temporary patch you can use the body_stream instead of using body combined with Ruby StringIO:

  .new('/', 'Content-Type': 'application/json')
  .tap { |req| req.body_stream = }
  .tap { |req| req.content_length = operation.bytesize }

or if you use Faraday, you can just apply following patch:

module NetHttpFaradayPatch
  def create_request(env)
    super.tap do |request|
      if env[:body].respond_to?(:read)
        request.content_length = env[:body].size


Here’s the proper fix, however Ruby 2.6.1 has not yet been released.


It’s a rather unpleasant bug, and I’m quite surprised that despite being fixed, new Ruby version hasn’t had been released yet Ruby 2.6.1 has been released and it fixes the issue. For now, if my patches work for you, that’s great but anyhow I would advise you to downgrade to Ruby 2.5.3 . It’s hard to be sure, that there aren’t other scenarios in which this bug may become even more problematic.

Cover photo by theilr on Attribution-ShareAlike 2.0 Generic (CC BY-SA 2.0) license.

Simplifying internal validations using Dry-Validation

When building APIs for other developers, it’s often important to draw the line between other programmers input data and the internal world of your library. This process is called data validation and you’re probably familiar with this name.

What you may not know, is the fact that it can be achieved in many ways.

One that I particularly like is by using the dry-validation library. Here’s an example on how you can separate the validation from the actual business logic without actually changing the API of your library.

The inline way

The easiest way to provide validations is to embed the checks in a place where you receive the data.

This approach is great for super simple cases like the one below:

def sum(arg1, arg2)
  raise ArgumentError unless arg1
  raise ArgumentError unless arg2

  arg1.to_i + arg2.to_i

sum(2, nil) #=> ArgumentError (ArgumentError)

However, if you decide to follow this road, you will quickly end up with something really unreadable.

This code sample is taken from the ruby-kafka library and it’s used to validate the method input. I’ve removed the business logic parts as they aren’t relevant to the context of this article:

def build(
  ca_cert_file_path: nil,
  ca_cert: nil,
  client_cert: nil,
  client_cert_key: nil,
  client_cert_chain: nil,
  ca_certs_from_system: nil
  return nil unless ca_cert_file_path ||
                    ca_cert ||
                    client_cert ||
                    client_cert_key ||
                    client_cert_chain ||

  if client_cert && client_cert_key
    # business irrelevant to the checks
    if client_cert_chain
      # business irrelevant to the checks
  elsif client_cert && !client_cert_key
    raise ArgumentError, "initialized with ssl_client_cert` but no ssl_client_cert_key"
  elsif !client_cert && client_cert_key
    raise ArgumentError, "initialized with ssl_client_cert_key, but no ssl_client_cert"
  elsif client_cert_chain && !client_cert
    raise ArgumentError, "initialized with ssl_client_cert_chain, but no ssl_client_cert"
  elsif client_cert_chain && !client_cert_key
    raise ArgumentError, "initialized with ssl_client_cert_chain, but no ssl_client_cert_key"

  # business

Despite looking simple, the if-elsif validation is really complex and it brings many things to the table:

  • it checks several variables,
  • mixes the checks together due to the if-flow,
  • in the end it actually only checks the presence of the variables,
  • despite expecting string values, it will work with anything that is provided,
  • it forces us to spec out the validation cases with the business logic as they are coupled together.

Luckily for us, there’s a better way to do that.

The private schema way

We can achieve the same functionality and much more just by extracting the validations into a separate internal class. Let’s build up only the interface for now.

Note, that I’m leaving the ArgumentError and the external API intact, as I don’t want this change to impact anything that is outside of this class:

require 'dry-validation'

# Empty schema for now, we will get there
SCHEMA = Dry::Validation.Schema {}

def build(
  ca_cert_file_path: nil,
  ca_cert: nil,
  client_cert: nil,
  client_cert_key: nil,
  client_cert_chain: nil,
  ca_certs_from_system: nil
  input = {
    ca_cert_file_path: ca_cert_file_path,
    ca_cert: ca_cert,
    client_cert: client_cert,
    client_cert_key: client_cert_key,
    client_cert_chain: client_cert_chain,
    ca_certs_from_syste: ca_certs_from_system

  # Do nothing if there's nothing to do
  return nil if input.values.all?(&:nil?)

  results =
  raise ArgumentError, results.errors unless results.success?

  # Business logic

We’ve managed to extract the validation logic outside.

Thanks to that, now we have:

  • separation of responsibilities,
  • business applying method that we can test against only valid cases,
  • validation object that we can test in isolation,
  • much cleaner API that can be easier expanded (new arguments, new data types supported, etc) and/or replaced,
  • way to handle more complex validations (types, formats, etc),
  • support for reporting multiple issues with the input at the same time.

We can now perform all the checks and only when everything is good, we will run the business. But what about the validation itself?

Actually all the validations below are copy-pasted from the karafka repository. Here’s the dry-validation documentation.

require 'dry-validation'

SCHEMA = Dry::Validation.Schema do
  ].each do |encryption_attribute|


    client_cert_with_client_cert_key: %i[
  ) do |client_cert, client_cert_key|
    client_cert.filled? > client_cert_key.filled?

    client_cert_key_with_client_cert: %i[
  ) do |client_cert, client_cert_key|
    client_cert_key.filled? > client_cert.filled?

    client_cert_chain_with_client_cert: %i[
  ) do |client_cert, client_cert_chain|
    client_cert_chain.filled? > client_cert.filled?

    client_cert_chain_with_client_cert_key: %i[
  ) do |client_cert_chain, client_cert_key|
    client_cert_chain.filled? > client_cert_key.filled?

The execution effect is also really good:

build(ca_cert: 2) #=> {:ca_cert=>["must be a string"]} (ArgumentError)


Whenever you find yourself adding some inline validations, stop and think twice, there’s probably a better and more extendable way to do it.

Originally published at The Castle blog.

Olderposts Newerposts

Copyright © 2019 Running with Ruby

Theme by Anders NorenUp ↑