Running with Ruby

Category: Ruby (page 1 of 100)

Diffend – OSS supply chain security and management platform for Ruby

I’m incredibly excited to announce a security platform for managing Ruby gems dependencies: diffend.io.

This platform is a result of my involvement in Ruby security matters for years. It all started in early 2018 with a tool to review gems versions diffs. While working on it, I’ve noticed that there’s much more that needs to be handled. Versions diffing while inevitable, by itself is insufficient, that’s why we’ve built this platform.

Getting started

If you’re just interested in the gems diffing, go to my.diffend.io and select any gem and versions you want to view. New releases for all the gems are computed in real-time, but for some of the older ones, you will have to wait a bit.

You can also use a shiny new link available on each RubyGems gems page to review changes against the previous release of the same gem:

If you would want to run a more thoughtful assessment, you can either run this script in your application main directory:

ruby <(curl -s https://my.diffend.io/api/setup/ruby)

or if you are like me and do not want to run scripts from the internet, you can just follow the super short manual with setup instructions here.

If something is not clear or you have any questions, please contact us at our Slack workspace with this invitation link or drop us a line at contact@diffend.io.

What does it do?

In short, Diffend allows you to:

  1. Review changes in between gems releases before you upgrade based on the gems content itself,
  2. Block attempts of even downloading potentially unwanted gems and their versions,
  3. Manage third party dependencies within your organization,
  4. Ensure OSS licensing consistency in your organization,
  5. Get insights on vulnerabilities, memory leaks and licensing problems of your dependencies,
  6. Make dependency audits a part of your workflow,
  7. Get real-time notifications about any new risks that occur in your production systems (coming soon)

It also runs certain types of heuristics and checks to pinpoint potentially “interesting” releases for further semi-manual inspection.

Why do we need it?

OSS supply chain attacks are becoming a more and more common thing. Looking at RubyGems or npm, there are plenty of examples of packages getting hijacked and malicious versions being uploaded. There were already several attacks that were detected and stopped thanks to Diffend and RubyGems close cooperation.

If you just update dependencies without checking them, you’re not actually sure of what you’re putting into production. You should not trust what’s on Github. An attacker can upload something to a registry without pushing it to Github. The only way to be sure is to look at what’s actually on the registry.

When it’s easy to work securely, people are more likely to do it. diffend.io, is another step towards improving Ruby’s security story by letting you generate diffs from any browser and share them as links. This also lends itself to automation: now you can connect Diffend with your Gemfile and make dependency audits a part of your workflow. We hope this will inspire the community with lots of new security ideas that don’t slow you down.

Is it secure?

Diffend was built with security in mind. Platform, plugin, and our gem collect the absolute minimum amount of data to provide you with the services. Both the Bundler plugin and the monitor will be open-sourced, but even now you can download and review their content.

On top of all of that, we’ve been super cautious about what we collect, that’s why:

  1. We do not collect credentials or environment variables;
  2. We do not execute any remote code from our plugin or gem. Never.
  3. We do not access anything except the Gemfile and Gemfile.lock content.
  4. We do not send to ourselves private access keys for any non-public gems.
  5. We are working on a fully anonymous mode where we do not track public IPs

Support us!

Diffend platform is free to use. You don’t even need an account to review the diffs (and you never will). If you like our platform, please consider convincing your company to support us with any amount of money. We’ll just invoice you for the service usage :)

This way, with a bit of funding, we might be able to push forward many security initiatives much faster.

What’s next?

At the moment we are working on several things:

  • Open-sourcing the plugin and the monitor,
  • Real-time production / staging based context aware Slack and e-mail notifications about new risks,
  • Improved heuristics and detection capabilities,
  • Modified Ruby VM for network tracking analysis with pre-execution permissions,
  • Ruby process behaviour tracking,
  • Open-sourcing several of the components for self-service,
  • Fully anonymous mode without collecting any public data.

Diffend is a platform in an alpha stage and under massive development. Some functionalities may not work on every operating system, and some other features may not be available or may be broken. We are working hard to fix and improve the platform, which is why we are counting on your feedback so that we can meet your exact needs faster!

Read more

Building a Ractor based logger that will work with non-Ractor compatible code

Recently Mike Perham shared a tweet with this comment and a code sample on the Ruby 3.0 Ractors.

If this code doesn’t work, how could Rails ever work? Ractor seems fundamentally incompatible with many heavily-used Rails APIs.

require 'logger'

class Rails
  def self.logger
    @logger ||= Logger.new(STDOUT)
  end
end

Ractor.new do
  Rails.logger.info "Hello"
end.take

During the weekend I’ve added support of Ractors in the Diffend.io, a free platform for an OSS supply chain security and management for Ruby and Rails, so I’m relatively fresh with the topic. Mike’s code illustrates one of the issues developers will face when making their code Ractors compatible.

When you try to run it, you will end up with an exception:

terminated with exception (report_on_exception is true):
`take': thrown by remote Ractor. (Ractor::RemoteError)
`logger': can not access instance variables of classes/modules
  from non-main Ractors (RuntimeError)

Is there any way to preserve the Rails#logger API and allow it to be used from any Ractor we want?

There is!

So, let’s start by explaining why this code cannot work:

  def self.logger
    @logger ||= Logger.new(STDOUT)
  end

There are actually 2 problems with this code, though only one is visible immediately:

  1. You cannot access instance variables of the shareable objects from Ractors other than the main one.
  2. You cannot access STDOUT from the non-main Ractor (at least not that way).

The good news is said between the lines: while we cannot use shareable objects and cannot refer to instance variables, we can preserve the Rails.logger API!

class Rails
  def self.logger
    rand
  end
end

Ractor.new do
  Rails.logger
end.take.then { p _1 }

#=> 0.06450369439220172

But we want to share a logger, right? Well, not exactly. What we want is to be able to use the same API to log pieces of information. And that’s the key point here.

We can bypass all of our problems quickly. We just need a separate Ractor that will run all the logging for our application with a standard logger compatible API.

What do we need to achieve this? Not much. We need to:

  1. Create a Ractor that will have the one and only application wide logger.
  2. Create API for logging.
  3. Connect the Ractor to the Rails#logger interface.

It all can be achieved with a few lines of code:

class Rogger < Ractor
  def self.new
    super do
      # STDOUT cannot be referenced but $stdout can
      logger = ::Logger.new($stdout)

      # Run the requested operations on our logger instance
      while data = recv
        logger.public_send(data[0], *data[1])
      end
    end
  end
 
  # Really cheap logger API :)
  def method_missing(m, *args, &_block)
    self << [m, *args]
  end
end

class Rails
  LOGGER = Rogger.new

  def self.logger
    LOGGER
  end
end

Ractor.new do
  Rails.logger.info "Hello"
end

and when we run it, we end up with a different challenge:

terminated with exception (report_on_exception is true):
ruby/3.0.0/logger/formatter.rb:15:in `call': can not access global variables $$ from non-main Ractors (RuntimeError)
  from ruby/3.0.0/logger.rb:586:in `format_message'
  from ruby/3.0.0/logger.rb:476:in `add'
  from ruby/3.0.0/logger.rb:529:in `info'
  from test.rb:23:in `public_send'
  from test.rb:23:in `block in new'

UPDATE: The pull request that I’m talking about below has been merged, so this monkey patch is no longer needed.

It turns out, the Ruby defaulf logging formatter is not Ractor-friendly. I’ve opened the pull request to fix this, so once that’s merged, the basic Ruby logger formatter will work just fine. For the time being, we will monkey patch it:

class Logger::Formatter
  def call(severity, time, progname, msg)
    Format % [
      severity[0..0],
      format_datetime(time),
      Process.pid,
      severity,
      progname,
      msg2str(msg)
    ]
  end
end

With this, we can run our logging from any ractor we want:

require 'logger'

class Logger::Formatter
  def call(severity, time, progname, msg)
    Format % [
      severity[0..0],
      format_datetime(time),
      Process.pid,
      severity,
      progname,
      msg2str(msg)
    ]
  end
end

class Rogger < Ractor
  def self.new
    super do
      logger = ::Logger.new($stdout)

      while data = recv
        logger.public_send(data[0], *data[1])
      end
    end
  end

  def method_missing(m, *args, &_block)
    self << [m, *args]
  end
end

class Rails
  LOGGER = Rogger.new

  def self.logger
    LOGGER
  end
end

Ractor.new do
  Rails.logger.info "Hello"
end

sleep(1)
ruby test.rb

I, [2020-09-28T18:23:56.181512 #11519]  INFO -- : Hello

Summary

Providing the Ractor support in the things like Rails won’t be easy. There are many challenges to tackle, but at the same time, I see it as an excellent opportunity to leverage new Ruby capabilities. It’s also a great chance to get away from anti-patterns that are in Ruby and Rails for as long as I can remember. There’s a whole new world of engineering that will be much easier to achieve thanks to Ractors.

This year, I want to also explore the possibility of running homogenous Docker containers with Ruby VM in which I could load balance services running in particular guilds. Theoretically, this could allow for sub-second mitigation of sudden traffic spikes without having many overprovisioned instances.


Cover photo by David Stanley on Attribution 2.0 Generic (CC BY 2.0) license.

Olderposts

Copyright © 2020 Running with Ruby

Theme by Anders NorenUp ↑