Tag: cache

Ruby global method cache invalidation impact on a single and multithreaded applications

Most of the programmers treat threads (even green threads in MRI) as a separate space, that they can use to simultaneously run some pieces of code. They also tend to assume, that there's no "indirect" relation between threads unless it is explicitly declared and/or wanted. Unfortunately it's not so simple as it seems...

But before we start talking about threads, you need to know what is (and a bit about how it works) Ruby method cache.

What is a Ruby global method cache?

There's a great explanation on what it is in "Ruby under a Microscope" book (although a bit outdated):

Depending on the number of superclasses in the chain, method lookup can be time consuming. To alleviate this, Ruby caches the result of a lookup for later use. It records which class or module implemented the method that your code called in two caches: a global method cache and an inline method cache.

Ruby uses the global method cache to save a mapping between the receiver and implementer classes.

The global method cache allows Ruby to skip the method lookup process the next time your code calls a method listed in the first column of the global cache. After your code has called Fixnum#times once, Ruby knows that it can execute the Integer#times method, regardless of from where in your program you call times .

Now the update: 2.1+ there's still a method cache and it is still global but based on the class hierarchy, invalidating the cache for only the class in question and any subclasses. So my case is valid in Ruby 2.1+.

Why would we care?

Because careless and often invalidation of this cache can have a quite big impact on our Ruby software. Especially when you have a multithreaded applications, since there's a single global method cache per process. And constant invalidating, will require a method lookup again and again...

60958575

What invalidates a Ruby global method cache?

There's quite a lot things that invalidate it. Quite comprehensive list is available here, but as a summary:

  • Defining new methods
  • Removing methods
  • Aliasing methods
  • Setting constants
  • Removing constants
  • Changing the visibility of constants
  • Defining classes
  • Defining modules
  • Module including
  • Module prepending
  • Module extending
  • Using a refinements
  • Garbage collecting a class or module
  • Marshal loading an extended object
  • Autoload
  • Non-blocking methods exceptions (some)
  • OpenStructs instantiations

How can I check if what I do invalidates global method cache?

You can use RubyVM.stat method. Note that this will show only chanages that add/remove something. Changing visibility will invalidate method cache, but won't be visible using this ,ethod.

RubyVM.stat #=> {:global_method_state=>134, :global_constant_state=>1085, :class_serial=>6514}
class Dummy; end #=> nil

# Note that global_constant_state changed
RubyVM.stat #=> {:global_method_state=>134, :global_constant_state=>1086, :class_serial=>6516}

def m; end #=> :m
RubyVM.stat #=> {:global_method_state=>135, :global_constant_state=>1086, :class_serial=>6516}

# Note that changing visibility invalidates cache, but you won't see that here
private :m #=> Object
RubyVM.stat #=> {:global_method_state=>135, :global_constant_state=>1086, :class_serial=>6516}
public :m #=> Object
RubyVM.stat #=> {:global_method_state=>135, :global_constant_state=>1086, :class_serial=>6516}

Benchmarking methodology

General methodology that I've decided to took is similar for single and multi threaded apps. I just create many instances of an element and execute a given message. This way MRI will have do a method lookup after each invalidation.

class Dummy
  def m1
    rand.to_s
  end

  def m2
    rand.to_s
  end
end

threads = []

benchmark_task = -> { 100000.times { rand(2) == 0 ? Dummy.new.m1 : Dummy.new.m2 } }

I invalidate method cache by creating a new OpenStruct object:


OpenStruct.new(m: 1)

Of course this would not work for multithreaded invalidation, since first invocation (in any thread) would place info in method cache. That's why I've decided for multithreading, to declare a lot of methods on our Dummy class:

METHODS_COUNT = 100000

class Dummy
  METHODS_COUNT.times do |i|
    define_method :"m#{i}" do
      rand.to_s
    end
  end
end

It's worth noting that in a single threaded environment, a huge number of new OpenStruct objects initializations can have an impact on a final result. That's why initializations were benchmarked as well. This allowed me to get the final performance difference.

Method cache invalidation for single threaded applications

Except some CPU fluctuations (I've been running it on one of my servers), the difference can be clearly seen:

single_perf
Here you can see the performance difference. Except some spikes, the difference is between 8 and 16% and the average is 12.97%.

perf_difStill, we need to remember, that this included 1 invalidation and few commands, so it doesn't exactly simulate the real app flow, where you would not have a ratio near to o 1:1 between commands and invalidations. But even then, with a pretty heavy loaded, single thread app, that uses a lot of OpenStructs or any other cache invalidating things, it might have a huge impact on its performance.

Method cache invalidation for multithreaded applications

Benchmarking method cache invalidation in multithreaded environment can be kinda tricky. I've decided to have 2 types of threads and a switch:

  • Benchmarked threads - threads that I would be monitoring in terms of their performance
  • Invalidating threads - threads that would invalidate ruby method cache all over again
  • Switch - allows to turn on/off cache invalidation after given number of iterations

Every invalidating thread looks like this:

threads << Thread.new do
  while true do
    # Either way create something, to neutralize GC impact
    @invalidate ? OpenStruct.new(m: 1) : Dummy.new
  end
end

Also I've decided to check performance for following thread combinations:

  • Benchmarked threads: 1 / Invalidating threads: 1
  • Benchmarked threads: 1 / Invalidating threads: 10
  • Benchmarked threads: 10 / Invalidating threads: 1
  • Benchmarked threads: 10 / Invalidating threads: 10

and here are the results.

Benchmarked threads: 1 / Invalidating threads: 1

th11

Benchmarked threads: 1 / Invalidating threads: 10

t110

Benchmarked threads: 10 / Invalidating threads: 1

t101-1 t101-2

Benchmarked threads: 10 / Invalidating threads: 10t1010-1 t1010-2

Conclusions

Even with 2 threads and when 1 of them is heavily invalidating method cache, performance drop can be really high (around 30% or even more).

The overall performance drop for multiple threads that work and multiple that invalidate is around 23,8%. Which means that if you have many threads (lets say you use Puma), and you work heavily with OpenStruct (or invalidate method cache in a different way) in all of them, you could get up to almost 24% more switching to something else. Of course probably you will gain less, because every library has some impact on the performance, but in general, the less you invalidate method cache, the faster you go.

The quite interesting was the combination 10:1. There's almost no performance impact there (around 4.3%). Why? Because after the first invocation of a given method, it's already cached so there won't be a cache miss when using method cache.

TL;DR: For multithreaded software written in Ruby, it's worth paying attention to what and when invalidates your method cache. You can get up to 24% better performance (but probably you'll get around 10% ;) ).

Ruby on Rails and Dragonfly processed elements tracking with two layer cache – Memcached na ActiveRecord underneath

Dragonfly is a great on-the-fly processing gem, especially for apps that rapidly change and need files processed by multiple processors (for example, when you need to generate many thumbnails for the same attached image).

Since it is a on-the-fly processor, it has some downsides, and the biggest one, in my opinion, is keeping track of which thumbnails we have already created. There's a nice example of how to do it using only ActiveRecord in the dragonfly docs, but unfortunately this solution won't be enough for a heavy imaged website. Since each request for a thumb will require a single SQL query, you might end up having 50-60 and even more thumb related queries per page.

Of course, they would probably get cached, so they would not take much time, but still, there's a better way to do it.

Memcached (dalli) to the rescue

Memcached uses LRU algorithm to remove old and unused data, so no need to worry about it exceeding memory limits. Instead, we can focus on putting there our thumbnails details, and the more often we request them, the higher change there will be, that we will get a cache hit. We need to remember, then Memcached is an in memory data storage, so our data might dissapear at any point. That's why it is still worth adding second ActiveRecord layer.

Configuring Rails to use Memcached as a cache storage

Before we go any further, we need to tell our app, that we want to store cached data in Memcached. To do so, please follow Dalli docs instructions. Dalli is the best Memcached ruby client there is and it can be easily integrated with Ruby on Rails framework.

Creating ActiveRecord dragonfly cache storage

Dragonfly docs example names the ActiveRecord cache model Thumb. We will name it DragonflyCache, since we might use it to store info not only about images. Here's an sample migration for this resource:

class CreateDragonFlyCache < ActiveRecord::Migration
  def change
    create_table :dragonfly_caches do |t|
      t.string :uid
      t.string :job
      t.timestamps
    end

    add_index :dragonfly_caches, :uid
    add_index :dragonfly_caches, :job
  end
end

Adding Memcached layer to a DragonflyCache model

We now have a single layer ActiveRecord cache storage for our app. We will use two Rails cache methods to add a second layer:

Rails.cache.read(key)
Rails.cache.write(key, value)

Cache read (hit)

The whole flow for cache read will be pretty simple:

  1. Check if job signature is in Memcached and if so - return it (do nothing else)
  2. If not, check if it can be found in SQL database and if so, store it in Memcached and return it
  3. If not found, return nil
# @return [::DragonflyCache] df cache instance that has a uid that we want
#   to get and use
# @param [String] job_signature that acts as a unique identifier for uid
def read(job_signature)
  stored = Rails.cache.read(key(job_signature))

  return new(uid: stored) if stored

  stored = find_by(job: job_signature)
  Rails.cache.write(key(job_signature), stored.uid) if stored

  stored
end

You might notice a method called key. When working with Memcached, I like to prefix keys, so there won't be any cache collisions (scenario when different data is there).

This method is pretty simple:

# @return [String] key made from job signature and a prefix
# @param [String] job_signature for which we want to get a key
def key(job_signature)
  "#{PREFIX}_#{job_signature}"
end

Cache write (store)

Cache write will be easier. We just have to store job details in both SQL database and Memcached:

# Stores a given job signature with uid in a DB and memcache it
# It is used as a fallback persistent storage, when memcached key
# is not found
# @param [String] job_signature that acts as a unique identifier
# @param [String] uid that is equal to our file path under which
#   we have this certain thumb/file
# @raise [ActiveRecord::RecordInvalid] raised when something goes
#   really wrong (should not happen)
def write(job_signature, uid)
  Rails.cache.write(key(job_signature), uid)

  create!(
    uid: uid,
    job: job_signature
  )
end

Now, we can incorporate the code above into DragonflyCache model.

Full DragonflyCache model

# DragonflyCache stores informations about processed dragonfly files that we have
# @see http://markevans.github.io/dragonfly/cache/
class DragonflyCache < ActiveRecord::Base
  # Prefix for all memcached dragonfly job signatures
  PREFIX = 'dragonfly'

  class << self
    # @return [::DragonflyCache] df cache instance that has a uid that we want
    #   to get and use
    # @param [String] job_signature that acts as a unique identifier for uid
    def read(job_signature)
      stored = Rails.cache.read(key(job_signature))

      return new(uid: stored) if stored

      stored = find_by(job: job_signature)
      Rails.cache.write(key(job_signature), stored.uid) if stored

      stored
    end

    # Stores a given job signature with uid in a DB and memcached
    # It is used as a fallback persistent storage, when memcached key
    # is not found
    # @param [String] job_signature that acts as a unique identifier
    # @param [String] uid that is equal to our file path under which
    #   we have this certain thumb/file
    # @raise [ActiveRecord::RecordInvalid] raised when something goes
    #   really wrong (should not happen)
    def write(job_signature, uid)
      Rails.cache.write(key(job_signature), uid)

      create!(
        uid: uid,
        job: job_signature
      )
    end

    private

    # @return [String] key made from job signature and a prefix
    # @param [String] job_signature for which we want to get a key
    def key(job_signature)
      "#{PREFIX}_#{job_signature}"
    end
  end
end

Connecting our DragonflyCache model with Dragonfly

This part is really easy. We have to add the following config options to our Dragonfly initializer:

Dragonfly.app.configure do
  define_url do |app, job, opts|
    cached = DragonflyCache.read(job.signature)

    if cached
      app.datastore.url_for(cached.uid)
    else
      app.server.url_for(job)
    end
  end

  before_serve do |job, env|
    DragonflyCache.write(job.signature, job.store)
  end
end

That's all. Now your Dragonfly cache should hit DB less often.

Copyright © 2024 Closer to Code

Theme by Anders NorenUp ↑