Tag: Sorting

Mongoid and Aggregation Framework: Get similar elements based on tags, ordered by total number of matches (similarity level)

February 25, 2014 / / 1 Comment

So, lets say we want have an Article model with tags array:

class Article
  include Mongoid::Document
  include Mongoid::Timestamps

  field :content, type: String, default: ''
  field :tags, type: Array, default: []
end

Let's try to pick any similar first (without similarity level)

We have an article, that has some tags (%w{ ruby rails mongoid mongodb }) and we would like to get similar articles. Nothing special (yet):

current_article = Article.first
similar = Article.in tags: current_article.tags

Let's also pick elements without our base article (current_article) though we decided to get similar articles, not similar or equal:

Article
  .ne(_id: current_article.id)
  .in(tags: current_article.tags)

We could even refactor it a bit...

class Article
  include Mongoid::Document
  include Mongoid::Timestamps

  scope :exclude, -> article { ne(_id: article.id) }
  scope :similar_to, -> article { exclude(article).in(tags: article.tags ) }

  field :content, type: String, default: ''
  field :tags, type: Array, default: []

  def similar
    @similar ||= self.class.similar_to self
  end
end

# Example usage:
current_article.similar #=> [Article, Article]

Seems pretty decent, but this won't give us most similar articles. It will just return most recent, that have equal at least one tag with our current_article. What should we do then?

Mongo Aggregation Framework to the rescue

To get such information, sorted in a proper way, we need to perform following steps:

  1. Don't include current_article in resultset
  2. Get all articles (except current one), that have at least one tag as current_article (we did this earlier)
  3. Count how many similar tags occurred in each of articles
  4. Sort articles by similarity
  5. Take first 10 articles

Step 1 - Excluding

# Mongoid
Article.where(id: {"$ne" => current_article.id})
# Mongo (this is still in Ruby - not in Mongo shell!)
"$match" => { 
  _id: { "$ne" => current_article.id }
}

Step 2 - All articles with at least one similar tag

# Mongoid
Article.in(tags: current_article.tags )
# Mongo (this is still in Ruby - not in Mongo shell!)
"$match" => { 
  tags: { "$in" => %w{ ruby rails mongoid mongodb } }
}

Step 3 - Unwind by tags

If you're not familiar with unwind look here. That way, we get article copy for every tag for each article.

{ "$unwind" => "$tags" }

Step 4 - Second matching

You may wonder, why we filter results again. Well The initial filtering was not required, but we did this to remove all non-related articles, so the data set is much smaller. Unfortunately unwind created document copy per each of the tags - even those that we don't want to. That's why we have to filter it again.

"$match" => { 
  tags: { "$in" => %w{ ruby rails mongoid mongodb } }
}

Note that we don't need to filter out again by ID, since in incoming dataset we already don't have the current_article document instance.

Step 5 - Grouping

Now we can group by documents ID. Also we will add sum for grouping, so we will know similarity level for each document. One point in sum equals one similar matching tag.

{ "$group" => {
    _id: "$_id", 
    matches: {"$sum" =>1}
  }
}

Step 6 - Sorting

Now we can sort by sum to have elements in descending order (most similar on top):

{ "$sort" => {matches:-1} }

Step 7 - 10 first elements

And the last step - limiting:

{ "$limit" => 10 }

Making it all work together

In order to execute this whole code in Ruby, we need to use Article.collection.aggregate method:

results = Article.collection.aggregate(
  {
    "$match" => { 
      tags: { 
        "$in" => current_article.tags 
      },
      _id: { 
        "$ne" => current_article.id 
      }
    },
  },  
  { 
    "$unwind" => "$tags" 
  },
  { 
    "$match" => { 
      tags: { 
        "$in" => current_article.tags 
      } 
    }
  },
  { 
    "$group" => {
      _id: "$_id", 
      matches: { "$sum" =>1 }
    }
  },
  { 
    "$sort" => { matches: -1 }  
  },
  { 
    "$limit" => 10 
  }
)

We won't get Ruby objects as a result (we'll get an array of hashes). We can process it further if we need similarity level, but if we just need similar articles (for example to display them) we can just:

Article.find results.map(&:first).map(&:last)

How to use Ransack helpers with MongoDB and Mongoid on Rails 4.0

September 30, 2013 / / 0 Comments

Ransack is a rewrite of MetaSearch. It allows you to create search forms and sorting for your ActiveRecord data. Somehow I really miss that, when working with Mongoid. To be honest I can even live without the whole Ransack search engine, but I liked the UI helpers for search forms and sorting data.

I've really wanted to be able to do something like that:

@users = current_app.users.order(params[:q]).page(current_page)

and 

%tr.condensed{:role => "row"}
  %th= sort_link @search, :guid, 'User id'
  %th= sort_link @search, :email, 'Email address'
  %th= sort_link @search, :current_country, 'Country'
  %th= sort_link @search, :current_city, 'City'

with Mongoid documents. It turns out, that if you don't need really sophisticated stuff, you can use Ransack helpers even with Mongoid. In order to use search/sort UI features, you need @search Ransack model instance. It is based on ActiveRecord data and unfortunately we don't have any models that would suit our needs. That's why we need to create a dummy one:

module System
  # In order to use Ransack helpers and engine for non AR models, we need to 
  # initialize @search with AR instance. To prevent searching (this is just a stub)
  # we do it usign ActiveRecord dummy class that is just a blank class
  class ActiveRecord < ActiveRecord::Base

    def self.columns
      @columns ||= []
    end

  end
end

This is enough to use it in controllers and views:

# In order to use Ransack helpers and engine for non AR models, we need to 
# initialize @search with AR instance. To prevent searching (this is just a stub)
# we do it with none
def search
  @search ||= ::System::ActiveRecord.none.search(params[:q])
end

It is a dummy ActiveRecord class, that will always return empty scope (none), but it is more than enough to allow us to work with UI helpers. To handle ordering of Mongoid document collection, we could use following code:

module System
  module Mongoid
      # Order engine used to order mongoid data
      # It is encapsulated in segment module, because we store here all the
      # things that are related to filtering/ordering data
      #
      # Since it returns the resource/scope that was provided, modified
      # accordingly to sorting rules, this class can be used in defined
      # scopes and it will work with scope chainings
    class Orderable
      # @param resource [Mongoid::Criteria] Mongoid class or a subset 
      #   of it (based on Mongoid::Criteria)
      # @param rules [Hash] hash with rules for ordering
      # @return [Mongoid::Criteria] mongoid criteria scope
      # @example
      #   System::Mongoid::Orderable.new(current_app.users, params[:q])
      def initialize(resource, rules = {})
        @rules = rules
        @resource = resource
      end

      # Applies given sort order if there is one.
      # If not, it will do nothing
      # @example Ordered example scope
      #   scope :order, -> order { System::Mongoid::Orderable.new(self, order).apply }
      def apply
        order unless @rules.blank?
        @resource.criteria
      end

      private

      # Sets given order based on order rules (if there are any)
      # or it will do nothing if no valid order rules provided
      def order
        order = "#{@rules[:s]}".split(' ')
        @resource = order.blank? ? @resource : @resource.order_by(order)
      end
    end
  end
end

This can be used to create scopes like that:

class User
  include Mongoid::Document
  include Mongoid::Attributes::Dynamic

  # Scope used to set all the params from ransack order engine
  # @param [Hash, nil] hash with order options or nil if no options
  # @return [Mongoid::Criteria] scoping chain
  # @example
  #   User.order(params[:q]).to_a #=> order results
  scope :order, -> order { System::Mongoid::Orderable.new(self, order).apply }
end

Copyright © 2024 Closer to Code

Theme by Anders Noren — Up ↑