Running with Ruby

Tag: Mongoid (page 1 of 6)

Mongoid (MongoDB) has_many/belongs_to relation and wrong index being picked

Sometimes when you have a belongs_to relations, you may notice, that getting an owner object can be really slow. Even if you add index like this:

class Activity
  include Mongoid::Document
  include Mongoid::Timestamps
  include Mongoid::Attributes::Dynamic
 
  belongs_to :person

  index({ person_id: 1 }, background: true)
end

Mongoid might not catch it. Unfortunately you can’t just explain it, since this relation is returning an object not a Mongoid::Criteria. Luckily you can just hook up to your slow-log and grep for it (or you can just benchmark it). Either way, if you notice that it’s not using index as it should, first declare it the way it should be declared:

class Activity
  include Mongoid::Document
  include Mongoid::Timestamps
  include Mongoid::Attributes::Dynamic
 
  belongs_to :person, index: true
end

If you already had an index on a person_id, you don’t need to recreate it. It should use it out of the box. If it doesn’t help, you can always use explain to overwrite the default finder method for this relation:

class Activity
  include Mongoid::Document
  include Mongoid::Timestamps
  include Mongoid::Attributes::Dynamic
 
  belongs_to :person, index: true

  def person
    # Here in explain you need to provide index name
    @person ||= Person.hint('person_id_1').find_by(person_id: person_id)
  end
end

Note: this post might be somehow related to this issue.

Mongoid and Aggregation Framework: Get similar elements based on tags, ordered by total number of matches (similarity level)

So, lets say we want have an Article model with tags array:

class Article
  include Mongoid::Document
  include Mongoid::Timestamps

  field :content, type: String, default: ''
  field :tags, type: Array, default: []
end

Let’s try to pick any similar first (without similarity level)

We have an article, that has some tags (%w{ ruby rails mongoid mongodb }) and we would like to get similar articles. Nothing special (yet):

current_article = Article.first
similar = Article.in tags: current_article.tags

Let’s also pick elements without our base article (current_article) though we decided to get similar articles, not similar or equal:

Article
  .ne(_id: current_article.id)
  .in(tags: current_article.tags)

We could even refactor it a bit…

class Article
  include Mongoid::Document
  include Mongoid::Timestamps

  scope :exclude, -> article { ne(_id: article.id) }
  scope :similar_to, -> article { exclude(article).in(tags: article.tags ) }

  field :content, type: String, default: ''
  field :tags, type: Array, default: []

  def similar
    @similar ||= self.class.similar_to self
  end
end

# Example usage:
current_article.similar #=> [Article, Article]

Seems pretty decent, but this won’t give us most similar articles. It will just return most recent, that have equal at least one tag with our current_article. What should we do then?

Mongo Aggregation Framework to the rescue

To get such information, sorted in a proper way, we need to perform following steps:

  1. Don’t include current_article in resultset
  2. Get all articles (except current one), that have at least one tag as current_article (we did this earlier)
  3. Count how many similar tags occurred in each of articles
  4. Sort articles by similarity
  5. Take first 10 articles

Step 1 – Excluding

# Mongoid
Article.where(id: {"$ne" => current_article.id})
# Mongo (this is still in Ruby - not in Mongo shell!)
"$match" => { 
  _id: { "$ne" => current_article.id }
}

Step 2 – All articles with at least one similar tag

# Mongoid
Article.in(tags: current_article.tags )
# Mongo (this is still in Ruby - not in Mongo shell!)
"$match" => { 
  tags: { "$in" => %w{ ruby rails mongoid mongodb } }
}

Step 3 – Unwind by tags

If you’re not familiar with unwind look here. That way, we get article copy for every tag for each article.

{ "$unwind" => "$tags" }

Step 4 – Second matching

You may wonder, why we filter results again. Well The initial filtering was not required, but we did this to remove all non-related articles, so the data set is much smaller. Unfortunately unwind created document copy per each of the tags – even those that we don’t want to. That’s why we have to filter it again.

"$match" => { 
  tags: { "$in" => %w{ ruby rails mongoid mongodb } }
}

Note that we don’t need to filter out again by ID, since in incoming dataset we already don’t have the current_article document instance.

Step 5 – Grouping

Now we can group by documents ID. Also we will add sum for grouping, so we will know similarity level for each document. One point in sum equals one similar matching tag.

{ "$group" => {
    _id: "$_id", 
    matches: {"$sum" =>1}
  }
}

Step 6 – Sorting

Now we can sort by sum to have elements in descending order (most similar on top):

{ "$sort" => {matches:-1} }

Step 7 – 10 first elements

And the last step – limiting:

{ "$limit" => 10 }

Making it all work together

In order to execute this whole code in Ruby, we need to use Article.collection.aggregate method:

results = Article.collection.aggregate(
  {
    "$match" => { 
      tags: { 
        "$in" => current_article.tags 
      },
      _id: { 
        "$ne" => current_article.id 
      }
    },
  },  
  { 
    "$unwind" => "$tags" 
  },
  { 
    "$match" => { 
      tags: { 
        "$in" => current_article.tags 
      } 
    }
  },
  { 
    "$group" => {
      _id: "$_id", 
      matches: { "$sum" =>1 }
    }
  },
  { 
    "$sort" => { matches: -1 }  
  },
  { 
    "$limit" => 10 
  }
)

We won’t get Ruby objects as a result (we’ll get an array of hashes). We can process it further if we need similarity level, but if we just need similar articles (for example to display them) we can just:

Article.find results.map(&:first).map(&:last)
Olderposts

Copyright © 2017 Running with Ruby

Theme by Anders NorenUp ↑