mongoid_traffic

Mongoid Traffic

Aggregated traffic logs stored in MongoDB. Fast and efficient logging via atomic updates of nested hashes in small number of MongoDB documents, semi-fast retrieveal and aggregation.

Installation

Add this line to your application's Gemfile:

gem 'mongoid_traffic'

And then execute:

$ bundle

Or install it yourself as:

$ gem install mongoid_traffic

Usage

Setup your class for storing the log:

class MyLog
  include Mongoid::Document
  include MongoidTraffic::Log
end

Log your traffic like this:

MongoidTraffic::Logger.log(MyLog)

Or, if you prefer, directly on the MyLog class:

MyLog.log

This will (by default) create four MyLog documents: each with :df(date_from) and :dt(date_to) fields specifying yearly, monthly, weekly and daily log. Each log has an :access_count attribute incremented with subsequent .log calls.

Optional arguments

Time scope:

By default, the .log method creates/updates a document with aggregations for year, month, week and day. Subset of those can be specified as:

MyLog.log(time_scope: %i[day])

The available options are: %i[year month week day]

Scope:

It is possible to scope the log by an arbitrary number of parameters.

class MyLog
  include Mongoid::Document
  include MongoidTraffic::Log

  belongs_to :page

  scope :for_page, -> (page) { where(page: page) }
end

For example:

MongoidTraffic::Logger.log(MyLog.for_page(page))

or

MyLog.for_page(page).log(page: my_page)

PS If you query by a scope often do not forget to add corresponding indexes to your log model.

Arbitrary counter:

It is possible to count any number of arbitrary additional values. For example, to count unique country etc.

First, specify the additional counter in the log model (prefer short aliased field names):

class MyLog
  include Mongoid::Document
  include MongoidTraffic::Log

  additional_counter :c, as: :country
end

Track access by country as:

MyLog.log(country: 'CZ')
MyLog.log(country: 'NL')

Which will create Hash with counts per country:

{ _id: …, df(date_from): …, dt(date_to): …, ac(access_count): 2, c(country): { 'CZ' => 1, 'NL' => 1 } }

You can take advantage of the fact, that the underlying queries support dot-notation, and track on deeply nested hashes. For example, should you want to track access per browser:

class MyLog
  include Mongoid::Document
  include MongoidTraffic::Log

  additional_counter :b, as: :browser
end

Then track access by browser as:

MyLog.log(browser: "Mac.Safari.8") # log access by Mac Safari 8
MyLog.log(browser: "Mac.Safari.7%2E1") # log access by Mac Safari 7.1

Which will create following log document:

{ _id: …, df(date_from): …, dt(date_to): …, ac(access_count): 2, b(browser): { 'Mac' => { 'Safari' => { '8' => 1, '7%2E1' => 1 } } } }

Please note all . not intended to denote nesting need to be escaped (here as %2E).

Accessing the log

The log can be accessed using a combination of Mongoid Criteria and aggregation methods.

Criteria

The following time based criteria are predefined as Mongoid scopes:

• .day(date)
• .week(week, year)
• .month(month, year)
• .year(year)
• .for_dates(date_from, date_to)

To select by log type:

• .daily
• .weekly
• .monthly
• .yearly

To narrow down by scope:

• .scoped_to(scope)

Aggregation method

• .aggregate_on(:access_count)
• .aggregate_on(ARBITRARY_COUNTER)

Behind the scenes, this method will take all documents returned by your criteria and combines the values of the specified field (in case of :access_count it is simple sum of the values, in other cases it is sum of nested hashes).

Examples

Time

MyLog.day(Date.today)

Eventually by date range (when using the .for_dates scope make sure to specify which log type you wish to access):

MyLog.daily.for_dates(Date.yesterday, Date.today)

Scope

MyLog.for_page(page).day(Date.today)

Make sure that the order of drilling down corresponds to the indexes on your model.

Aggregations

On access count:

MyLog.day(Date.today).scoped_to('/pages/123').aggregate_on(:access_count) # => 1

The scope query accepts regular expressions, which allows for aggregations on specific parts of your site. For example should you want to query for all pages that have path beginning with '/blog':

MyLog.month(8, 2014).scoped_to(/\A\/blog/).aggregate_on(:access_count) # => 1

On additional counter:

MyLog.day(Date.today).scoped_to('/pages/123').aggregate_on(:country) # => { 'CZ' => 1, 'NL' => 1 }

Pluck

For plotting charts you might make use of standard :pluck:

MyLog.daily.for_dates(Date.today - 1.week, Date.today).pluck(:date_from, :access_count) # => returns array of dates and counts per day

Further reading

Based on the approach described by John Nunemaker here and here.

Contributing

• Fork it ( https://github.com/tomasc/mongoid_traffic/fork )
• Create your feature branch (git checkout -b my-new-feature)
• Commit your changes (git commit -am 'Add some feature')
• Push to the branch (git push origin my-new-feature)
• Create a new Pull Request