A gem for caching with ReadySet within Rails applications.
:star: If you find this gem useful, please consider giving us a star on GitHub! Your support helps us continue to innovate and deliver exciting new features.
ReadySet is a database acceleration engine that acts as a read replica and implements a novel type of query caching that automatically keeps your caches up-to-date by watching your database's replication stream. ReadySet helps you scale your database by shedding load and reducing query latency.
This gem makes it easy to use ReadySet from within your Rails application by allowing you to selectively route queries to ReadySet. The high-level features of this gem include:
Readyset.route method that takes a block and routes to ReadySet any
queries within that blockNote that ReadySet only guarantees support for PostgreSQL right now, so this gem only supports PostgreSQL.
If you run into any trouble with the below steps, please feel free to reach out via our community Slack!
bundle install:
gem 'readyset'
config/database.yml file with ReadySet's connection
information. If you currently connect to only one database, you'll need to
move your primary database connection information to be nested under a new
key named primary.
development:
primary:
# This is the connection information for your primary database
database: testdb
username: postgres
password: readyset
adapter: postgresql
port: 5432
readyset:
# This is the connection information for ReadySet
database: testdb
username: postgres
password: readyset
adapter: readyset
host: "127.0.0.1"
port: 5433
# This setting tells Rails that there's no need to run migrations or other database
# administration tasks against ReadySet
database_tasks: false
rails readyset:status:
$ rails readyset:status
+----------------------------+------------------------+
| Database Connection | Connected |
| Connection Count | 1 |
| Snapshot Status | Completed |
| Maximum Replication Offset | (0/6DBBD78, 0/6DBBFF0) |
| Minimum Replication Offset | (0/6DBBD78, 0/6DBBFF0) |
| Last started Controller | 2024-01-17 18:49:02 |
| Last completed snapshot | 2024-01-19 15:18:02 |
| Last started replication | 2024-01-19 15:18:02 |
+----------------------------+------------------------+
rails readyset:tables:
$ rails readyset:tables
+---------------------------------+-------------+-------------+
| table | status | description |
+---------------------------------+-------------+-------------+
| "public"."posts" | Snapshotted | |
+---------------------------------+-------------+-------------+
Readyset.configure wherever you configure other gems in your
application, and set any desired configuration options:
Readyset.configure do |config|
# Set your config options here
end
config.active_record.query_log_tags_enabled = true wherever you configure
your ActiveRecord settings. This will append information to your
ActiveRecord query logs that tells you where the given query was routed (e.g.
to ReadySet or to your primary database)Follow the instructions above to set up ReadySet and install the gem
Route a query to ReadySet in your application like so:
Readyset.route do
Post.where(user_id: user_id)
end
Start up your application and drive traffic through the part of your application that invokes the query you routed in the previous step
If you are running Rails 7 and enabled query log tags as explained in step 5
above, you should see an annotation next to the query in your
application logs that denotes where the query was routed. If the
destination tag has the value readyset, the query was routed to
ReadySet.
If you are running Rails 6, you can validate that the query was routed to
ReadySet by running rails readyset:proxied_queries. A "proxied" query is
one that was served by ReadySet but was proxied to your primary database,
since a cache for the query does not yet exist:
$ rails readyset:proxied_queries
+--------------------+-------------------------------------------------------+-------------+-------+
| id | text | supported | count |
+--------------------+-------------------------------------------------------+-------------+-------+
| q_281c5f9b8e4013bb | SELECT | yes | 1 |
| | * | | |
| | FROM | | |
| | "posts" | | |
| | WHERE | | |
| | ("user_id" = $1) | | |
+--------------------+-------------------------------------------------------+-------------+-------+
Create a cache for the query by running
rails readyset:proxied_queries:cache_all_supported. This will create caches for
all of the queries proxied by ReadySet that are supported to be cached. You
can verify that the expected caches were created by running
rails readyset:caches:
$ rails readyset:caches
+--------------------+--------------------+-------------------------------------+--------+-------+
| id | name | text | always | count |
+--------------------+--------------------+-------------------------------------+--------+-------+
| q_281c5f9b8e4013bb | q_281c5f9b8e4013bb | SELECT | false | 0 |
| | | "public"."posts"."user_id" | | |
| | | FROM | | |
| | | "public"."posts" | | |
| | | WHERE | | |
| | | ("public"."posts"."user_id" = $1) | | |
+--------------------+--------------------+-------------------------------------+--------+-------+
Drive traffic through the part of your application that invokes your cached
query. The first invocation of the query will be a cache miss, but the
second will be served from the cache. You can verify that the cache was
successfully used by looking at the count column in the output of
rails readyset:caches:
$ rails readyset:caches
+--------------------+--------------------+-------------------------------------+--------+-------+
| id | name | text | always | count |
+--------------------+--------------------+-------------------------------------+--------+-------+
| q_281c5f9b8e4013bb | q_281c5f9b8e4013bb | SELECT | false | 1 |
| | | "public"."posts"."user_id" | | |
| | | FROM | | |
| | | "public"."posts" | | |
| | | WHERE | | |
| | | ("public"."posts"."user_id" = $1) | | |
+--------------------+--------------------+-------------------------------------+--------+-------+
Queries in arbitrary code blocks can be routed to ReadySet using the
Readyset.route method like so:
Readyset.route do
Post.where(user_id: user_id)
end
Any queries invoked in the given block will be routed to the ReadySet instance
configured in your config/database.yml file; however, until a cache is created
for a particular query, invocations of that query against ReadySet will be proxied
to your database. To create a cache for a specific query, you have a few options:
.create_readyset_cache directly on an ActiveRecord query in the
Rails console:
Post.where(user_id: 1).create_readyset_cache!
rails readyset:proxied_queries:cache_all_supported
rails readyset:proxied_queries:drop_all.Readyset::Query::ProxiedQuery.all
#cache! on the queries in this list for which you'd like to
create caches on ReadySet.rails readyset:proxied_queries:supported
rails readyset:create_cache command like so:
rails 'readyset:create_cache[your_query_id]'
Once a cache has been created for a particular query, it will persist on ReadySet across restarts (although any in-memory cached data will be lost when ReadySet goes down). You can view the list of existing caches using the provided Rake task:
rails readyset:caches
To drop a given cache in the list printed by the above command, you can pass the
name of the cache to the readyset:caches:drop Rake task like so:
rails 'readyset:caches:drop[my_cache]'
You can also view the list of existing caches in an interactive form via the Rails console:
Readyset::Query::CachedQuery.all
You can invoke #drop! on any of the caches in this list to remove the cache
from ReadySet.
The gem includes an extension to ActionController that allows you to route to
ReadySet all of the queries that occur within the context of a given controller
action:
class PostsController < ActionController
route_to_readyset only: :show
def show
@post = Post.where(id: params[:id])
end
end
route_to_readyset takes the same parameters as Rails's
around_filters.
The gem also includes an extension that allows you to define queries in your model that will be routed to ReadySet:
class Post < ApplicationRecord
readyset_query :posts_for_user, ->(user_id) { where(user_id: user_id) }
end
The above example will define a .posts_for_user class method on the Post
model that invokes the query Post.where(user_id: user_id) against ReadySet.
Note that other invocations of this query outside of the context of the
.posts_for_user method will not be routed to ReadySet.
This feature allows you to specify which queries should be routed to ReadySet
in a centralized location and prevents the need to use Readyset.route
everywhere a cached query is invoked.
Once you have a set of caches you are happy with in your development environment, you'll need a way to easily reproduce the same set of caches in other environments (e.g. staging, production, etc.). To facilitate this, the gem includes a "migration" feature, that allows you to dump the current set of caches to a "migration file" and re-create these caches using the same migration file.
The following Rake task dumps the current set of caches to the
db/readyset_caches.rb file:
rails readyset:caches:dump
This file should be checked into version control with your application code. To update a ReadySet instance so that its set of caches matches the caches in your migration file:
rails readyset:caches:migrate
This command 1) drops any caches that exist on ReadySet that are not present in
the migration file and 2) creates any caches that are present in the migration
file that do not exist on ReadySet. To run the command for a particular Rails
environment, you can set the RAILS_ENV environment variable.
To handle situations where ReadySet is unreachable for any reason, the gem includes an automatic failover feature. The gem tracks the number of ReadySet connection failures over a configurable window of time, and if the number of errors exceeds the configured threshold, any queries previously being routed to ReadySet will be routed to the primary database. A background task is started up that periodically attempts to establish a connection to ReadySet and check its status. When the task confirms that ReadySet is available again, it allows queries to be routed to ReadySet again.
This feature is disabled by default. To enable it, set the
config.failover.enabled configuration option to true. You can read about the
other available configuration options here.
The gem's configuration options can be set by passing a block to
Readyset.configure and setting options on the yielded
Readyset::Configuration object. The available options are documented below.
The values below are the default values for each of the options.
Readyset.configure do |config|
# Whether the gem's automatic failover feature should be enabled.
config.failover.enabled = false
# Sets the interval upon which the background task will check
# ReadySet's availability after failover has occurred.
config.failover.healthcheck_interval = 5.seconds
# Sets the time window over which connection errors are counted
# when determining whether failover should occur.
config.failover.error_window_period = 1.minute
# Sets the number of errors that must occur within the configured
# error window period in order for failover to be triggered.
config.failover.error_window_size = 10
# The file in which cache migrations should be stored.
config.migration_path = File.join(Rails.root, 'db/readyset_caches.rb')
end
FAQs
Unknown package
We found that readyset demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.