schema_plus_core

SchemaPlus::Core

SchemaPlus::Core creates an internal extension API to ActiveRecord. The idea is that:

• SchemaPlus::Core does the monkey-patching so clients don't have to know too much about the internal of ActiveRecord.

• SchemaPlus::Core's extension API is consistent across the various connection adapters, so clients don't have to figure out how to extend each connection adapter independently.

• SchemaPlus::Core's extension API intends to remain reasonably stable even as ActiveRecord changes.

By itself, SchemaPlus::Core does not change any behavior or add any external features to ActiveRecord. It just makes the API available to clients.

SchemaPlus::Core is a client of schema_monkey, using modware to define middleware callback stacks.

Compatibility

SchemaPlus::Core is tested on:

• ruby 2.5 with activerecord 5.2, using postgresql:9.6, mysql2 or sqlite3
• ruby 2.5 with activerecord 6.0, using postgresql:9.6, mysql2 or sqlite3
• ruby 2.5 with activerecord 6.1, using postgresql:9.6, mysql2 or sqlite3
• ruby 2.7 with activerecord 5.2, using postgresql:9.6, mysql2 or sqlite3
• ruby 2.7 with activerecord 6.0, using postgresql:9.6, mysql2 or sqlite3
• ruby 2.7 with activerecord 6.1, using postgresql:9.6, mysql2 or sqlite3
• ruby 2.7 with activerecord 7.0, using postgresql:9.6, mysql2 or sqlite3
• ruby 3.0 with activerecord 6.0, using postgresql:9.6, mysql2 or sqlite3
• ruby 3.0 with activerecord 6.1, using postgresql:9.6, mysql2 or sqlite3
• ruby 3.0 with activerecord 7.0, using postgresql:9.6, mysql2 or sqlite3
• ruby 3.1 with activerecord 6.0, using postgresql:9.6, mysql2 or sqlite3
• ruby 3.1 with activerecord 6.1, using postgresql:9.6, mysql2 or sqlite3
• ruby 3.1 with activerecord 7.0, using postgresql:9.6, mysql2 or sqlite3

As of version 2.0.0, `schema_plus_core` supports only ActiveRecord >= 5.0. ActiveRecord 4.2.x is supported in version 1.x, maintained in the 1.x branch.

Breaking changes in version 2.0

• SchemaPlus::Core::Schema::Tables middleware was replaced by SchemaPlus::Core::DataSources. Parameters have also changed.
• SchemaPlus::Core::Dumper::Indexes middleware was removed; instead SchemaPlus::Core::Dumper::Table sets both columns and indexes on env.table.

Installation

As usual:

gem "schema_plus_core"                # in a Gemfile
gem.add_dependency "schema_plus_core" # in a .gemspec

Usage

The API is in the form of a collection of modware middleware callback stacks. A client of the API uses schema_monkey to insert middleware modules into the stacks. As per schema_monkey, the typical module structure looks like:

require "schema_plus/core"

module MyClient
  module Middleware
    #
    # Middleware modules to insert in SchemaPlus::Core API stacks
    #
  end
  module ActiveRecord
    #
    # direct ActiveRecord enhancements, should your client need any.
    #
  end
end

SchemaMonkey.register MyClient

For example, a client could use the Migration::Index stack to automatically make an index unique if any column starts with 'u':

require "schema_plus/core"

module AutoUniquify
  module Middleware

    module Migration
      module Index
        def before(env)
          env.options[:unique] = true if env.column_names.grep(/^u/).any?
        end
      end
    end

  end
end

SchemaMonkey.register AutoUniquify

Ideally most clients will not need to define direct ActiveRecord enhancements, other than perhaps to create new methods on public classes. If you have a client that needs more complex monkey-patching, that could be a sign that SchemaPlus::Core's API is missing some useful functionality -- consider submitting a PR to SchemaPlus::Core add it!

API details

For organizational clarity, the SchemaPlus::Core stacks are grouped into modules based on broad categories. In the Env field tables below, Initialized

Schema

Stacks for general operations queries pertaining to the entire database schema:

• Schema::Define

  Wrapper around the ActiveRecord::Schema.define method loads a dumped schema file (schema.rb).

  Env Field	Description	Initial value
  :info	Schema information hash	args
  :block	The proc containing the schema definition statements	args

  The base implementation calls the block to define the schema.

• Schema::Indexes

  Wrapper around the connection.indexes(table_name) method. Env contains:

  Env Field	Description	Initial value
  :index_definitions	The result of the lookup	[]
  :connection	The current ActiveRecord connection	context
  :table_name	The name of the table to query	arg

  The base implementation appends its results to env.index_definitions

• Schema::DataSources

  Wrapper around the connection.data_sources() method. Env contains:

  Env Field	Description	Initialized
  :data_sources	The result of the lookup	[]
  :connection	The current ActiveRecord connection	context

  The base implementation appends its results to env.tables

Model

Stacks for class methods on ActiveRecord models.

• Model::Columns

  Wrapper around the Model.columns query

  Env Field    | Description | Initialized
  

  --- | --- | --- :columns | The resulting Column objects | [] :model | The model Class being queried | context

  The base implementation appends its results to env.columns

• Model::ResetColumnInformation

  Wrapper around the Model.reset_column_information method

    Env Field    | Description | Initialized
    --- | --- | ---
    `:model` | The model Class being reset | *context*
  

  The base implementation performs the reset.

• Model::Association::Declaration

  Wrapper around the Model.has_many, Model.has_and_belongs_to_many, Model.has_one, and Model.belongs_to methods

    Env Field    | Description | Initialized
    --- | --- | ---
    `:model`     | The model Class being defined | *context*
    `:name`      | The name of the association being defined. | *arg*
    `:scope`     | The scope lambda associated with the association | *arg*
    `:options`   | Options associated with the association. | *arg*
    `:extension` | Extensions to the association to be made. | *arg*
  

  The base implementation creates the association.

Migration

Stacks for operations that change the schema. In some cases the operation immediately modifies the database schema, in others the operation defines ActiveRecord objects (e.g., column definitions in a create_table definition) and the actual modification of the database schema will happen some time later.

• Migration::Column

  Callback stack for various ways to define or modify a column.

  Env Field	Description	Initialized
  :caller	The ActiveRecord instance responsible for performing the action	context
  :operation	One of :add, :change, :define, :record	context
  :table_name	The name of the table	arg
  :column_name	The name of the column	arg
  :type	The ActiveRecord column type (:integer, :datetime, etc.)	arg
  :implements_reference	This implements a migration.add_reference, t.references or t.belongs_to [See below]	context
  :options	The column options	arg, default {}

  The base implementation performs the column operation. No value is returned.

  Notes:

  • The :operation field has the following meanings:

    • :add - The column will be added immediately (Migration#add_column)
    • :change - The column will be changed immediately (Migration#change_column)
    • :define - The column will be added to table definition, which will be emitted later
    • :record - The column info will be added to a migration command recorder, for later playback in reverse by Migration#down

  • In the case of a table definition using t.references or t.belongs_to, the :type field will be set to :reference and the :column_name will include the "_id" suffix

  • ActiveRecord's base implementation handles migration.add_reference, t.references and t.belongs_to by making nested calls to migration.add_column or t.column to create the resulting column (or two columns, for polymorphic references). SchemaPlus::Core invokes the Migration::Column stack for both the outer migration.add_reference, t.references or t.belongs_to call, as well as for the nested migration.add_column or t.column call; in the nested call, env.implements_reference will be truthy.

  • Sqlite3 implements change_column by a creating a new table. This will result in nested calls to add_column, invoking the Migration::Column stack for each; SchemaPlus::Core does not currently provide a way to distinguish those calls from explicit top-level calls.

• Migration::CreateTable

  Creates a new table

  Env Field	Description	Initialized
  :caller	The ActiveRecord instance responsible for creating the table	context
  :table_name	The name of the table	arg
  :options	Create table options	arg
  :block	Proc containing table definition statements	arg

  The base implementation creates the table, yielding a table_definition instance to the block (if a block is given).

• Migration::DropTable

  Drops a table from the database

  Env Field	Description	Initialized
  :connection	The current ActiveRecord connection	context
  :table_name	The name of the table	arg
  :options	Drop table options	arg

  The base implementation drops the table. No value is returned.

• Migration::RenameTable

  Renames a table

  Env Field	Description	Initialized
  :connection	The current ActiveRecord connection	context
  :table_name	The existing name of the table	arg
  :new_name	The target name of the table	arg

  The base implementation renames the table. No value is returned.

• Migration::Index

  Callback stack for various ways to define an index.

  Env Field	Description	Initialized
  :caller	The ActiveRecord instance responsible for performing the action	context
  :operation	:add or :define	context
  :table_name	The name of the table	arg
  :column_names	The names of the columns	arg
  :options	The index options	arg, default {}

  The base implementation performs the index creation operation. No value is returned.

  Notes:

  • The :operation field has the following meanings:

    • :add - The index will be added immediately (Migration#add_index)
    • :define - The index will be added to a table definition, which will be emitted later.

Sql

Stacks for internal operations that generate SQL.

• Sql::ColumnOptions

  Callback stack around generation of the SQL options for a column definition.

  Env Field	Description	Initialized
  :sql	The resulting SQL	""
  :caller	The ActiveRecord::SchemaCreation instance	context
  :connection	The current ActiveRecord connection	context
  :column	The column definition object	context
  :options	The column definition options	context

  The base implementation appends the options SQL to env.sql

• Sql::IndexComponents

  Callback stack around generation of the SQL for an index definition.

  Env Field	Description	Initialized
  :sql	The resulting SQL components, in a struct with fields :name, :type, :columns, :options, :algorithm, :using	empty struct
  :connection	The current ActiveRecord connection	context
  :table_name	The name of the table	context
  :column_names	The names of the columns	context
  :options	The index options	context

  The base implementation overwrites the contents of env.sql

  Notes:

  • SQLite3 ignores the :type, :algoritm, and :using fields of env.sql

• Sql::Table

  Callback stack around generation of the SQL for a table

  Env Field	Description	Initialized
  :sql	The resulting SQL components in a struct with fields :command, :name, :body, :options, :quotechar	empty struct
  :caller	The ActiveRecord::SchemaCreation instance	context
  :connection	The current ActiveRecord connection	context
  :table_definition	The TableDefinition object for the table	context

  The base implementation overwrites the contents of env.sql

  Notes:

  • env.sql.command contains the index creation command such as CREATE TABLE or CREATE TEMPORARY TABLE
  • env.sql.quotechar contains the quote character ', ", or ` to wrap env.sql.name in.

Query

Stacks around low-level query execution

• Query::Exec

  Callback stack wraps the emission of sql to the underlying dbms gem.

  Env Field    | Description | Initialized
  

  --- | --- | --- :result | The result of the database query | unset :caller | The ActiveRecord::SchemaCreation instance | context :sql | The SQL string | context :binds | Values to substitute into the SQL string :query_name | Label sometimes used by ActiveRecord logging | arg

Dumper

SchemaPlus::Core provides a state object and of callbacks to various phases of the schema dumping process. The dumping process fleshes out the state object-- nothing is actually written to the dump file until after the state is fleshed out.

Schema Dump state

• Class SchemaPlus::Core::SchemaDump

  An instance of SchemaPlus::Core::SchemaDump gets passed to each of the callback stacks; the dump gets built up by fleshing out its contents. SchemaDump has the following fields and methods:

  • dump.initial = [] - an array of strings containing statements to start the schema with, such as enable_extension 'hstore'
  • dump.data = OpenStruct.new - a place for clients to store arbitrary data between phases
  • dump.tables = {} - a hash mapping table names to SchemaDump::Table objects
  • dump.final = [] - an array of strings containing statements to end the schema with.
  • dump.depends(table_name, [prerequisite_table_names]) - call this method to ensure that the definition of table_name won't be output before its prerequisites.

• Class SchemaPlus::Core::SchemaDump::Table

  Each table in the dump has its contents in a SchemaDump::Table object, with these fields:

  • table.name - the table name
  • table.pname - table as actually used in SQL (without any prefixes or suffixes)
  • table.options - a string containing the options to Migration.create_table
  • table.columns = [] - an array of SchemaDump::Table::Column objects
  • table.indexes = [] - an array of SchemaDump::Table::Index objects
  • table.statements - a collection of statements to include in the table definition; each is a string that should start with "t."
  • table.trailer - a collection of migration statements to include immediately outside the table definition. Each is a string
  • table.alt - In some cases, ActiveRecord is unable to dump a table in the form of a migration create_table statement; in this case table.pname will be nil, and table.alt will contain the alternate string to dump instead. (E.g. if the table contains custom types, ActiveRecord will be unable to handle it and will just dump an error message as a comment.)

• Class SchemaPlus::Core::SchemaDump::Table::Column

  Each column in a table has its contents in a SchemaDump::Table::Column object, with these fields and methods:

  • column.name - the column name
  • column.type - the column type (i.e., what comes after "t.")
  • column.options - a hash containing the options for the column
  • column.comments - an array of comment strings for the column

• Class SchemaPlus::Core::SchemaDump::Table::Index

  Each index in a table has its contents in a SchemaDump::Table::Index object, with these fields and methods:

  • index.name - the index name
  • index.columns - the columns that are in the index
  • index.options - a hash containing the options for the column

Schema Dump Middleware stacks

• Dumper::Initial

  Callback stack wraps the creation of initial statements for the dump.

  Env Field	Description	Initialized
  :initial	The initial statements	[]
  :dump	The SchemaDump object	context
  :dumper	The current ActiveRecord::SchemaDumper instance	context
  :connection	The current ActiveRecord connection	context

  The base method appends initial statements to env.initial.

• Dumper::Tables

  Callback stack wraps the dumping of all tables.

  Env Field	Description	Initialized
  :dump	The SchemaDump object	context
  :dumper	The current ActiveRecord::SchemaDumper instance	context
  :connection	The current ActiveRecord connection	context

  The base method iterates through all tables, dumping each.

• Dumper::Table

  Callback stack wraps the dumping of each table

  Env Field	Description	Initialized
  :table	A SchemaDump::Table object	table.name only
  :dump	The SchemaDump object	context
  :dumper	The current ActiveRecord::SchemaDumper instance	context
  :connection	The current ActiveRecord connection	context

  The base method iterates through all columns and indexes of the table, and overwrites the contents of table,

  Notes:

  • When the stack is called, env.dump.tables[env.table.name] contains the env.table object.
  • The base method sets both env.table.columns and env.tables.indexes.

Release Notes

• 3.1.0 Add AR 6.1 and 7.0, add Ruby 3.1
• 3.0.0 Drop AR < 5.2 and add AR 6.0 support. Drop Ruby < 2.5 and add Ruby 3.0 support.
• 2.2.3 Fix dumping complex expression based indexes in AR 5.x
• 2.2.2 Fixed dumping tables in postgresql in AR 5.2 when the PK is not a bigint.
• 2.2.1 Fixed expression index handling in AR5.x.
• 2.2.0 Added AR5.2 support. Thanks to @jeremyyap
• 2.1.1 Bug fix: Don't lose habtm options. Thanks to @iagopiimenta
• 2.1.0 Added AR5.1 support. Thanks to @iagopiimenta
• 2.0.1 Tighten up AR dependency. Thanks to @myabc.
• 2.0.0 Added AR5 support, removed AR4.2 support. Thanks to @boazy.
• 1.0.2 Missing require
• 1.0.1 Explicit gem dependencies
• 1.0.0 Clean up SchemaDump::Table::Column and SchemaDump::Table::Index API: #options is now a hash and #comments is now an array; no longer have add_option and add_comment methods.
• 0.6.2 Bug fix: don't choke on INHERITANCE in table definition (#7). Thanks to @ADone.
• 0.6.1 Make sure to require pathname (#5)
• 0.6.0 Added table.alt to dumper; Bug fix: Don't crash when AR fails to dump a table. Thanks to @stenver for tracking it down
• 0.5.1 Bug fix: Don't choke on a quoted newline in a CREATE TABLE statement (#3). Thanks to @mikeauclair
• 0.5.0 Added Migration::DropTable
• 0.4.0 Added implements_reference to Migration::Column stack env
• 0.3.1 Pass along (undocumented) return values from association declarations (#2). Thanks to @lowjoel
• 0.3.0 Added Model::Association::Declaration (#1). Thanks to @lowjoel.
• 0.2.1 Added Migration::CreateTable and Schema::Define; removed dependency on (defunct) schema_monkey_rails gem. [Oops, this should have been a minor version bump]
• 0.2.0 Added Migration::DropTable
• 0.1.0 Initial release

Development & Testing

Are you interested in contributing to SchemaPlus::Core? Thanks! Please follow the standard protocol: fork, feature branch, develop, push, and issue pull request.

Some things to know about to help you develop and test:

• schema_dev: SchemaPlus::Core uses schema_dev to facilitate running rspec tests on the matrix of ruby, activerecord, and database versions that the gem supports, both locally and on github actions

  To to run rspec locally on the full matrix, do:

    $ schema_dev bundle install
    $ schema_dev rspec
  

  You can also run on just one configuration at a time; For info, see schema_dev --help or the schema_dev README.

  The matrix of configurations is specified in schema_dev.yml in the project root.

• schema_monkey: SchemaPlus::Core is implemented as a schema_monkey client, using schema_monkey's convention-based protocols for extending ActiveRecord and using middleware stacks.