znowflake_client

Znowflake Client for Ruby

Retrieves unique, 53 bit, numeric IDs from the Znowflake service.

Znowflake supports unique ID generation for clients that can't generate ULIDs, which is the preferred ID format at Zendesk, and should be used for new database objects wherever possible. See the Zendesk ID Generation Standards for more context and check out the Guide ID Strategy on how Guide is using this to move towards ULID.

Usage

Add this line to your application's Gemfile:

gem 'znowflake_client'

Configure the client and point it at your Znowflake instance, which will be one of:

• Service inside k8s? znowflake.svc.cluster.local:12080
• Service outside k8s? znowflake.service.consul:12080
• Local dev or test? localhost:12080 or znowflake:12080

ZnowflakeClient.configure do |configuration|
  configuration.host = '' # e.g. 'localhost'
  configuration.port = '' # e.g. '12080'
  configuration.open_timeout = 0.1 # defaults to 0.1, only override if you must
  configuration.read_timeout = 0.1 # defaults to 0.1, only override if you must
  configuration.service_name = 'my_awesome_zendesk_service'
end

Generating IDs with Rails

The ZnowflakeClient::ActiveRecordExtension sets a before_create ActiveRecord callback to retrieve an ID from Znowflake on record creation, overriding the id attribute with the returned value. (Requires ActiveRecord to be defined)

# A specific model
class Foo < ActiveRecord::Base
  include ZnowflakeClient::ActiveRecordExtension

  # ...
end

# All models in a common superclass.
class ApplicationRecord < ActiveRecord::Base
  include ZnowflakeClient::ActiveRecordExtension

  self.abstract_class = true
end

# Monkey patching, if that's your thing (not recommend)
# config/initializers/01_monkey_patches.rb
ActiveRecord::Base.include(ZnowflakeClient::ActiveRecordExtension)

Generating IDs without Rails

ZnowflakeClient.generate_id(table_name: 'a_table_in_the_db')
# or
ZnowflakeClient.generate_ids(table_name: 'a_table_in_the_db', count: 42)

Transitioning from ZendeskUniqueIdMigration to ZnowflakeClient

If your application is already using ZendeskUniqueIdMigration and you want to transition to using ZnowflakeClient directly, follow these steps:

Step 1: Update the Gemfile

Remove zendesk_unique_id_migration and ensure znowflake_client is present in your Gemfile:

-gem 'zendesk_unique_id_migration'
 gem 'znowflake_client'

Then run bundle install.

Step 2: Update the initializer

Update your initializer to remove the setup for ZendeskUniqueIdMigration and GlobalUid.disable!.

Here is an example of how your initializer might look like after the changes:

require 'zendesk_database_support'

ZnowflakeClient.configure do |config|
  config.host = ENV.fetch('ZNOWFLAKE_HOST')
  config.port = ENV.fetch('ZNOWFLAKE_PORT')
  config.service_name = DD_SERVICE
end

Step 3: Update your models

You should now remove ZendeskUniqueIdMigration::UseZnowflakeWithGlobalUidFallback and include ZnowflakeClient::ActiveRecordExtension in your models.

Here's an example for a generic model:

 class MyModel < ActiveRecord::Base
-  prepend ZendeskUniqueIdMigration::UseZnowflakeWithGlobalUidFallback
+  include ZnowflakeClient::ActiveRecordExtension
   # ... rest of your code
 end

Step 4: Test your changes

Now, when creating a new record, it will call the ZnowflakeClient to generate a unique ID. If Znowflake fails to generate an ID, it will raise a ZnowflakeClient::Error which you can handle at the application level.

Make sure to run your tests and manually verify that your application behaves as expected with these changes. If Znowflake service is not available in your testing or local development environment, you can use ZnowflakeClient's 'fake' mode as described in the readme.

Application development with Znowflake Client

While developing locally or running tests it might not be desirable to run a real instance of the Znowflake service.

ZnowflakeClient provides a 'fake' mode that can be enabled by adding the following code to your test/development environment.

When configured like this, ZnowflakeClient will not make any outbound requests, but instead generate Znowflake IDs based on the current time and an internal counter.

require 'znowflake_client/testing/fake_znowflake'
ZnowflakeClient.client = ZnowflakeClient::Testing.client

This code is not loaded by default and must be explicitly required. Only do so while in test/development.

The generated IDs follow the Znowflake binary scheme. The 'chunk' is selected based on the current UNIX time when the client object is instantiated. A single, in-memory counter increments to return the sequence of IDs.

Note that for simplicity the generated IDs are sequential, and this is not the case in production.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/zendesk/znowflake_client-ruby for questions ping the #ruby-core-team on Slack.

Development

After checking out the repo, run bundle install to install dependencies. You can also run bin/console for an interactive prompt that will allow you to experiment.

Test

The Znowflake client is dependent on the Znowflake server.

To run the tests:

# Ensure dynamodb and znowflake are running
zdi dynamodb status || zdi dynamodb start
zdi znowflake status || zdi znowflake start

# Run all specs; use the 'spec' or 'integration' tasks to run the respective tests
bundle exec rake

Releasing

• Changes should be reflected in the changelog.
• Update the version number in version.rb
• Run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to artifactory/jfrog.