@dialpad/i18n-services

i18n-services

Introduction

@dialpad/i18n-services contains all the tooling needed to manage the resources for our i18n packages, i.e. @dialpad/i18n-vue2 for legacy apps and @dialpad/i18n for modern apps. this includes:

• Uploading new strings to a translation service.
• Downloading translated strings from a translation service.
• Loading bundled resources at runtime for a given locale.
• Generating a "Dialpadistan" locale from en-US.

How it works?

There are essentially two main concepts behind the scenes:

BaseLocaleManager

The BaseLocaleManager having the common logic that oversees internationalization (i18n) in a Vue2.js and a Vue3.js application using the Fluent localization framework. Its responsibilities include:

• Loading and caching translation bundles: Manages the .ftl data, which maps keys to text for each locale.
• Switching the current locale: Provides methods to change the active language dynamically.
• Handling bundle namespaces: Manages potential duplicate bundle requests from multiple sources, though this feature is still being refined.
• Initializing the Vue plugin: Exposes translation methods ($t, $ta) to Vue templates, enabling localized content within the application.

BundleSource

The second key concept is the BundleSource interface, which abstracts the loading and management of localization resources. This interface has two primary implementations: RawBundleSource and HTTPBundleSource, each designed for different use cases.

• RawBundleSource: This implementation handles localization resources defined within the application. It organizes these resources defined directly in the application, and imported directly at runtime (for example, using import(./[locale].ftl). This approach is ideal for applications with mostly static content.

• HTTPBundleSource: This implementation fetches localization resources dynamically from a server. It caches these resources locally, allowing for efficient access while still being flexible enough to load translations on demand.

Disclaimer: Both implementations are provisional and may change as we better understand our needs. Currently, RawBundleSource is the preferred option for most applications.

Synchronous Resource Loading with builtResources

While the dynamicResources method is used for asynchronously loading resources, the RawBundleSource class also provides a builtResources method for synchronous resource loading. This is useful when you have the localization resources available at compile time and prefer to import them directly without dealing with promises.

How to use builtResources

Instead of dynamically importing .ftl files, you can import them directly and use the builtResources method to process the resources. Here's an example:

import { RawBundleSource } from '@dialpad/i18n-services';
// Import your .ftl files directly
import enUSResource from './locales/en-US.ftl';
import esResource from './locales/es.ftl';

// Create an array of BuiltResource objects
const builtResourcesArray: BuiltResource[] = [
  ['en-US', 'namespace', enUSResource],
  ['es', 'namespace', esResource],
  // ... add other locales and namespaces as needed
];

// Convert BuiltResource objects to Resource objects using builtResources
const resources: Resource[] =
  RawBundleSource.builtResources(builtResourcesArray);

// Now, resources can be used to instantiate RawBundleSource or for any other purpose
const bundleSource = new RawBundleSource({ resources });

If you're looking for the 'Dialpadistan' translator, please skip to this section.

Setup

Installation

pnpm add @dialpad/i18n-services

Configuration

You can find the needed configuration for using $t and $ta functions in your application in the i18n for vue2 and i18n for vue3 documentation. In both cases the needed configuration should be the same.

Usage

Adding Resources at Runtime with addSources

API Method: addSources

The addSources method allows you to add multiple translation resources to the LocaleManager at runtime. Once all resources are loaded, it triggers the ready promise indicating that the locale manager is fully initialized and ready to handle localization requests.

Parameters

• sources: An array of BuiltResource items where each item contains:
  • locale: The locale code for the translation resource.
  • namespace: The namespace associated with the resource.
  • source: The actual source string or a promise that resolves to the source string.

Example

// Assuming you have a LocaleManager instance
const manager = new LocaleManager(...);

// Define your resources
const resourcesToAdd: BuiltResource[] = [
  // Array of [locale, namespace, source] tuples
  ['en-US', 'namespace1', sourceStringOrPromise],
  ['es-ES', 'namespace1', anotherSourceStringOrPromise],
  // Add more resources as needed
];

// Add the resources to the LocaleManager
manager.addSources(resourcesToAdd);

I18n interface

The API for our i18n tools is the same either for the vue3 or vue2 supported versions. You can find the more details about how to use the tool in the i18n for vue2 and i18n for vue3 documentation.

Dialpadistan translator

This feature uses the @fluent/syntax parser and a set of regex and grammar rules to convert an en-US.ftl file into an dp-DP.ftl inside the same directory. The "dialpadistani" language is just the original language transformed character-per-character to non-latin characters which is useful to test the proper render of different fonts.

For example:

# src/localization/en-US.ftl

MESSAGE_INPUT_LABEL = New message input
FEED_NEW_SEPARATOR = Beginning of unread items

Will translate to:

# src/localization/dp-DP.ftl

MESSAGE_INPUT_LABEL = Ŋάŵ ɱάššëğά īŋþø†
FEED_NEW_SEPARATOR = Sάğīŋŋīŋğ ůƒ øŋřάëḍ ī†άɱš

To execute a Dialpadistan translation you should run the following script pointing to your Fluent English file: (Note: it can either be a relative path or an absolute path)

pnpm pnpm --filter=@dialpad/i18n-services run translate:dialpadistan /your/english/fluent/file/en-US.ftl

When succeeding translating, you will notice a message on your terminal stating the folder where the output file will be placed. This folder is the same as your English Fluent file.

e.g. in this case you will see File transformed and saved as /your/english/fluent/file/dp-DP.ftl.

Commit hook

This translator includes a post-commit that automatically translates english files to dialpadistani. This hook checks the commit for changed en-US.ftl files and will only run the translator against those files that had changes in the commit.

If this script finally makes any change to dp-DP.ftl files, it will automatically commit those changes with the message: chore(auto-generated): translate en-US.ftl files to dialpadistani

Known issues

There is one specific scenario where this translator is failing, and we don't consider it to be harmful. This is because of a bug on @fluent/syntax parser.

And this is when having a comment right in the middle between an equal sign and a value, and that comment has at least one placeholder, whatever's after the text of the first placeholder will be translated (and if the comment it's multi-lined every comment line will be squashed and inlined after that placeholder as well)

e.g.

MY_KEY =
    # My awesome comment
    # And here the {$placeholder} is present
        .aria-label = { $name } ({ $muted ->
        [true] Muted{" "}
        *[other] {""}
    })

will be translated to:

MY_KEY =
    # My awesome comment
    # And here the {$placeholder} os sintirp
        .aria-label = { $name } ({ $muted ->
        [true] Muted{" "}
        *[other] {""}
    })

This is because the way@fluent/syntax parser works at the current version. In order to fix this it would require an amount of effort that is probably not worth it. It will imply either to patch the parser behavior somehow or extra usage of regex pre and post processing files, which will not only increase complexity of the solution but also reduce the robustness of it. And given the fact that is just a comment, we prefer not to solve it in-house and wait until they fix or improve this behavior on upcoming versions.

But keep in mind that whenever adding comments, it should be before the whole KEY = VALUE and not in between.

Context screenshots

Context screenshots are, as their name suggest, screenshots that provide context to recognize where each translation is used in each app.

Each screenshot should contain visual information of where the translation can be found in the app with enough context to be easily located. If other dev looks at the screenshot they should be able to locate it fairly easily in your app.

There should be one screenshot per each translation key on each .ftl file. You can reuse the same screenshot for multiple keys, but each file will correspond to one key, so you must make one copy for each translation key. The screenshot filename will be {translationKeyHere}.png. We only support .png images for now.

There is a PR check which will fail if any translation key is missing its corresponding screenshot. There is an exception list for paths that should be ignored by this check. Avoid using it unless it's completely necessary. If you strictly need to use it, remember to mention in your commit message why you are using it.

These screenshots will be located in a directory called context-screenshots sibling to each .ftl file in your project. If you have an .ftl file, it should have a sibling screenshots directory. Each screenshot has a file size limit of 20MB.

The screenshots are uploaded to smartling along with the CSV/FTL files through a github action when a PR to main is merged.

Resources

• Fluent docs
• I18n best practices
• i18n for vue2 docs
• i18n for vue3 docs

Translation Workflows

The following GitHub workflows handle various aspects of the translation process:

1. Sync Translations Workflow

Name: Pull translations from third party translation service

Purpose: This workflow synchronizes translations by pulling them from a third-party translation service (Smartling) and automatically creates a pull request if there are any changes.

Triggers:

• Push to the main branch
• Daily scheduled run at midnight (cron: 0 0 * * *)
• Manual trigger (workflow_dispatch)

Configuration:

• Uses the pull-translations.yml workflow from dialpad/wcf-sdk@v0.6.0
• Smartling project ID is retrieved from repository variables
• Working directory is set to "./packages/i18n"
• Node version is determined from the ".nvmrc" file
• Requires Smartling API credentials stored in GitHub secrets

2. Force Pull Translations Workflow

Name: Manual Force Pull Translations

Purpose: This workflow allows for manual forced pulls of translations from the translation service, with configurable options.

Triggers:

• Manual trigger only (workflow_dispatch)

Input Parameters:

• USE_FTL_FLOW: Option to use FTL flow (boolean, default: true)
• SKIP_UNAVAILABLE_LOCALES: Option to skip unavailable locales without failing (boolean, default: true)
• WORKING_DIRECTORY: Specifies the working directory (string, default: ".")
• NODE_VERSION_FILE: Path to the Node version file (string, optional)

Configuration:

• Uses the force-pull-translations.yml workflow from dialpad/wcf-sdk@v0.6.0
• Smartling project ID is retrieved from repository variables
• Requires Smartling API credentials stored in GitHub secrets

3. Translation Screenshots Check Workflow

Name: Check translations screenshots

Purpose: This workflow verifies if the screenshots for translations are up-to-date with their respective FTL files.

Triggers:

• Pull requests targeting the main branch

Configuration:

• Uses the screenshots-translations.yml workflow from dialpad/wcf-sdk@v0.6.0
• Smartling project ID is retrieved from repository variables
• Working directory is set to "./packages/i18n"
• Node version is determined from the ".nvmrc" file
• Requires Smartling API credentials stored in GitHub secrets

4. Upload to Translation Service Workflow

Name: Push en-US.ftl to third party translation service

Purpose: This workflow pushes the English (en-US.ftl) source files to the third-party translation service (Smartling) for translation.

Triggers:

• Push to the main branch

Configuration:

• Uses the push-translations.yml workflow from dialpad/wcf-sdk@v0.6.0
• Smartling project ID is retrieved from repository variables
• Working directory is set to "./packages/i18n"
• Node version is determined from the ".nvmrc" file
• Requires Smartling API credentials stored in GitHub secrets

All workflows use Smartling as the translation service and require proper API credentials (SMARTLING_API_USER and SMARTLING_API_SECRET) stored as GitHub secrets.