@copepod/kv

@copepod/kv

Pluggable, statically-configured modular key-value stores.

[!CAUTION] Under development: This module is under development and not ready for production use. The API is neither complete nor stable, and may change without notice.

Installation

# pnpm
pnpm add @copepod/kv

# bun
bunx add @copepod/kv

# npm
npx add @copepod/kv

# yarn
yarn add @copepod/kv

# deno
deno add npm:@copepod/kv

Configuration

Configuration is static: there is no API to configure stores, and all the necessary configuration is in the configuration file.

The configuration file associates store identifiers with backends and backend parameters. It can be a JSON file, a YAML file, a Javascript or Typescript file, and must be named kv.config.* where * is the extension of the file. It's also possible to simply pass a configuration in the project's package.json file under the kv key.

Examples

package.json

{
  "name": "my-project",
  "type": "module",
  "version": "1.0.0",
  // other fields omitted
  "kv": {
    "stores": [
      {
        "id": "build-cache",
        "use": "@copepod/kv/fs-simple",
        "with": {
          "path": ".cache"
        }
      },
      // Add more stores here
    ]
  }
}

kv.config.ts

import type { Config } from '@copepod/kv/config'

const buildCache: {
  // Store id
  id: 'build-cache',
  // Backend module
  use: '@copepod/kv/fs-simple',
  // Backend configuration
  with: {
    path: '.cache',
  },
}

export default {
  stores: [
    buildCache,
  ],
} satisfies Config

Usage

import { kv } from '@copepod/kv'

// Get a store. If no store is defined for that id, returns `undefined`.
const store = await kv.store('build-cache')

// Get value with key 'image.jpg' from store
const value = await store.get('image.jpg')

// Set value with key 'image.jpg' in store 'images'
const success = await store.set('image.jpg', value)

// Now delete that entry
await store.set('image.jpg', undefined)

API

• kv.store<Key>(store: string): Promise<Store<Key> | undefined>: Get a store by its id.
• store.get(key: Key): Promise<Uint8Array | undefined>: Get a value from a store. The key can be any keyable material understood by the store backend.
• store.set(key: Key, value: Uint8Array | undefined): Promise<boolean>: Set a value in a store. Passing undefined as the value will delete the key from the store. Returns true if the operation was successful, false otherwise.

Backends

Backends are modules that implement the key-value store interface. Here is a very simple backend that stores values in memory (which is admitedly not very useful):

import type { Store } from '@copepod/kv/types'

export interface Config {
  [key: string]: any
}

export default class MemoryStore implements Store<string> {
  private cache: Map<string, Uint8Array>

  constructor(params: Config) {
    this.cache = new Map()
  }

  async get(key) {
    return cache.get(key)
  }

  async set(key, value) {
    if (value === undefined) {
      cache.delete(key)
    }
    else {
      cache.set(key, value)
    }
  }
}

Backends are composable. You can create a backend that uses another backend to store values, and add additional features. Here's a backend that supports composite keys and use the previous backend to store values:

import type { Store } from '@copepod/kv/types'
import MemoryStore from './MemoryStore'

export interface CompositeKey {
  a: string
  b: string
}

export default class CompositeKeyMemoryStore implements Store<CompositeKey> {
  private underlyingStore: MemoryStore

  constructor(params: { [key: string]: any }) {
    // You could derive a different configuration for the underlying
    // store from the params, here.
    this.underlyingStore = new MemoryStore(params)
  }

  async get(key) {
    return this.underlyingStore.get(JSON.stringify([key.a, key.b]))
  }

  async set(key, value) {
    return this.underlyingStore.set(JSON.stringify([key.a, key.b]), value)
  }
}

You can of course do more interesting things in your backends. Here is one that stores images on the filesystem based on transformation parameters:

import type { Store } from '@copepod/kv/types'
import { readFile, writeFile } from 'node:fs/promises'

export interface Config {
  path: string
  [key: string]: any
}

export interface Key {
  name: string
  type: string
  width: number
  height: number
  [key: string]: string | number
}

export default class ImageStore implements Store<TransformParams> {
  private path: string

  constructor(params) {
    const { path } = params as Config
    this.path = path
  }

  async get(key) {
    const { name, type, width, height, ...rest } = key
    const other = Object.entries(rest).toSorted((a, b) => a[0].localeCompare(b[0])).map(([k, v]) => `${k}=${v}`).join(',')
    const path = `${this.path}/${name}[${width},${height},${other}].${type}`
    try {
      return await readFile(path)
    }
    catch {
      return undefined
    }
  }

  async set(key, value) {
    const { name, type, width, height } = key
    const other = Object.entries(rest).toSorted((a, b) => a[0].localeCompare(b[0])).map(([k, v]) => `${k}=${v}`).join(',')
    const path = `${this.path}/${name}[${width},${height},${other}].${type}`
    try {
      await writeFile(path, value)
      return true
    }
    catch {
      return false
    }
  }
}

FAQ

Can I set a TTL?

No, this module does not support TTLs. It provides simple key-value stores with no expiration mechanism.

TTLs can be implemented by the backends, for instance by accepting expiration dates in composite keys.

Like it? Buy me a coffee!

If you like anything here, consider buying me a coffee using one of the following platforms:

---