clickhouse-ts

clickhouse-ts by @bytadaniel

Select, insert in-memory batches safely and enjoy Clickhouse!

Hello from LookFor.sale developer which produces WB Analytics!

TS client for clickhouse database with using in-memory caching rows for bulk insertion

💙 Typescript

This package is written in TypeScript because Node.js typing development is already an industry standard.

🖐 Batch insert*

It has been empirically verified that in-memory collecting rows is the most efficient and consistent way to insert into Clickhouse. To work with built-in caching, you just need to call the useCaching() method

💪 Transparent and Stability

clickhouse-ts doesn't use a lot of abstractions and dependencies, so it's fast and stable.

🏗 Ready for production

The Lookforsale team has been using clickhouse-ts effectively for over a year under extreme loads!

👍 Batch validation

Double checking data for anomalies during in-memory caching and when inserting a finished batch.

✨ Flexibility

Flexible configuration of the Clickhouse client instance and support for all features provided by Clickhouse developers.

🔐 Security

SQL Injection Protection with sqlstring

🌈 Free for use

The package has a public license and is available for download to any developer!

*How can I insert in-memory batches?

Starting from version 2.0.0 the caching module should be imported separately. This is because clickcache package, like clickhouse-ts, is going to be part of my Clickhouse Node.js ecosystem. In addition, it planned to introduce data validation, as in Joi and model generation, as in mongodb/mongoose.

Usage

const client = new Clickhouse(
  {
    url: 'url',
    port: 8443,
    user: 'user',
    password: 'password',
    database: 'database',
    ca: fs.readFileSync('cert.crt')
  },
  {
    defaultResponseFormat: 'JSON',
    clickhouseOptions: {
      /* https://clickhouse.tech/docs/en/operations/settings/settings/ */
      send_progress_in_http_headers: '1'
    }
  }
)

Insert

const response = await client.insert('table_strings', rows, {
  responseFormat: 'CSVWithNames' // or other format
  // other query options
})

Select

await clickhouseInstance.query<{ t: string }>('WITH now() as t SELECT t', {
  responseFormat: 'TSV',
  // ...other query options
})

Create

await clickhouseInstance.query(`
  CREATE TABLE strings (
    date DateTime('UTC'),
    string String
  ) Engine = ReplacingMergeTree()
  PARTITION BY toMonday(date)
  ORDER BY (date, string)
`)