gpt-tokens

gpt-tokens

GPT tokens / price Calculate

Install

# npm or yarn

npm install gpt-tokens
yarn add gpt-tokens

Usage

import { GPTTokens } from 'gpt-tokens'

const usageInfo = new GPTTokens({
    model: 'gpt-3.5-turbo',
    messages: [
        {
            'role'   : 'system',
            'content': 'You are a helpful assistant',
        },
        {
            'role'   : 'user',
            'content': '',
        },
    ]
})

// Tokens: 18
console.log('Tokens:', usageInfo.usedTokens)
// Price USD: 0.000036
console.log('Price USD: ', usageInfo.usedUSD)

You can test in Tiktokenizer

Support Models

• gpt-3.5-turbo
• gpt-3.5-turbo-0301
• gpt-4
• gpt-4-0314
• gpt-4-32k
• gpt-4-32k-0314

For web

If use vite, you need do this in vite.config.js

import wasm             from 'vite-plugin-wasm'
import topLevelAwait    from 'vite-plugin-top-level-await'
import { defineConfig } from 'vite'

export default defineConfig({
    plugins: [wasm(), topLevelAwait()],
})

Compatibility

As the dependency @dqbd/tiktoken is a WASM library, you can find the compatibility methods in the NPM Readme of @dqbd/titoken.

Dependencies

• OpenAI tiktoken (Python lib) openai/tiktoken
• @dqbd/titoken (Typescript) dqbd/tiktoken