@dqbd/tiktoken
Advanced tools
Changelog
[v0.4.0]
decode_batch
and decode_bytes_batch
Changelog
[v0.3.0]
gpt-3.5-turbo
models to encoding_for_model
encoding_for_model
to better support future model versionsChangelog
[v0.2.0]
tiktoken.encoding_for_model
to get the encoding for a specific modelThank you to @fritzo, @arvid220u, @khanhvu207, @henriktorget for various small corrections