w95/tiktoken
The Tiktoken API is a tool that enables developers to calculate the token usage of their OpenAI API requests before sending them, allowing for more efficient use of tokens.
This tool helps developers estimate the cost of their OpenAI API calls. You provide text messages and specify an OpenAI model, and it tells you how many tokens that input consumes. This is essential for developers building applications that use OpenAI models, allowing them to manage expenses and optimize API usage.
No commits in the last 6 months.
Use this if you are a developer building an application with OpenAI's API and need to predict token usage and manage costs.
Not ideal if you are not a developer or do not work directly with OpenAI's API requests.
Stars
7
Forks
—
Language
Python
License
MIT
Category
Last pushed
Mar 31, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/w95/tiktoken"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
aiqinxuancai/TiktokenSharp
Token calculation for OpenAI models, using `o200k_base` `cl100k_base` `p50k_base` encoding.
pkoukk/tiktoken-go
go version of tiktoken
dqbd/tiktokenizer
Online playground for OpenAPI tokenizers
microsoft/Tokenizer
Typescript and .NET implementation of BPE tokenizer for OpenAI LLMs.
lenML/tokenizers
a lightweight no-dependency fork from transformers.js (only tokenizers)