anasfik/gpt_3_encoder

A pure Dart implementation of OpenAI's original Python encoder/decoder for GPT models.

20
/ 100
Experimental

This tool helps developers working with large language models to accurately convert human-readable text into numerical tokens, and vice-versa. It ensures that text processed by GPT-2 or GPT-3 models is correctly formatted for input and that their outputs can be reliably interpreted. Dart developers integrating these models into their applications would find this useful.

No commits in the last 6 months.

Use this if you are a Dart developer building applications that need to precisely manage text tokenization for GPT-2 or GPT-3 models.

Not ideal if you are a non-developer or are working with language models that do not use OpenAI's specific byte pair encoding (BPE) tokenizer.

Dart-development language-model-integration text-tokenization AI-application-development
Stale 6m No Package No Dependents
Maintenance 0 / 25
Adoption 4 / 25
Maturity 16 / 25
Community 0 / 25

How are scores calculated?

Stars

7

Forks

Language

Dart

License

MIT

Last pushed

May 10, 2023

Commits (30d)

0

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/anasfik/gpt_3_encoder"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.