tiktoken is a fast BPE tokeniser for use with OpenAI's models
tiktoken is a fast BPE tokeniser for use with OpenAI's models.
| Release | Stable | Testing |
|---|---|---|
| Fedora Rawhide | 0.13.0-4.fc45 | - |
| Fedora 44 | 0.13.0-2.fc44 | - |
| Fedora 43 | 0.13.0-2.fc43 | - |
You can contact the maintainers of this package via email at
python-tiktoken dash maintainers at fedoraproject dot org.