rust-tokenizers+hf-hub-devel Subpackage of rust-tokenizers
Implementation of today's most used tokenizers in Rust
Provides an implementation of today's most used tokenizers, with a focus on performances and versatility. This package contains library source intended for building other packages which use the "hf-hub" feature of the "tokenizers" crate.
Releases Overview
| Release | Stable | Testing |
|---|---|---|
| Fedora Rawhide | 0.22.2-5.fc45 | - |
| Fedora 44 | 0.22.2-5.fc44 | - |
| Fedora 43 | 0.22.2-1.fc43 | 0.22.2-4.fc43 |
| Fedora 42 | 0.22.2-1.fc42 | 0.22.2-4.fc42 |
File a new bug report »
Package Info
Related Packages
Package Info
- Upstream: https://crates.io/crates/tokenizers
- License(s): Apache-2.0
- Maintainer: xanderlent
Related Packages
- rust-tokenizers+default-devel
- rust-tokenizers+esaxx_fast-devel
- rust-tokenizers+fancy-regex-devel
- rust-tokenizers+http-devel
- rust-tokenizers+indicatif-devel
- View More »
You can contact the maintainers of this package via email at
rust-tokenizers dash maintainers at fedoraproject dot org.
Sources on Pagure