Command-line tool

sentencepiece

brew/sentencepiece

Checked 25 Oct 2026, 21:09 UTC · Listing refreshed from Homebrew, winget, npm, Scoop and cargo; npm packages checked against OSV.dev

Unsupervised text tokenizer and detokenizer

Website Source

At a glance

Type

Command-line tool

Install on

macOS, Linux

Last checked

25 Oct 2026, 21:09 UTC

Install it

macOS

brew install sentencepiece

Linux

brew install sentencepiece

Related command-line tools

• brush-parser
  POSIX/bash shell tokenizer and parsers (used by brush-shell)
• jison
  A parser generator with Bison's API jison bison yacc parser generator lexer flex tokenizer compiler
• jison-lex
  lexical analyzer generator used by jison jison parser generator lexer flex tokenizer
• tokenx
  Fast token estimation with 95%+ average accuracy in a 2kB bundle ai cli llm token tokenizer estimation tiktoken…
• fileql
  Run SQL-like query on local files instead of database files using the GitQL SDK database sql files engine parser gitql…
• flex
  Fast Lexical Analyzer, generates Scanners (tokenizers)
• text-tables
  A terminal/text table prettifier with no dependencies
• cli-pdf-to-text
  A CLI pdf to plain text converter

Listing text is written by the maker; AptLynx lists tools and does not vouch for them. Find another tool · Browse by topic · For agents