tinker_cookbook.tokenizer_utils.SupportsTmlTokenizer

class tinker_cookbook.tokenizer_utils.SupportsTmlTokenizer(Protocol)

Tokenizer facade that exposes an underlying TmlTokenizer.

Fields:

• tml_tokenizer (TmlTokenizer)