Skip to content

tinker_cookbook.tokenizer_utils.is_tokenizer_registered

tinker_cookbook.tokenizer_utils.is_tokenizer_registered(name)

Check if a tokenizer name is registered.

Parameters:

  • name (str) – The tokenizer name to check.

Returns: bool: True if the name has been registered via :func:register_tokenizer.