Tokenizer (:py:class:`~lingpy.sequence.tokenizer.Tokenizer`)
============================================================

.. currentmodule:: lingpy.sequence.tokenizer

.. autoclass:: Tokenizer

   
   .. automethod:: __init__

   
   .. rubric:: Methods

   .. autosummary::
   
      ~Tokenizer.characters
      ~Tokenizer.combine_modifiers
      ~Tokenizer.exists_multiple_columns
      ~Tokenizer.find_missing_characters
      ~Tokenizer.grapheme_clusters
      ~Tokenizer.graphemes
      ~Tokenizer.remove_spaces
      ~Tokenizer.rules
      ~Tokenizer.tokenize
      ~Tokenizer.tokenize_ipa
      ~Tokenizer.transform
      ~Tokenizer.transform_rules