Class Tokenizer.Token

  • Enclosing interface:
    Tokenizer

    public static class Tokenizer.Token
    extends Object
    An internal token
    • Field Detail

      • text

        public final String text
        The Text.
      • charStartIndex

        public final int charStartIndex
        The Char start index.
      • charEndIndex

        public final int charEndIndex
        The Char end index.
      • index

        public int index
        The Index.
    • Constructor Detail

      • Token

        public Token​(String text,
                     TokenType type,
                     int startChar,
                     int endChar,
                     int index)
        Default constructor
        Parameters:
        text - The text covered by the token
        type - The type of token
        startChar - The first character offset
        endChar - The last character offset
        index - The token index