Package com.gengoai.hermes.morphology
Class Tokenizer.Token
- java.lang.Object
-
- com.gengoai.hermes.morphology.Tokenizer.Token
-
-
Field Summary
Fields Modifier and Type Field Description int
charEndIndex
The Char end index.int
charStartIndex
The Char start index.int
index
The Index.Map<AttributeType<?>,Object>
properties
The Properties.String
text
The Text.TokenType
type
The Type.
-
-
-
Field Detail
-
text
public final String text
The Text.
-
type
public TokenType type
The Type.
-
charStartIndex
public final int charStartIndex
The Char start index.
-
charEndIndex
public final int charEndIndex
The Char end index.
-
index
public int index
The Index.
-
properties
public final Map<AttributeType<?>,Object> properties
The Properties.
-
-