Interface GenericToken<T extends GenericToken<T>>

  • All Known Implementing Classes:
    AntlrToken, JavaccToken

    public interface GenericToken<T extends GenericToken<T>>
    Represents a language-independent token such as constants, values language reserved keywords, or comments.
    • Method Summary

      All Methods Static Methods Instance Methods Abstract Methods Default Methods 
      Modifier and Type Method Description
      int getBeginColumn()
      Gets the column offset from the start of the begin line where the token's region begins
      int getBeginLine()
      Gets the line where the token's region begins
      int getEndColumn()
      Gets the column offset from the start of the end line where the token's region ends
      default int getEndInDocument()
      Exclusive end offset in the source file text.
      int getEndLine()
      Gets the line where the token's region ends
      String getImage()
      Returns the token's text.
      int getKind()
      Gets a unique integer representing the kind of token this is.
      T getNext()
      Obtain the next generic token according to the input stream which generated the instance of this token.
      T getPreviousComment()
      Obtain a comment-type token which, according to the input stream which generated the instance of this token, precedes this instance token and succeeds the previous generic token (if there is any).
      default int getStartInDocument()
      Inclusive start offset in the source file text.
      boolean isEof()
      Returns true if this token is an end-of-file token.
      default boolean isImplicit()
      Returns true if this token is implicit, ie was inserted artificially and has a zero-length image.
      static Iterable<JavaccToken> previousSpecials​(JavaccToken from)
      Returns an iterable that enumerates all special tokens belonging to the given token.
      static Iterator<JavaccToken> range​(JavaccToken from, JavaccToken to)
      Returns an iterator that enumerates all (non-special) tokens between the two tokens (bounds included).
    • Method Detail

      • getNext

        T getNext()
        Obtain the next generic token according to the input stream which generated the instance of this token.
        Returns:
        the next generic token if it exists; null if it does not exist
      • getPreviousComment

        T getPreviousComment()
        Obtain a comment-type token which, according to the input stream which generated the instance of this token, precedes this instance token and succeeds the previous generic token (if there is any).
        Returns:
        the comment-type token if it exists; null if it does not exist
      • getImage

        String getImage()
        Returns the token's text.
      • isEof

        boolean isEof()
        Returns true if this token is an end-of-file token. This is the last token of token sequences that have been fully lexed.
      • getStartInDocument

        default int getStartInDocument()
        Inclusive start offset in the source file text.
      • getEndInDocument

        default int getEndInDocument()
        Exclusive end offset in the source file text.
      • getBeginLine

        int getBeginLine()
        Gets the line where the token's region begins
        Returns:
        a non-negative integer containing the begin line
      • getEndLine

        int getEndLine()
        Gets the line where the token's region ends
        Returns:
        a non-negative integer containing the end line
      • getBeginColumn

        int getBeginColumn()
        Gets the column offset from the start of the begin line where the token's region begins
        Returns:
        a non-negative integer containing the begin column
      • getEndColumn

        int getEndColumn()
        Gets the column offset from the start of the end line where the token's region ends
        Returns:
        a non-negative integer containing the begin column
      • isImplicit

        default boolean isImplicit()
        Returns true if this token is implicit, ie was inserted artificially and has a zero-length image.
      • range

        static Iterator<JavaccToken> range​(JavaccToken from,
                                           JavaccToken to)
        Returns an iterator that enumerates all (non-special) tokens between the two tokens (bounds included).
        Parameters:
        from - First token to yield (inclusive)
        to - Last token to yield (inclusive)
        Returns:
        An iterator
        Throws:
        IllegalArgumentException - If the first token does not come before the other token
      • previousSpecials

        static Iterable<JavaccToken> previousSpecials​(JavaccToken from)
        Returns an iterable that enumerates all special tokens belonging to the given token.
        Parameters:
        from - Token from which to start, note that the returned iterable does not contain that token
        Returns:
        An iterator, possibly empty, not containing the parameter
        Throws:
        NullPointerException - If the parameter s null
      • getKind

        @Experimental
        int getKind()
        Gets a unique integer representing the kind of token this is. The semantics of this kind depend on the language.

        Note: This is an experimental API.

        The returned constants can be looked up in the language's "*ParserConstants", e.g. CppParserConstants or JavaParserConstants. These constants are considered internal API and may change at any time when the language's grammar is changed.