Package net.sourceforge.pmd.lang.ast

Interface GenericToken<T extends GenericToken<T>>

All Superinterfaces:
Comparable<T>, Reportable
All Known Implementing Classes:
AntlrToken, JavaccToken
public interface GenericToken<T extends GenericToken<T>> extends Comparable<T>, Reportable
Represents a token, part of a token chain in a source file. Tokens are the individual "words" of a programming language, such as literals, identifiers, keywords, or comments. Tokens are produced by a lexer and are used by a parser implementation to build an AST Node. Tokens should generally not be manipulated in rules directly as they have little to no semantic information.

  • Method Summary

    Modifier and Type
    Method
    Description
    default int
    compareTo(T o)
    This must return true if this token comes before the other token.
    default String
    getImage()
    Returns the token's text as a string.
    CharSequence
    getImageCs()
    Returns the text of the token as a char sequence.
    int
    getKind()
    Gets a unique integer representing the kind of token this is.
    T
    getNext()
    Obtain the next generic token according to the input stream which generated the instance of this token.
    T
    getPreviousComment()
    Obtain a comment-type token which, according to the input stream which generated the instance of this token, precedes this instance token and succeeds the previous generic token (if there is any).
    TextRegion
    getRegion()
    Returns a text region with the coordinates of this token.
    default boolean
    imageEquals(CharSequence charSeq)
    Returns true if the image of this token equals the given charsequence.
    boolean
    isEof()
    Returns true if this token is an end-of-file token.
    default boolean
    isImplicit()
    Returns true if this token is implicit, ie was inserted artificially and has a zero-length image.
    static <T extends GenericToken<T>>
    Iterable<T>
    previousSpecials(T from)
    Returns an iterable that enumerates all special tokens belonging to the given token.
    static <T extends GenericToken<T>>
    Iterable<T>
    range(T from, T to)
    Returns an iterator that enumerates all (non-special) tokens between the two tokens (bounds included).
    static <T extends GenericToken<T>>
    Stream<T>
    streamRange(T from, T to)
    Returns a stream corresponding to range(GenericToken, GenericToken).

    Methods inherited from interface net.sourceforge.pmd.reporting.Reportable

    getReportLocation

  • Method Details

    • getNext

      T getNext()
      Obtain the next generic token according to the input stream which generated the instance of this token.
      Returns:
      the next generic token if it exists; null if it does not exist

    • getPreviousComment

      T getPreviousComment()
      Obtain a comment-type token which, according to the input stream which generated the instance of this token, precedes this instance token and succeeds the previous generic token (if there is any).
      Returns:
      the comment-type token if it exists; null if it does not exist

    • getImage

      default String getImage()
      Returns the token's text as a string.

    • getImageCs

      CharSequence getImageCs()
      Returns the text of the token as a char sequence. This should be preferred when you can use eg StringUtils to do some processing, without having to create a string.

    • imageEquals

      default boolean imageEquals(CharSequence charSeq)
      Returns true if the image of this token equals the given charsequence. This does not create a string.
      Parameters:
      charSeq - A character sequence

    • getRegion

      TextRegion getRegion()
      Returns a text region with the coordinates of this token.

    • isEof

      boolean isEof()
      Returns true if this token is an end-of-file token. This is the last token of token sequences that have been fully lexed.

    • isImplicit

      default boolean isImplicit()
      Returns true if this token is implicit, ie was inserted artificially and has a zero-length image.

    • compareTo

      default int compareTo(T o)
      This must return true if this token comes before the other token. If they start at the same index, then the smaller token comes before the other.
      Specified by:
      compareTo in interface Comparable<T extends GenericToken<T>>

    • range

      static <T extends GenericToken<T>> Iterable<T> range(T from, T to)
      Returns an iterator that enumerates all (non-special) tokens between the two tokens (bounds included).
      Parameters:
      from - First token to yield (inclusive)
      to - Last token to yield (inclusive)
      Returns:
      An iterator
      Throws:
      IllegalArgumentException - If the first token does not come before the other token

    • streamRange

      static <T extends GenericToken<T>> Stream<T> streamRange(T from, T to)
      Returns a stream corresponding to range(GenericToken, GenericToken).

    • previousSpecials

      static <T extends GenericToken<T>> Iterable<T> previousSpecials(T from)
      Returns an iterable that enumerates all special tokens belonging to the given token.
      Parameters:
      from - Token from which to start, note that the returned iterable does not contain that token
      Returns:
      An iterator, possibly empty, not containing the parameter
      Throws:
      NullPointerException - If the parameter s null

    • getKind

      int getKind()
      Gets a unique integer representing the kind of token this is. The semantics of this kind depend on the language.

      The returned constants can be looked up in the language's "*ParserConstants", e.g. CppParserConstants or JavaParserConstants. These constants are considered internal API and may change at any time when the language's grammar is changed.