Class JavaccToken
- java.lang.Object
-
- net.sourceforge.pmd.lang.ast.impl.javacc.JavaccToken
-
- All Implemented Interfaces:
Comparable<JavaccToken>
,GenericToken<JavaccToken>
,Reportable
public class JavaccToken extends Object implements GenericToken<JavaccToken>
A generic token implementation for JavaCC parsers.Largely has the same interface as the default generated token class. The main difference is that the position of the token is encoded as a start and end offset in the source file, instead of a (begin,end)x(line,column) 4-tuple. This offers two practical advantages:
- It allows retrieving easily the underlying text of a node (just need to cut a substring of the file text). Other attributes like lines and column bounds can be derived as well - though this should not be done systematically because it's costlier.
- It's a bit lighter. Token instances are one of the most numerous class in a typical PMD run and this may reduce GC pressure.
-
-
Field Summary
Fields Modifier and Type Field Description static int
EOF
Kind for EOF tokens.static int
IMPLICIT_TOKEN
Kind for implicit tokens.int
kind
An integer that describes the kind of this token.JavaccToken
next
A reference to the next regular (non-special) token from the input stream.JavaccToken
specialToken
This field is used to access special tokens that occur prior to this token, but after the immediately preceding regular (non-special) token.
-
Constructor Summary
Constructors Constructor Description JavaccToken(int kind, String image, int startInclusive, int endExclusive, JavaccTokenDocument document)
Constructor with aString
image (seethe other ctor
).JavaccToken(int kind, Chars image, int startInclusive, int endExclusive, JavaccTokenDocument document)
Builds a new token of the specified kind.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description JavaccTokenDocument
getDocument()
Returns the document owning this token.String
getImage()
Returns the token's text as a string.Chars
getImageCs()
Returns the text of the token as a char sequence.int
getKind()
Gets a unique integer representing the kind of token this is.JavaccToken
getNext()
Obtain the next generic token according to the input stream which generated the instance of this token.JavaccToken
getPreviousComment()
Obtain a comment-type token which, according to the input stream which generated the instance of this token, precedes this instance token and succeeds the previous generic token (if there is any).TextRegion
getRegion()
Returns a text region with the coordinates of this token.FileLocation
getReportLocation()
Returns the location at which this element should be reported.static JavaccToken
implicitBefore(JavaccToken next)
Creates an implicit token, with zero length, that is linked to the given token as its special predecessor.boolean
isEof()
Returns true if this token is an end-of-file token.boolean
isImplicit()
Returns true if this token is implicit, ie was inserted artificially and has a zero-length image.static JavaccToken
newImplicit(int offset, JavaccTokenDocument document)
Returns a new implicit token, positioned at the given offset.JavaccToken
replaceImage(CharStream charStream)
Returns a new token with the same kind as this one, whose image is replaced by the one marked on the char stream.String
toString()
JavaccToken
withKind(int newKind)
Returns a new token with the given kind, and all other parameters identical to this one.-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface net.sourceforge.pmd.lang.ast.GenericToken
compareTo, imageEquals
-
-
-
-
Field Detail
-
EOF
public static final int EOF
Kind for EOF tokens.- See Also:
- Constant Field Values
-
IMPLICIT_TOKEN
public static final int IMPLICIT_TOKEN
Kind for implicit tokens. Negative because JavaCC only picks positive numbers for token kinds.- See Also:
- Constant Field Values
-
kind
public final int kind
An integer that describes the kind of this token. This numbering system is determined by JavaCCParser, and a table of these numbers is stored in the file ...Constants.java.
-
next
public JavaccToken next
A reference to the next regular (non-special) token from the input stream. If this is the last token from the input stream, or if the token manager has not read tokens beyond this one, this field is set to null. This is true only if this token is also a regular token. Otherwise, see below for a description of the contents of this field.
-
specialToken
public JavaccToken specialToken
This field is used to access special tokens that occur prior to this token, but after the immediately preceding regular (non-special) token. If there are no such special tokens, this field is set to null. When there are more than one such special token, this field refers to the last of these special tokens, which in turn refers to the next previous special token through its specialToken field, and so on until the first special token (whose specialToken field is null). The next fields of special tokens refer to other special tokens that immediately follow it (without an intervening regular token). If there is no such token, this field is null.
-
-
Constructor Detail
-
JavaccToken
public JavaccToken(int kind, Chars image, int startInclusive, int endExclusive, JavaccTokenDocument document)
Builds a new token of the specified kind.- Parameters:
kind
- Kind of tokenimage
- Image of the token (after translating escapes if any)startInclusive
- Start character of the token in the text file (before translating escapes)endExclusive
- End of the token in the text file (before translating escapes)document
- Document owning the token
-
JavaccToken
public JavaccToken(int kind, String image, int startInclusive, int endExclusive, JavaccTokenDocument document)
Constructor with aString
image (seethe other ctor
).
-
-
Method Detail
-
getDocument
public final JavaccTokenDocument getDocument()
Returns the document owning this token.
-
isEof
public boolean isEof()
Description copied from interface:GenericToken
Returns true if this token is an end-of-file token. This is the last token of token sequences that have been fully lexed.- Specified by:
isEof
in interfaceGenericToken<JavaccToken>
-
getKind
public int getKind()
Description copied from interface:GenericToken
Gets a unique integer representing the kind of token this is. The semantics of this kind depend on the language.The returned constants can be looked up in the language's "*ParserConstants", e.g. CppParserConstants or JavaParserConstants. These constants are considered internal API and may change at any time when the language's grammar is changed.
- Specified by:
getKind
in interfaceGenericToken<JavaccToken>
-
getNext
public JavaccToken getNext()
Description copied from interface:GenericToken
Obtain the next generic token according to the input stream which generated the instance of this token.- Specified by:
getNext
in interfaceGenericToken<JavaccToken>
- Returns:
- the next generic token if it exists; null if it does not exist
-
getPreviousComment
public JavaccToken getPreviousComment()
Description copied from interface:GenericToken
Obtain a comment-type token which, according to the input stream which generated the instance of this token, precedes this instance token and succeeds the previous generic token (if there is any).- Specified by:
getPreviousComment
in interfaceGenericToken<JavaccToken>
- Returns:
- the comment-type token if it exists; null if it does not exist
-
getImageCs
public Chars getImageCs()
Description copied from interface:GenericToken
Returns the text of the token as a char sequence. This should be preferred when you can use egStringUtils
to do some processing, without having to create a string.- Specified by:
getImageCs
in interfaceGenericToken<JavaccToken>
-
getImage
public String getImage()
Description copied from interface:GenericToken
Returns the token's text as a string.- Specified by:
getImage
in interfaceGenericToken<JavaccToken>
-
getRegion
public final TextRegion getRegion()
Description copied from interface:GenericToken
Returns a text region with the coordinates of this token.- Specified by:
getRegion
in interfaceGenericToken<JavaccToken>
-
getReportLocation
public FileLocation getReportLocation()
Description copied from interface:Reportable
Returns the location at which this element should be reported.Use this instead of
Node.getBeginColumn()
/Node.getBeginLine()
, etc.- Specified by:
getReportLocation
in interfaceReportable
-
isImplicit
public boolean isImplicit()
Description copied from interface:GenericToken
Returns true if this token is implicit, ie was inserted artificially and has a zero-length image.- Specified by:
isImplicit
in interfaceGenericToken<JavaccToken>
-
replaceImage
public JavaccToken replaceImage(CharStream charStream)
Returns a new token with the same kind as this one, whose image is replaced by the one marked on the char stream.- Parameters:
charStream
- Char stream from which to start- Returns:
- A new token
-
withKind
public JavaccToken withKind(int newKind)
Returns a new token with the given kind, and all other parameters identical to this one.- Parameters:
newKind
- Char stream from which to start- Returns:
- A new token
-
implicitBefore
public static JavaccToken implicitBefore(JavaccToken next)
Creates an implicit token, with zero length, that is linked to the given token as its special predecessor.- Parameters:
next
- Token before which to insert the new token- Returns:
- A new token
-
newImplicit
public static JavaccToken newImplicit(int offset, JavaccTokenDocument document)
Returns a new implicit token, positioned at the given offset.- Parameters:
offset
- Offset of the tokendocument
- Document owning the token- Returns:
- A new token
-
-