Package net.sourceforge.pmd.cpd.impl

Class JavaCCTokenizer

java.lang.Object

net.sourceforge.pmd.cpd.impl.JavaCCTokenizer

All Implemented Interfaces:

Tokenizer

public abstract class JavaCCTokenizer
extends Object
implements Tokenizer

Field Summary

Fields inherited from interface net.sourceforge.pmd.cpd.Tokenizer

DEFAULT_SKIP_BLOCKS_PATTERN, IGNORE_ANNOTATIONS, IGNORE_IDENTIFIERS, IGNORE_LITERALS, IGNORE_USINGS, OPTION_IGNORE_LITERAL_SEQUENCES, OPTION_SKIP_BLOCKS, OPTION_SKIP_BLOCKS_PATTERN

Constructor Summary

Constructors

Constructor	Description
JavaCCTokenizer()

Method Summary

Modifier and Type	Method	Description
protected String	getImage(JavaccToken token)
protected TokenManager<JavaccToken>	getLexerForSource(TextDocument sourceCode)
protected TokenFilter<JavaccToken>	getTokenFilter(TokenManager<JavaccToken> tokenManager)
protected abstract TokenManager<JavaccToken>	makeLexerImpl(CharStream sourceCode)
protected TokenEntry	processToken(Tokens tokenEntries, JavaccToken currentToken)
protected JavaccTokenDocument.TokenDocumentBehavior	tokenBehavior()
void	tokenize(SourceCode sourceCode, Tokens tokenEntries)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JavaCCTokenizer

public JavaCCTokenizer()

Method Detail

getLexerForSource

protected TokenManager<JavaccToken> getLexerForSource(TextDocument sourceCode)
                                               throws IOException

Throws:

IOException

tokenBehavior

protected JavaccTokenDocument.TokenDocumentBehavior tokenBehavior()

makeLexerImpl

protected abstract TokenManager<JavaccToken> makeLexerImpl(CharStream sourceCode)

getTokenFilter

protected TokenFilter<JavaccToken> getTokenFilter(TokenManager<JavaccToken> tokenManager)

processToken

protected TokenEntry processToken(Tokens tokenEntries,
                                  JavaccToken currentToken)

getImage

protected String getImage(JavaccToken token)

tokenize

public void tokenize(SourceCode sourceCode,
                     Tokens tokenEntries)
              throws IOException

Specified by:

tokenize in interface Tokenizer

Throws:

IOException