Package net.sourceforge.pmd.cpd.impl

Class AntlrTokenizer

java.lang.Object

net.sourceforge.pmd.cpd.impl.AntlrTokenizer

All Implemented Interfaces:

Tokenizer

public abstract class AntlrTokenizer
extends Object
implements Tokenizer

Generic implementation of a Tokenizer useful to any Antlr grammar.

Field Summary

Fields inherited from interface net.sourceforge.pmd.cpd.Tokenizer

DEFAULT_SKIP_BLOCKS_PATTERN, IGNORE_ANNOTATIONS, IGNORE_IDENTIFIERS, IGNORE_LITERALS, IGNORE_USINGS, OPTION_IGNORE_LITERAL_SEQUENCES, OPTION_SKIP_BLOCKS, OPTION_SKIP_BLOCKS_PATTERN

Constructor Summary

Constructors

Constructor	Description
AntlrTokenizer()

Method Summary

Modifier and Type	Method	Description
protected abstract org.antlr.v4.runtime.Lexer	getLexerForSource(org.antlr.v4.runtime.CharStream charStream)
protected AntlrTokenFilter	getTokenFilter(AntlrTokenManager tokenManager)
void	tokenize(SourceCode sourceCode, Tokens tokenEntries)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AntlrTokenizer

public AntlrTokenizer()

Method Detail

getLexerForSource

protected abstract org.antlr.v4.runtime.Lexer getLexerForSource(org.antlr.v4.runtime.CharStream charStream)

tokenize

public void tokenize(SourceCode sourceCode,
                     Tokens tokenEntries)

Specified by:

tokenize in interface Tokenizer

getTokenFilter

protected AntlrTokenFilter getTokenFilter(AntlrTokenManager tokenManager)