Package net.sourceforge.pmd.cpd

Interface Tokenizer

All Known Implementing Classes:

AbstractTokenizer, AntlrTokenizer, AnyTokenizer, JavaCCTokenizer

public interface Tokenizer

Field Summary

Fields

Modifier and Type	Field	Description
static String	DEFAULT_SKIP_BLOCKS_PATTERN
static String	IGNORE_ANNOTATIONS
static String	IGNORE_IDENTIFIERS
static String	IGNORE_LITERALS
static String	IGNORE_USINGS	Ignore using directives in C#.
static String	OPTION_IGNORE_LITERAL_SEQUENCES	Ignore sequences of literals (e.g, 0,0,0,0...).
static String	OPTION_SKIP_BLOCKS	Enables or disabled skipping of blocks like a pre-processor.
static String	OPTION_SKIP_BLOCKS_PATTERN	Configures the pattern, to find the blocks to skip.

Method Summary

Modifier and Type	Method	Description
void	tokenize(SourceCode sourceCode, Tokens tokenEntries)

Field Detail

IGNORE_LITERALS

static final String IGNORE_LITERALS

See Also:

Constant Field Values

IGNORE_IDENTIFIERS

static final String IGNORE_IDENTIFIERS

See Also:

Constant Field Values

IGNORE_ANNOTATIONS

static final String IGNORE_ANNOTATIONS

See Also:

Constant Field Values

OPTION_IGNORE_LITERAL_SEQUENCES

static final String OPTION_IGNORE_LITERAL_SEQUENCES

Ignore sequences of literals (e.g, 0,0,0,0...).

See Also:

Constant Field Values

IGNORE_USINGS

static final String IGNORE_USINGS

Ignore using directives in C#. The default value is false.

See Also:

Constant Field Values

OPTION_SKIP_BLOCKS

static final String OPTION_SKIP_BLOCKS

Enables or disabled skipping of blocks like a pre-processor. It is a boolean property. The default value is true.

See Also:

OPTION_SKIP_BLOCKS_PATTERN, Constant Field Values

OPTION_SKIP_BLOCKS_PATTERN

static final String OPTION_SKIP_BLOCKS_PATTERN

Configures the pattern, to find the blocks to skip. It is a string property and contains of two parts, separated by |. The first part is the start pattern, the second part is the ending pattern. Default value is "#if 0|#endif".

See Also:

DEFAULT_SKIP_BLOCKS_PATTERN, Constant Field Values

DEFAULT_SKIP_BLOCKS_PATTERN

static final String DEFAULT_SKIP_BLOCKS_PATTERN

See Also:

Constant Field Values

Method Detail

tokenize

void tokenize(SourceCode sourceCode,
              Tokens tokenEntries)
       throws IOException

Throws:

IOException