Package net.sourceforge.pmd.cpd

Class AnyTokenizer

java.lang.Object

net.sourceforge.pmd.cpd.AnyTokenizer

All Implemented Interfaces:

Tokenizer

public class AnyTokenizer
extends Object
implements Tokenizer

This class does a best-guess try-anything tokenization.

Author:

jheintz

Field Summary

Fields

Modifier and Type	Field	Description
static String	TOKENS

Fields inherited from interface net.sourceforge.pmd.cpd.Tokenizer

DEFAULT_SKIP_BLOCKS_PATTERN, IGNORE_ANNOTATIONS, IGNORE_IDENTIFIERS, IGNORE_LITERALS, IGNORE_USINGS, OPTION_SKIP_BLOCKS, OPTION_SKIP_BLOCKS_PATTERN

Constructor Summary

Constructors

Constructor	Description
AnyTokenizer()

Method Summary

Modifier and Type	Method	Description
void	tokenize(SourceCode sourceCode, Tokens tokenEntries)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TOKENS

public static final String TOKENS

See Also:

Constant Field Values

Constructor Detail

AnyTokenizer

public AnyTokenizer()

Method Detail

tokenize

public void tokenize(SourceCode sourceCode,
                     Tokens tokenEntries)

Specified by:

tokenize in interface Tokenizer