Class AnyTokenizer

  • All Implemented Interfaces:

    public class AnyTokenizer
    extends Object
    implements Tokenizer
    Simple tokenization into words and separators. Can ignore end-of-line comments and recognize double/single quoted string literals. It is not a goal to be very customizable, or have very high quality. Higher-quality lexers should be implemented with a lexer generator.

    In PMD 7, this replaces AbstractTokenizer, which provided nearly no more functionality.

    • Constructor Detail

      • AnyTokenizer

        public AnyTokenizer()
      • AnyTokenizer

        public AnyTokenizer​(String eolCommentStart)