Class AbstractTokenizer

  • All Implemented Interfaces:
    Tokenizer

    public abstract class AbstractTokenizer
    extends Object
    implements Tokenizer
    Author:
    Zev Blut zb@ubit.com, Romain PELISSE belaran@gmail.com
    • Field Detail

      • ignorableCharacter

        protected List<String> ignorableCharacter
      • ignorableStmt

        protected List<String> ignorableStmt
      • oneLineCommentChar

        protected char oneLineCommentChar
      • spanMultipleLinesString

        protected boolean spanMultipleLinesString
      • spanMultipleLinesLineContinuationCharacter

        protected Character spanMultipleLinesLineContinuationCharacter
    • Constructor Detail

      • AbstractTokenizer

        public AbstractTokenizer()