Interface CpdCapableLanguage

    • Method Detail

      • createCpdTokenizer

        default Tokenizer createCpdTokenizer​(LanguagePropertyBundle bundle)
        Create a new Tokenizer for this language, given a property bundle with configuration. The bundle was created by this instance using Language.newPropertyBundle(). It can be assumed that the bundle will never be mutated anymore, and this method takes ownership of it.
        Parameters:
        bundle - A bundle of properties created by this instance.
        Returns:
        A new language processor