Class PrimerHtmlParser

  • All Implemented Interfaces:
    ai.platon.pulsar.common.config.Parameterized , ai.platon.pulsar.crawl.parse.Parser

    
    public final class PrimerHtmlParser
     implements Parser
                        

    Primer HTML parser is a built-in parser while others should be a protocol-plugin. Primer HTML parser uses Jsoup to parse the page content to a HTML document. The parser can also register ai.platon.pulsar.crawl.parse.ParseFilters to extend the system.

    • Enum Constant Summary

      Enum Constants 
      Enum Constant Description
    • Method Summary

      Modifier and Type Method Description
      Duration getTimeout() Optimized for html content To parse non-html content, the parser might run into an endless loop, run it in a separate coroutine to protect the process
      Params getParams()
      ParseResult parse(WebPage page) This method parses content in WebPage instance
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Constructor Detail

      • PrimerHtmlParser

        PrimerHtmlParser(ImmutableConfig conf)
      • PrimerHtmlParser

        PrimerHtmlParser(ParseFilters parseFilters, ImmutableConfig conf)
    • Method Detail

      • getTimeout

         Duration getTimeout()

        Optimized for html content To parse non-html content, the parser might run into an endless loop, run it in a separate coroutine to protect the process

      • parse

         ParseResult parse(WebPage page)

        This method parses content in WebPage instance