Class PrimerHtmlParser
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.config.Parameterized,ai.platon.pulsar.skeleton.crawl.parse.Parser
public final class PrimerHtmlParser implements Parser
Primer HTML parser is a built-in parser while others should be a protocol-plugin. Primer HTML parser uses Jsoup to parse the page content to a HTML document. The parser can also register ai.platon.pulsar.skeleton.crawl.parse.ParseFilters to extend the system.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classPrimerHtmlParser.Companion
-
Field Summary
Fields Modifier and Type Field Description private final Durationtimeoutpublic final static PrimerHtmlParser.CompanionCompanion
-
Constructor Summary
Constructors Constructor Description PrimerHtmlParser(ImmutableConfig conf)PrimerHtmlParser(ParseFilters parseFilters, ImmutableConfig conf)
-
Method Summary
Modifier and Type Method Description DurationgetTimeout()Optimized for html content To parse non-html content, the parser might run into an endless loop, run it in a separate coroutine to protect the process ParamsgetParams()ParseResultparse(WebPage page)This method parses content in WebPage instance -
-
Constructor Detail
-
PrimerHtmlParser
PrimerHtmlParser(ImmutableConfig conf)
-
PrimerHtmlParser
PrimerHtmlParser(ParseFilters parseFilters, ImmutableConfig conf)
-
-
Method Detail
-
getTimeout
Duration getTimeout()
Optimized for html content To parse non-html content, the parser might run into an endless loop, run it in a separate coroutine to protect the process
-
getParams
Params getParams()
-
parse
ParseResult parse(WebPage page)
This method parses content in WebPage instance
-
-
-
-