Class PageParser
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.config.Configurable,ai.platon.pulsar.common.config.Parameterized,ai.platon.pulsar.skeleton.crawl.common.LazyConfigurable,java.lang.AutoCloseable
public final class PageParser implements Parameterized, LazyConfigurable, AutoCloseable
-
-
Field Summary
Fields Modifier and Type Field Description private final ConcurrentSkipListSet<CharSequence>unparsableTypesprivate final ParserFactoryparserFactoryprivate ImmutableConfigconfprivate final MiscMessageWritermessageWriter
-
Constructor Summary
Constructors Constructor Description PageParser(ParserFactory parserFactory, ImmutableConfig conf)PageParser(ImmutableConfig conf)PageParser(ParserFactory parserFactory, ImmutableConfig conf, MiscMessageWriter messageWriter)
-
Method Summary
Modifier and Type Method Description final ConcurrentSkipListSet<CharSequence>getUnparsableTypes()final ParserFactorygetParserFactory()ImmutableConfiggetConf()UnitsetConf(ImmutableConfig conf)final MiscMessageWritergetMessageWriter()Unitconfigure(ImmutableConfig conf1)final ParseResultparse(WebPage page)Parses given web page and stores parsed content within page. Unitclose()-
-
Constructor Detail
-
PageParser
PageParser(ParserFactory parserFactory, ImmutableConfig conf)
-
PageParser
PageParser(ImmutableConfig conf)
- Parameters:
conf- The configuration
-
PageParser
PageParser(ParserFactory parserFactory, ImmutableConfig conf, MiscMessageWriter messageWriter)
-
-
Method Detail
-
getUnparsableTypes
final ConcurrentSkipListSet<CharSequence> getUnparsableTypes()
-
getParserFactory
final ParserFactory getParserFactory()
-
getConf
ImmutableConfig getConf()
-
getMessageWriter
final MiscMessageWriter getMessageWriter()
-
parse
final ParseResult parse(WebPage page)
Parses given web page and stores parsed content within page. Puts a meta-redirect to outlinks.
- Parameters:
page- The web page- Returns:
ParseResult A non-null ParseResult contains the main result and status of this parse
-
-
-
-