Package ai.platon.pulsar.parse.js

Class JSParseFilter

  • All Implemented Interfaces:
    ai.platon.pulsar.common.config.Parameterized , ai.platon.pulsar.crawl.parse.ParseFilter , ai.platon.pulsar.crawl.parse.Parser , java.lang.AutoCloseable 
    
public final class JSParseFilter
extends AbstractParseFilter implements Parser

    This class is a heuristic link extractor for JavaScript files and code snippets. The general idea of a two-pass regex matching comes from Heritrix. Parts of the code come from OutlinkExtractor.java by Stephan Strittmatter.

    Author:

    Andrzej Bialecki <ab@getopt.org>

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description

    • Constructor Summary

      Constructors 
      Constructor Description
      JSParseFilter(ImmutableConfig conf)

    • Enum Constant Summary

      Enum Constants 
      Enum Constant Description

    • Method Summary

      Modifier and Type Method Description
      Duration getTimeout()
      final ImmutableConfig getConf()
      ParseResult parse(WebPage page)

      • Methods inherited from class ai.platon.pulsar.crawl.parse.AbstractParseFilter

        addFirst, addLast, close, filter, getChildren, getId, getParent, initialize, isRelevant, onAfterFilter, onBeforeFilter, setParent

      • Methods inherited from class ai.platon.pulsar.crawl.parse.ParseFilter

        getParentId, isLeaf, isRoot

      • Methods inherited from class ai.platon.pulsar.common.config.Parameterized

        getParams

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • JSParseFilter

        JSParseFilter(ImmutableConfig conf)