Package ai.platon.pulsar.crawl.protocol.http

Class AbstractNativeHttpProtocol

  • All Implemented Interfaces:
    ai.platon.pulsar.common.config.Configurable , ai.platon.pulsar.crawl.protocol.Protocol , java.lang.AutoCloseable 
    
public abstract class AbstractNativeHttpProtocol
extends AbstractHttpProtocol

    TODO: just use Jsoup

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description

    • Field Summary

      Fields 
      Modifier and Type Field Description
      public final static int BUFFER_SIZE
      public final static AbstractHttpProtocol.Companion Companion
      public final boolean supportParallel
      public ImmutableConfig conf

    • Enum Constant Summary

      Enum Constants 
      Enum Constant Description

    • Method Summary

      Modifier and Type Method Description
      void setConf(ImmutableConfig jobConf)
      Array<byte> processGzipEncoded(Array<byte> compressed, URL url)
      Array<byte> processDeflateEncoded(Array<byte> compressed, URL url)
      abstract Response getResponse(@NotNull() String url, @NotNull() WebPage page, boolean followRedirects)

      • Methods inherited from class ai.platon.pulsar.crawl.protocol.http.AbstractHttpProtocol

        close, getConf, getProtocolOutput, getProtocolOutputDeferred, getResponse, getResponseDeferred, getResponses, getRobotRules, getSupportParallel, isActive, reset, setConf, toString

      • Methods inherited from class ai.platon.pulsar.crawl.protocol.Protocol

        cancel, cancelAll, setResponse

      • Methods inherited from class java.lang.AutoCloseable

        close

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait