Interface Protocol
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.config.Configurable,ai.platon.pulsar.skeleton.crawl.common.LazyConfigurable,java.lang.AutoCloseable
public interface Protocol implements LazyConfigurable, AutoCloseable
A retriever for url content. Implemented by protocol extensions.
-
-
Method Summary
Modifier and Type Method Description UnitsetResponse(Response response)Collection<Response>getResponses(Collection<WebPage> pages, VolatileConfig volatileConfig)Unitreset()Reset the protocol environment, so the peer host view the client as a new one Unitcancel(WebPage page)Cancel the page UnitcancelAll()Cancel all fetching tasks abstract ProtocolOutputgetProtocolOutput(WebPage page)Returns the ProtocolOutput for a fetch list entry. abstract ProtocolOutputgetProtocolOutputDeferred(WebPage page)Returns the ProtocolOutput for a fetch list entry. abstract BaseRobotRulesgetRobotRules(WebPage page)Retrieve robot rules applicable for this url. Unitclose()abstract BooleangetSupportParallel()-
Methods inherited from class ai.platon.pulsar.skeleton.crawl.common.LazyConfigurable
configure -
Methods inherited from class ai.platon.pulsar.common.config.Configurable
getConf, setConf -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Method Detail
-
setResponse
Unit setResponse(Response response)
-
getResponses
Collection<Response> getResponses(Collection<WebPage> pages, VolatileConfig volatileConfig)
-
getProtocolOutput
abstract ProtocolOutput getProtocolOutput(WebPage page)
Returns the ProtocolOutput for a fetch list entry.
-
getProtocolOutputDeferred
abstract ProtocolOutput getProtocolOutputDeferred(WebPage page)
Returns the ProtocolOutput for a fetch list entry.
-
getRobotRules
abstract BaseRobotRules getRobotRules(WebPage page)
Retrieve robot rules applicable for this url.
- Parameters:
page- The Web page- Returns:
robot rules (specific for this url or default), never null
-
getSupportParallel
abstract Boolean getSupportParallel()
-
-
-
-