Package ai.platon.pulsar.skeleton.crawl
Interface Crawler
-
- All Implemented Interfaces:
-
java.lang.AutoCloseable
public interface Crawler implements AutoCloseable
-
-
Method Summary
Modifier and Type Method Description abstract Unitpause()abstract Unitresume()abstract Unitawait()Wait until all tasks are done. abstract Unitreport()abstract UnitonWillLoad(UrlAware url)abstract UnitonLoad(UrlAware url)abstract UnitonLoaded(UrlAware url, WebPage page)abstract IntegergetId()The crawl id abstract StringgetName()The crawl name abstract Function2<Integer, UrlAware, Duration>getRetryDelayPolicy()Delay policy for retry tasks abstract UnitsetRetryDelayPolicy(Function2<Integer, UrlAware, Duration> retryDelayPolicy)Delay policy for retry tasks -
-
Method Detail
-
onWillLoad
abstract Unit onWillLoad(UrlAware url)
-
getRetryDelayPolicy
abstract Function2<Integer, UrlAware, Duration> getRetryDelayPolicy()
Delay policy for retry tasks
-
setRetryDelayPolicy
abstract Unit setRetryDelayPolicy(Function2<Integer, UrlAware, Duration> retryDelayPolicy)
Delay policy for retry tasks
-
-
-
-