Package ai.platon.pulsar.crawl.impl
Class AbstractCrawlLoop
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.StartStopRunnable,ai.platon.pulsar.crawl.CrawlLoop
public abstract class AbstractCrawlLoop implements CrawlLoop
-
-
Field Summary
Fields Modifier and Type Field Description private final Integeridprivate final UrlFeederurlFeederprivate final List<PriorityDataCollector<UrlAware>>collectorsprivate final Crawlercrawlerprivate final Stringabstractprivate final Stringreportprivate final Stringnameprivate final ImmutableConfigconfig
-
Constructor Summary
Constructors Constructor Description AbstractCrawlLoop(String name, ImmutableConfig config)
-
Method Summary
Modifier and Type Method Description IntegergetId()abstract UrlFeedergetUrlFeeder()The fetch iterable from which all fetch tasks are taken List<PriorityDataCollector<UrlAware>>getCollectors()abstract CrawlergetCrawler()StringgetAbstract()StringgetReport()StringgetName()ImmutableConfiggetConfig()-
-
Constructor Detail
-
AbstractCrawlLoop
AbstractCrawlLoop(String name, ImmutableConfig config)
-
-
Method Detail
-
getUrlFeeder
abstract UrlFeeder getUrlFeeder()
The fetch iterable from which all fetch tasks are taken
-
getCollectors
List<PriorityDataCollector<UrlAware>> getCollectors()
-
getCrawler
abstract Crawler getCrawler()
-
getAbstract
String getAbstract()
-
getConfig
ImmutableConfig getConfig()
-
-
-
-