Package ai.platon.pulsar.crawl.impl
Class StreamingCrawlLoop
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.StartStopRunnable,ai.platon.pulsar.crawl.CrawlLoop
public class StreamingCrawlLoop extends AbstractCrawlLoop
-
-
Field Summary
Fields Modifier and Type Field Description private final BooleanisRunningprivate final UrlFeederurlFeederprivate final Crawlercrawlerprivate final Integeridprivate final List<PriorityDataCollector<UrlAware>>collectorsprivate final Stringabstractprivate final Stringreportprivate final Stringnameprivate final ImmutableConfigconfig
-
Constructor Summary
Constructors Constructor Description StreamingCrawlLoop(ImmutableConfig unmodifiedConfig, String name)
-
Method Summary
Modifier and Type Method Description final BooleanisRunning()UrlFeedergetUrlFeeder()The fetch iterable from which all fetch tasks are taken CrawlergetCrawler()Unitstart()Unitstop()Unitawait()Wait until the loop is started and all tasks are done. -
Methods inherited from class ai.platon.pulsar.crawl.impl.AbstractCrawlLoop
getAbstract, getCollectors, getConfig, getId, getName, getReport -
Methods inherited from class ai.platon.pulsar.common.StartStopRunnable
restart -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Constructor Detail
-
StreamingCrawlLoop
StreamingCrawlLoop(ImmutableConfig unmodifiedConfig, String name)
-
-
Method Detail
-
getUrlFeeder
UrlFeeder getUrlFeeder()
The fetch iterable from which all fetch tasks are taken
-
getCrawler
Crawler getCrawler()
-
start
@Synchronized() Unit start()
-
stop
@Synchronized() Unit stop()
-
-
-
-