Package ai.platon.pulsar.common.collect
Class CircularHyperlinkCollector
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.collect.CrawlableFatLinkCollector,ai.platon.pulsar.common.collect.collector.DataCollector,ai.platon.pulsar.common.collect.collector.PriorityDataCollector,kotlin.Comparable
public class CircularHyperlinkCollector extends HyperlinkCollector
-
-
Field Summary
Fields Modifier and Type Field Description private Stringnameprivate final Integersizeprivate final IntegerestimatedSizeprivate UrlNormalizerPipelineurlNormalizerprivate final ConcurrentSkipListMap<String, CrawlableFatLink>fatLinksprivate final PulsarSessionsessionprivate final Queue<NormUrl>seedsprivate final Integercapacityprivate IntegercollectCountprivate final DurationcollectTimeprivate IntegercollectedCountprivate Stringcountryprivate final InstantcreateTimeprivate InstantdeadTimeprivate Stringdistrictprivate final IntegerestimatedExternalSizeprivate final IntegerexternalSizeprivate InstantfirstCollectTimeprivate final Integeridprivate final BooleanisDeadprivate final Set<String>labelsprivate Stringlangprivate InstantlastCollectedTimeprivate final Integerpriority
-
Constructor Summary
Constructors Constructor Description CircularHyperlinkCollector(PulsarSession session, NormUrl seed, Priority13 priority)CircularHyperlinkCollector(PulsarSession session, Queue<NormUrl> seeds, Priority13 priority)
-
Method Summary
Modifier and Type Method Description StringgetName()UnitsetName(String name)IntegergetSize()IntegergetEstimatedSize()IntegercollectTo(List<UrlAware> sink)Unitclear()-
Methods inherited from class ai.platon.pulsar.common.collect.HyperlinkCollector
dump, getFatLinks, getSeeds, getSession, getUrlNormalizer, hasMore, remove, remove, setUrlNormalizer -
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractPriorityDataCollector
getCapacity, getPriority, toString -
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractDataCollector
collectTo, collectTo, collectTo, getCollectCount, getCollectTime, getCollectedCount, getCountry, getCreateTime, getDeadTime, getDistrict, getEstimatedExternalSize, getExternalSize, getFirstCollectTime, getId, getLabels, getLang, getLastCollectedTime, setCollectCount, setCollectedCount, setCountry, setDeadTime, setDistrict, setFirstCollectTime, setLang, setLastCollectedTime -
Methods inherited from class ai.platon.pulsar.common.collect.collector.DataCollector
deepClear, isDead -
Methods inherited from class ai.platon.pulsar.common.collect.collector.PriorityDataCollector
compareTo -
Methods inherited from class ai.platon.pulsar.common.collect.CrawlableFatLinkCollector
removeAll, removeAll -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Constructor Detail
-
CircularHyperlinkCollector
CircularHyperlinkCollector(PulsarSession session, NormUrl seed, Priority13 priority)
-
CircularHyperlinkCollector
CircularHyperlinkCollector(PulsarSession session, Queue<NormUrl> seeds, Priority13 priority)
-
-
-
-