Class PeriodicalHyperlinkCollector
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.collect.CrawlableFatLinkCollector,ai.platon.pulsar.common.collect.collector.DataCollector,ai.platon.pulsar.common.collect.collector.PriorityDataCollector,kotlin.Comparable
public class PeriodicalHyperlinkCollector extends CircularHyperlinkCollector
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classPeriodicalHyperlinkCollector.Companion
-
Field Summary
Fields Modifier and Type Field Description private Stringnameprivate final Integersizeprivate final IntegerestimatedSizeprivate final NormURLseedprivate UrlNormalizerPipelineurlNormalizerprivate final ConcurrentSkipListMap<String, CrawlableFatLink>fatLinksprivate final PulsarSessionsessionprivate final Queue<NormURL>seedsprivate final Integercapacityprivate IntegercollectCountprivate final DurationcollectTimeprivate IntegercollectedCountprivate Stringcountryprivate final InstantcreateTimeprivate InstantdeadTimeprivate Stringdistrictprivate final IntegerestimatedExternalSizeprivate final IntegerexternalSizeprivate InstantfirstCollectTimeprivate final Integeridprivate final BooleanisDeadprivate final Set<String>labelsprivate Stringlangprivate InstantlastCollectedTimeprivate final Integerprioritypublic final static PeriodicalHyperlinkCollector.CompanionCompanion
-
Constructor Summary
Constructors Constructor Description PeriodicalHyperlinkCollector(PulsarSession session, NormURL seed, Priority13 priority)
-
Method Summary
Modifier and Type Method Description StringgetName()UnitsetName(String name)IntegergetSize()IntegergetEstimatedSize()final NormURLgetSeed()BooleanhasMore()IntegercollectTo(List<UrlAware> sink)-
Methods inherited from class ai.platon.pulsar.skeleton.common.collect.HyperlinkCollector
dump, getFatLinks, getSeeds, getSession, getUrlNormalizer, remove, remove, setUrlNormalizer -
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractPriorityDataCollector
getCapacity, getPriority, toString -
Methods inherited from class ai.platon.pulsar.common.collect.collector.AbstractDataCollector
collectTo, collectTo, collectTo, getCollectCount, getCollectTime, getCollectedCount, getCountry, getCreateTime, getDeadTime, getDistrict, getEstimatedExternalSize, getExternalSize, getFirstCollectTime, getId, getLabels, getLang, getLastCollectedTime, setCollectCount, setCollectedCount, setCountry, setDeadTime, setDistrict, setFirstCollectTime, setLang, setLastCollectedTime -
Methods inherited from class ai.platon.pulsar.common.collect.collector.DataCollector
deepClear, isDead -
Methods inherited from class ai.platon.pulsar.skeleton.common.collect.CircularHyperlinkCollector
clear -
Methods inherited from class ai.platon.pulsar.common.collect.collector.PriorityDataCollector
compareTo -
Methods inherited from class ai.platon.pulsar.common.collect.CrawlableFatLinkCollector
removeAll, removeAll -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Constructor Detail
-
PeriodicalHyperlinkCollector
PeriodicalHyperlinkCollector(PulsarSession session, NormURL seed, Priority13 priority)
-
-
-
-