S-index

A B C D E F G H I J K L M N O P Q R S T U V W _

S

sanitize(org.jsoup.nodes.Document,java.lang.Boolean) - function in ai.platon.pulsar.crawl.parse.html.JsoupUtils
scan(java.lang.String) - function in ai.platon.pulsar.context.PulsarContext: Scan webpages in the storage whose url start with urlPrefix
scan(java.lang.String,kotlin.Array) - function in ai.platon.pulsar.context.PulsarContext
scan(java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.context.PulsarContext
scan(java.lang.String) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Scan pages in the storage
scan(java.lang.String) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Scan pages in the storage
scan(java.lang.String,kotlin.Array) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Scan pages in the storage
scan(java.lang.String,kotlin.Array) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Scan pages in the storage
scan(java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Scan pages in the storage
scan(java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Scan pages in the storage
scan(java.lang.String,java.lang.String) - function in ai.platon.pulsar.crawl.component.WebDbComponent
schedule() - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler: Schedule a queue with the given priority and given poolId
schedule(ai.platon.pulsar.crawl.fetch.batch.data.PoolId,java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler: Null queue id means the queue with top priority Consume a fetch item and try to download the target web page
schedule(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler: Schedule the queues with top priority
scheduleOn(java.util.concurrent.ScheduledExecutorService) - function in ai.platon.pulsar.common.metrics.CodahaleSlf4jReporter.Builder: Specifies the executor to use while scheduling reporting of metrics.
schedulerIds() - function in ai.platon.pulsar.crawl.fetch.batch.TaskSchedulers
ScoreVectorWritable - class in ai.platon.pulsar.crawl.scoring.io: Created by vincent on 17-4-20.
ScoringFilter - class in ai.platon.pulsar.crawl.scoring: A contract defining behavior of scoring plugins.
ScoringFilter.Companion - class in ai.platon.pulsar.crawl.scoring.ScoringFilter
ScoringFilterException - class in ai.platon.pulsar.crawl.scoring: Specialized exception for errors during scoring.
ScoringFilters - class in ai.platon.pulsar.crawl.scoring
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrape(java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch a webpage located by the given url, and then extract fields specified by field selectors.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrapeOutPages(java.lang.String,java.lang.String,java.util.Map) - function in ai.platon.pulsar.session.PulsarSession: Load or fetch out pages specified by out link selector, and then extract fields specified by field selectors from each out page.
scrollDown(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver
scrollDown(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollDown(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollDownAsync(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollDownAsync(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollDownAsync(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
scrollTo(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollTo(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToAsync(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToAsync(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToAsync(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
scrollToBottom() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver
scrollToBottom() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToBottom() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToBottomAsync() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToBottomAsync() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToBottomAsync() - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
scrollToMiddle(java.lang.Float) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver
scrollToMiddle(java.lang.Float) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToMiddle(java.lang.Float) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToMiddleAsync(java.lang.Float) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToMiddleAsync(java.lang.Float) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToMiddleAsync(java.lang.Float) - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
scrollToTop() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver
scrollToTop() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToTop() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollToTopAsync() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToTopAsync() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollToTopAsync() - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
scrollUp(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver
scrollUp(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollUp(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
scrollUpAsync(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollUpAsync(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
scrollUpAsync(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
search(ai.platon.pulsar.crawl.fetch.batch.data.PoolId,java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.batch.data.PoolQueue
SeedBuilder - class in ai.platon.pulsar.crawl.inject: Created by vincent on 16-9-24.
SeedBuilder.Companion - class in ai.platon.pulsar.crawl.inject.SeedBuilder
SequentialPrivacyContextIdGenerator - class in ai.platon.pulsar.crawl.fetch.privacy
serialize() - function in ai.platon.pulsar.common.urls.AbstractUrl
ServiceUnavailableException - class in ai.platon.pulsar.common
SessionAware - class in ai.platon.pulsar.session
set(java.lang.String,java.lang.String) - function in ai.platon.pulsar.crawl.parse.html.OpenMapFields
setAcceptUnknownOptions(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setAcceptUnknownOptions(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setAdddays(java.lang.Long) - function in ai.platon.pulsar.common.options.GenerateOptions
setAliases(java.util.Map) - function in ai.platon.pulsar.crawl.parse.ParserConfig
setAllow(kotlin.collections.HashSet) - function in ai.platon.pulsar.crawl.filter.BlockFilter
setAllowLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setAllowParameterOverwriting(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setAllowParameterOverwriting(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setAnchorRegex(java.lang.String) - function in ai.platon.pulsar.common.options.LinkOptions
setArgs(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setArgs(java.lang.String) - function in ai.platon.pulsar.common.urls.ListenableUrl
setArgs(java.lang.String) - function in ai.platon.pulsar.common.urls.UrlAware
setArgs(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The url arguments
setArgs(java.util.Map) - function in ai.platon.pulsar.persist.ParseStatus
setAuthToken(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The auth token, used for authorization purpose.
setAuthToken(java.lang.String) - function in ai.platon.pulsar.common.urls.StatefulHyperlink
setAuthToken(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setAuthToken(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setBadSeeds(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setBaseHref(java.net.URL) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: Sets the baseHref.
setBatchId(java.lang.String) - function in ai.platon.pulsar.common.options.FetchOptions
setBatchId(java.lang.String) - function in ai.platon.pulsar.common.options.GenerateOptions
setBatchSize(java.lang.Integer) - function in ai.platon.pulsar.common.collect.PeriodicalLocalFileHyperlinkCollector
setBatchStat(ai.platon.pulsar.crawl.fetch.BatchStat) - function in ai.platon.pulsar.crawl.fetch.FetchTask
setBbsUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setBlogUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setBrowseEvent(ai.platon.pulsar.crawl.BrowseEvent) - function in ai.platon.pulsar.crawl.PageEvent: The browse phase event handler
setBrowseEvent(ai.platon.pulsar.crawl.BrowseEvent) - function in ai.platon.pulsar.crawl.event.AbstractPageEvent: The browse phase event handler
setBrowseEvent(ai.platon.pulsar.crawl.BrowseEvent) - function in ai.platon.pulsar.crawl.event.AbstractPageEvent: The browse phase event handler
setBrowser(ai.platon.pulsar.common.browser.BrowserType) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: The default browser is chrome with pulsar implemented web driver.
setBrowser(ai.platon.pulsar.common.browser.BrowserType) - function in ai.platon.pulsar.common.options.LoadOptions: Specify which browser to use, google chrome is the default.
setBytesThoRate(java.lang.Double) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler.Status
setClassName(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.JsEvaluation
setClickTarget(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The selector for element to click.
setClosed(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry: Indicate if the tab for this navigation is closed.
setCode(java.lang.Short,java.lang.Integer) - function in ai.platon.pulsar.persist.ParseStatus
setCollectCount(java.lang.Integer) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setCollected(java.lang.Integer) - function in ai.platon.pulsar.common.collect.PeriodicalLocalFileHyperlinkCollector.Companion.Counters
setCollectedCount(java.lang.Integer) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setCollects(java.lang.Integer) - function in ai.platon.pulsar.common.collect.PeriodicalLocalFileHyperlinkCollector.Companion.Counters
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.common.config.IndexingFilter
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.common.config.Protocol
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.protocol.RobotRulesParser: Set the Configuration object
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.protocol.RobotRulesParser: Set the Configuration object
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.protocol.http.AbstractHttpProtocol
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.protocol.http.AbstractHttpProtocol
setConf(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.protocol.http.AbstractNativeHttpProtocol
setContains(java.lang.String) - function in ai.platon.pulsar.crawl.filter.TextFilter
setContainsAny(java.lang.String) - function in ai.platon.pulsar.crawl.filter.TextFilter
setContainsNone(java.lang.String) - function in ai.platon.pulsar.crawl.filter.TextFilter
setContext(ai.platon.pulsar.context.PulsarContext) - function in ai.platon.pulsar.context.ContextAware
setContextClassLoader(java.lang.ClassLoader) - function in java.lang.Thread
setCountry(java.lang.String) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setCountry(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setCountry(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: Required website country
setCpuLoad(java.lang.Double) - function in ai.platon.pulsar.crawl.CoreMetrics.Companion
setCrawlEvent(ai.platon.pulsar.crawl.CrawlEvent) - function in ai.platon.pulsar.crawl.PageEvent: The crawl phase event handler
setCrawlEvent(ai.platon.pulsar.crawl.CrawlEvent) - function in ai.platon.pulsar.crawl.event.AbstractPageEvent: The crawl phase event handler
setCrawlEvent(ai.platon.pulsar.crawl.CrawlEvent) - function in ai.platon.pulsar.crawl.event.AbstractPageEvent: The crawl phase event handler
setCrawlId(java.lang.String) - function in ai.platon.pulsar.common.options.FetchOptions
setCrawlId(java.lang.String) - function in ai.platon.pulsar.common.options.GenerateOptions
setCrawlId(java.lang.String) - function in ai.platon.pulsar.common.options.InjectOptions
setCRITICAL_CPU_THRESHOLD(java.lang.Double) - function in ai.platon.pulsar.common.AppSystemInfo.Companion
setDaemon(java.lang.Boolean) - function in java.lang.Thread
setDeadline(java.time.Instant) - function in ai.platon.pulsar.common.options.LoadOptions: The deadline to finish the task, if the deadline is exceeded, the task should be discarded as soon as possible.
setDeadTime(java.time.Instant) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setDefaultCharEncoding(java.lang.String) - function in ai.platon.pulsar.common.EncodingDetector
setDELIMITER(java.lang.String) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setDepth(java.lang.Integer) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setDepth(java.lang.Integer) - function in ai.platon.pulsar.common.urls.Hyperlink
setDescription(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.JsEvaluation
setDetail(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.common.urls.NormUrl
setDetailUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setDisallow(kotlin.collections.HashSet) - function in ai.platon.pulsar.crawl.filter.BlockFilter
setDistrict(java.lang.String) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setDistrict(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setDistrict(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: Required website district
setDoc(ai.platon.pulsar.crawl.index.IndexDocument) - function in ai.platon.pulsar.crawl.index.io.IndexDocumentWritable
setDocument(ai.platon.pulsar.dom.FeaturedDocument) - function in ai.platon.pulsar.crawl.parse.ParseResult
setDocumentReadyTime(java.time.Instant) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry: The time when the document is ready.
setDropContent(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If the option is set, do not persist the page content which is usually very large.
setEnableReporter(java.lang.Boolean) - function in ai.platon.pulsar.crawl.CoreMetrics
setEntity(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The entity name of the page, for example, article, product, hotel, flower, etc, it's optional.
setEvent(ai.platon.pulsar.crawl.PageEvent) - function in ai.platon.pulsar.crawl.common.url.CompletableListenableHyperlink: The event handler
setEvent(ai.platon.pulsar.crawl.PageEvent) - function in ai.platon.pulsar.crawl.common.url.ListenableHyperlink: The event handler
setEvent(ai.platon.pulsar.crawl.PageEvent) - function in ai.platon.pulsar.crawl.common.url.ParsableHyperlink
setEvent(ai.platon.pulsar.crawl.PageEvent) - function in ai.platon.pulsar.crawl.common.url.StatefulListenableHyperlink: The event handler
setEventExceptionHandler(kotlin.jvm.functions.Function1) - function in ai.platon.pulsar.common.event.AbstractEventEmitter
setException(java.lang.Throwable) - function in ai.platon.pulsar.crawl.fetch.FetchResult
setException(ai.platon.pulsar.crawl.fetch.driver.JsException) - function in ai.platon.pulsar.crawl.fetch.driver.JsEvaluation
setExpandAtSign(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setExpandAtSign(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setExpireAt(java.time.Instant) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: The default time to expire
setExpireAt(java.time.Instant) - function in ai.platon.pulsar.common.options.LoadOptions: The expiry time point.
setExpiredLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setExpires(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: The default expiry time, some time we may need expire all pages by default, for example, in test mode
setExpires(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The expiry duration.
setFailed(java.lang.Integer,java.lang.String) - function in ai.platon.pulsar.persist.ParseStatus
setFeederCompleted() - function in ai.platon.pulsar.crawl.fetch.batch.TaskMonitor
setFetchInterval(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The fetch interval, used for periodically fetch tasks.
setFetchInterval(java.time.Duration) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setFetchLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setFetchMode(ai.platon.pulsar.persist.metadata.FetchMode) - function in ai.platon.pulsar.common.options.FetchOptions
setFetchMode(ai.platon.pulsar.persist.metadata.FetchMode) - function in ai.platon.pulsar.common.options.LoadOptions: The fetch mode.
setFetchPriority(java.lang.Integer) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setFetchSchedule(ai.platon.pulsar.persist.WebPage,ai.platon.pulsar.crawl.schedule.ModifyInfo) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchSchedule(ai.platon.pulsar.persist.WebPage,ai.platon.pulsar.crawl.schedule.ModifyInfo) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchSchedule(ai.platon.pulsar.persist.WebPage,ai.platon.pulsar.crawl.schedule.ModifyInfo) - function in ai.platon.pulsar.crawl.schedule.AdaptiveFetchSchedule: Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchSchedule(ai.platon.pulsar.persist.WebPage,ai.platon.pulsar.crawl.schedule.ModifyInfo) - function in ai.platon.pulsar.crawl.schedule.FetchSchedule: Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchSchedule(ai.platon.pulsar.persist.WebPage,ai.platon.pulsar.crawl.schedule.ModifyInfo) - function in ai.platon.pulsar.crawl.schedule.NewsFetchSchedule: Sets the fetchInterval and fetchTime on a successfully fetched page.
setFetchTime(java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.ModifyInfo: The actual latest fetch time, WebPage.
setFieldByAnnotation(ai.platon.pulsar.common.options.LoadOptions,java.lang.String,java.lang.Object) - function in ai.platon.pulsar.common.options.LoadOptions.Companion: Set the field value who has an annotation annotationName.
setFingerprint(ai.platon.pulsar.common.browser.Fingerprint) - function in ai.platon.pulsar.crawl.fetch.privacy.PrivacyAgent
setFirstCollectTime(java.time.Instant) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setFlowStatus(ai.platon.pulsar.common.FlowState) - function in ai.platon.pulsar.crawl.parse.FilterResult
setFlowStatus(ai.platon.pulsar.common.FlowState) - function in ai.platon.pulsar.crawl.parse.ParseResult
setFORBID_ALL_RULES(crawlercommons.robots.BaseRobotRules) - function in ai.platon.pulsar.crawl.protocol.RobotRulesParser.Companion: A BaseRobotRules object appropriate for use when the robots.txt file is not fetched due to a 403/Forbidden response; all requests are disallowed.
setFreshLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setGOOD_CONTENT_TEXT_LENGTH(java.lang.Integer) - function in ai.platon.pulsar.crawl.signature.TextProfileSignature.Companion
setHardRedirect(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If false, return the original page record but the redirect target's content, otherwise, return the page record of the redirected target.
setHelp(java.lang.Boolean) - function in ai.platon.pulsar.common.options.CommonOptions
setHelp(java.lang.Boolean) - function in ai.platon.pulsar.common.options.CommonOptions
setHelp(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setHelp(java.lang.Boolean) - function in ai.platon.pulsar.common.options.PulsarOptions
setHost(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask.Key
setHostName(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setHref(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setHref(java.lang.String) - function in ai.platon.pulsar.common.urls.ListenableUrl
setHref(java.net.URL) - function in ai.platon.pulsar.common.urls.NormUrl
setHref(java.lang.String) - function in ai.platon.pulsar.common.urls.UrlAware
setHref(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The hypertext reference, It defines the address of the document, which this time is linked from
setIdleTimeout(java.time.Duration) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: The idle timeout.
setIdleTimeout(java.time.Duration) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: The idle timeout.
setIdleTimeout(java.time.Duration) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: The idle timeout.
setIframe(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The iframe id to switch to.
setIGNORED_PAGE_FIELDS(java.util.Set) - function in ai.platon.pulsar.crawl.component.LoadComponent.Companion
setIgnoreFailure(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: If true, still fetch the page even if it is gone.
setIgnoreFailure(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: Retry fetching the page even if it's failed last time.
setIgnoreUrlQuery(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If true, remove the query parameters in the url.
setIncognito(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: Run browser in incognito mode.
setIndex(java.lang.Boolean) - function in ai.platon.pulsar.common.options.FetchOptions
setIndexed(java.lang.Boolean) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setIndexerCollection(java.lang.String) - function in ai.platon.pulsar.common.options.FetchOptions
setIndexerUrl(java.lang.String) - function in ai.platon.pulsar.common.options.FetchOptions
setIndexerUrl(java.lang.String) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setIndexingFilters(ai.platon.pulsar.crawl.index.IndexingFilters) - function in ai.platon.pulsar.crawl.component.IndexComponent
setIndexJIT(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler
setIndexThreadCount(java.lang.Integer) - function in ai.platon.pulsar.crawl.index.jit.indexer.JITIndexer
setIndexUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setIndexWriters(ai.platon.pulsar.crawl.index.IndexWriters) - function in ai.platon.pulsar.crawl.component.IndexComponent
setInitialDelay(java.time.Duration) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
setInterval(java.time.Duration) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
setItemBrowser(ai.platon.pulsar.common.browser.BrowserType) - function in ai.platon.pulsar.common.options.LoadOptions: The browser used to visit the item pages.
setItemExpireAt(java.time.Instant) - function in ai.platon.pulsar.common.options.LoadOptions: If an item page is expired, it should be fetched from the web again.
setItemExpires(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The same as expires, but only works for item pages.
setItemId(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask.Key
setItemPageLoadTimeout(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The same as pageLoadTimeout, but only works for item pages.
setItemRequireAnchors(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Re-fetch item pages whos anchors is less than requireAnchors.
setItemRequireImages(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Re-fetch item pages who's images is less than requireImages.
setItemRequireNotBlank(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: Re-fetch the item pages if the required text is blank.
setItemRequireSize(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Re-fetch item pages smaller than requireSize.
setItemScriptTimeout(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The same as scriptTimeout, but only works for item pages.
setItemScrollCount(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The same as scrollCount, but only works for item pages.
setItemScrollInterval(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The same as scrollInterval, but only works for item pages.
setJobID(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask.Key
setJobName(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.batch.FetchMonitor: Initialize in setup using job conf
setJobName(java.lang.String) - function in ai.platon.pulsar.crawl.impl.StreamingCrawler: The job name.
setKey(java.lang.String) - function in ai.platon.pulsar.crawl.index.IndexDocument
setKeywords(java.util.Map) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setLabel(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The task label, it's optional and can be used to group tasks.
setLang(java.lang.String) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setLang(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setLang(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: Required website language
setLastActiveTime(java.time.Instant) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: The time of the last action.
setLastActiveTime(java.time.Instant) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry: The last active time.
setLastActiveTime(java.time.Instant) - function in ai.platon.pulsar.crawl.fetch.privacy.PrivacyContext
setLastCollectedTime(java.time.Instant) - function in ai.platon.pulsar.common.collect.collector.AbstractDataCollector
setLastFailedLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setLAZY_PAGE_FIELDS(java.util.Set) - function in ai.platon.pulsar.crawl.component.LoadComponent.Companion
setLazyFlush(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: Lazy flush.
setLazyFlush(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If false, pages are flushed into database as soon as possible.
setLimit(java.lang.Integer) - function in ai.platon.pulsar.common.options.FetchOptions
setLimit(java.lang.Integer) - function in ai.platon.pulsar.common.options.GenerateOptions
setLimit(java.lang.Integer) - function in ai.platon.pulsar.common.options.InjectOptions
setLoadArgs(java.lang.String) - function in ai.platon.pulsar.common.collect.LocalFileHyperlinkCollector
setLoadArgs(java.lang.String) - function in ai.platon.pulsar.common.collect.LocalFileHyperlinkCollector
setLoadedSeeds(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setLoadEvent(ai.platon.pulsar.crawl.LoadEvent) - function in ai.platon.pulsar.crawl.PageEvent: The load phase event handler
setLoadEvent(ai.platon.pulsar.crawl.LoadEvent) - function in ai.platon.pulsar.crawl.event.AbstractPageEvent: The load phase event handler
setLoadEvent(ai.platon.pulsar.crawl.LoadEvent) - function in ai.platon.pulsar.crawl.event.AbstractPageEvent: The load phase event handler
setLocation(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry: The location of the page, it shows in the browser window, can differ from url.
setLOG(org.slf4j.Logger) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setLOG(org.slf4j.Logger) - function in ai.platon.pulsar.crawl.index.IndexWriters
setLOG(org.slf4j.Logger) - function in ai.platon.pulsar.crawl.index.IndexerMapping.Companion
setLogLevel(java.lang.Integer) - function in ai.platon.pulsar.common.options.LinkOptions
setLoss(java.lang.Integer) - function in ai.platon.pulsar.crawl.parse.html.OpenMapFields
setMainRequestCookies(java.util.List) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry
setMainRequestHeaders(java.util.Map) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry
setMainRequestId(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry
setMainResponseHeaders(java.util.Map) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry
setMainResponseStatus(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry
setMainResponseStatusText(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry
setMajorCode(java.lang.Short) - function in ai.platon.pulsar.persist.ParseStatus
setMappedName(java.lang.String) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setMAX_COUNTERS(java.lang.Integer) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setMAX_COUNTERS_IN_GROUP(java.lang.Integer) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setMAX_GROUPS(java.lang.Integer) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setMaxAnchorLength(java.lang.Integer) - function in ai.platon.pulsar.common.options.LinkOptions
setMaxUrlLength(java.lang.Integer) - function in ai.platon.pulsar.common.options.LinkOptions
setMediaUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setMinAnchorLength(java.lang.Integer) - function in ai.platon.pulsar.common.options.LinkOptions
setMinConfidence(int) - function in ai.platon.pulsar.common.EncodingDetector
setMinorCode(java.lang.Integer) - function in ai.platon.pulsar.persist.ParseStatus
setMinorCode(java.lang.Integer,java.lang.String) - function in ai.platon.pulsar.persist.ParseStatus
setMinUrlLength(java.lang.Integer) - function in ai.platon.pulsar.common.options.LinkOptions
setModified(java.lang.Integer) - function in ai.platon.pulsar.crawl.schedule.ModifyInfo
setModifiedAt(java.time.Instant) - function in ai.platon.pulsar.common.urls.StatefulHyperlink
setModifiedAt(java.time.Instant) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setModifiedAt(java.time.Instant) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setModifiedTime(java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.ModifyInfo
setMultiValued(java.lang.Boolean) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setName(java.lang.String) - function in ai.platon.pulsar.common.collect.CircularHyperlinkCollector
setName(java.lang.String) - function in ai.platon.pulsar.common.collect.CircularLocalFileHyperlinkCollector: The collector name
setName(java.lang.String) - function in ai.platon.pulsar.common.collect.HyperlinkCollector
setName(java.lang.String) - function in ai.platon.pulsar.common.collect.LocalFileHyperlinkCollector: The collector name
setName(java.lang.String) - function in ai.platon.pulsar.common.collect.PeriodicalHyperlinkCollector
setName(java.lang.String) - function in ai.platon.pulsar.common.collect.PeriodicalLocalFileHyperlinkCollector: The collector name
setName(java.lang.String) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setName(java.lang.String) - function in ai.platon.pulsar.crawl.parse.html.OpenMapFields
setName(java.lang.String) - function in java.lang.Thread
setNavigateEntry(ai.platon.pulsar.crawl.fetch.driver.NavigateEntry) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: The current navigation entry.
setNavigateEntry(ai.platon.pulsar.crawl.fetch.driver.NavigateEntry) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: The current navigation entry.
setNavigateEntry(ai.platon.pulsar.crawl.fetch.driver.NavigateEntry) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: The current navigation entry.
setNetCondition(ai.platon.pulsar.common.options.Condition) - function in ai.platon.pulsar.common.options.LoadOptions: Indicates the network condition.
setNextPageSelector(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The selector for next page anchor.
setNJitRetry(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: There are several cases to enable jit retry.
setNJitRetry(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Retry at most n times at fetch phase immediately if RETRY(1601) code return.
setNMaxRetry(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Retry to fetch at most n times, if page.fetchRetries nMaxRetry, the page is marked as gone and do not fetch it again until -refresh is set to clear page.
setNMaxRetry(java.lang.Integer) - function in ai.platon.pulsar.common.urls.AbstractUrl
setNMaxRetry(java.lang.Integer) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The maximum retry times
setNoCache(java.lang.Boolean) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: A convenience method.
setNoFilter(java.lang.Boolean) - function in ai.platon.pulsar.common.options.GenerateOptions
setNoFilter(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If true, no filter will be applied when parse links.
setNoFollow(java.lang.Boolean) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: A convenience method.
setNoIndex(java.lang.Boolean) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: A convenience method.
setNoNorm(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If true, no normalizer will be applied when parse links.
setNoNormalizer(java.lang.Boolean) - function in ai.platon.pulsar.common.options.GenerateOptions
setNoRedirect(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: Do not redirect.
setNotContains(java.lang.String) - function in ai.platon.pulsar.crawl.filter.TextFilter
setNRetries(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.FetchTask
setNumPoolThreads(java.lang.Integer) - function in ai.platon.pulsar.common.options.FetchOptions
setNumReduceTasks(java.lang.Integer) - function in ai.platon.pulsar.common.options.FetchOptions: Note: in browser mode, reducer tasks/fetch threads/pool threads are not used use browser instances/active browser tabs instead
setNumTasksSuccess(java.lang.Long) - function in ai.platon.pulsar.crawl.fetch.BatchStat
setObjects(java.lang.Object) - function in ai.platon.pulsar.common.options.PulsarOptions
setObjects(java.lang.Object) - function in ai.platon.pulsar.common.options.PulsarOptions
setOptions(ai.platon.pulsar.common.options.FetchOptions) - function in ai.platon.pulsar.crawl.fetch.batch.FetchMonitor: Initialize in setup using job conf
setOrder(java.lang.Integer) - function in ai.platon.pulsar.common.urls.Hyperlink
setOrder(java.lang.Integer) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The order of this hyperlink in its referrer page
setOrder(java.lang.Integer) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The order of this hyperlink in its referrer page
setOutLinkPattern(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The pattern to select out links in the portal page.
setOutLinkSelector(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The selector to extract links in portal pages.
setPage(ai.platon.pulsar.persist.WebPage) - function in ai.platon.pulsar.crawl.fetch.batch.IFetchEntry
setPAGE_FIELDS(java.util.Set) - function in ai.platon.pulsar.crawl.component.LoadComponent.Companion
setPageGoneSchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: This method specifies how to schedule refetching of pages marked as GONE.
setPageGoneSchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: This method specifies how to schedule refetching of pages marked as GONE.
setPageGoneSchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AdaptiveFetchSchedule: This method specifies how to schedule refetching of pages marked as GONE.
setPageGoneSchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AdaptiveFetchSchedule: This method specifies how to schedule refetching of pages marked as GONE.
setPageGoneSchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.FetchSchedule: This method specifies how to schedule refetching of pages marked as GONE.
setPageLoadTimeout(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The maximum time to wait for a page to finish.
setPageReferrer(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry: The referrer claimed by the page.
setPageRetrySchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setPageRetrySchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setPageRetrySchedule(ai.platon.pulsar.persist.WebPage,java.time.Instant,java.time.Instant,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.FetchSchedule: This method adjusts the fetch schedule if fetching needs to be re-tried due to transient errors.
setPagesThroughputRate(java.lang.Double) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler.Status
setPageViews(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setParent(ai.platon.pulsar.crawl.parse.ParseFilter) - function in ai.platon.pulsar.crawl.parse.AbstractParseFilter: The parent filter
setParent(ai.platon.pulsar.crawl.parse.ParseFilter) - function in ai.platon.pulsar.crawl.parse.AbstractParseFilter: The parent filter
setParent(ai.platon.pulsar.crawl.parse.ParseFilter) - function in ai.platon.pulsar.crawl.parse.ParseFilter: The parent filter
setParse(java.lang.Boolean) - function in ai.platon.pulsar.common.options.FetchOptions
setParse(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: Trigger the parse phase or not.
setParse(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If true, parse the page when it's just be fetched.
setParse(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler
setParsers(java.util.List) - function in ai.platon.pulsar.crawl.parse.ParseResult
setParsers(java.lang.String,java.util.List) - function in ai.platon.pulsar.crawl.parse.ParserConfig
setPendingFetchItems(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler.Status
setPendingStart(java.time.Instant) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask
setPersist(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: Persist fetched pages as soon as possible.
setPrevFetchTime(java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.ModifyInfo: The previous actual latest fetch time
setPrevModifiedTime(java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.ModifyInfo
setPriority(java.lang.Integer) - function in ai.platon.pulsar.common.urls.AbstractUrl
setPriority(java.lang.Integer) - function in ai.platon.pulsar.common.urls.ListenableUrl
setPriority(java.lang.Integer) - function in ai.platon.pulsar.common.urls.UrlAware
setPriority(java.lang.Integer) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The priority
setPriority(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask.Key
setPriority(java.lang.Integer) - function in java.lang.Thread
setProtocol(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask.Key
setProxy(ai.platon.pulsar.common.proxy.ProxyEntry) - function in ai.platon.pulsar.crawl.fetch.privacy.BrowserId
setProxy(java.lang.String,java.lang.String,java.lang.String,java.lang.String) - function in ai.platon.pulsar.crawl.fetch.privacy.BrowserId
setProxyEntry(ai.platon.pulsar.common.proxy.ProxyEntry) - function in ai.platon.pulsar.crawl.fetch.FetchTask
setRawEvent(ai.platon.pulsar.crawl.PageEvent) - function in ai.platon.pulsar.common.options.LoadOptions
setRawItemEvent(ai.platon.pulsar.crawl.PageEvent) - function in ai.platon.pulsar.common.options.LoadOptions
setReadonly(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: Specify if the load execution is read only or not.
setReadyFetchItems(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler.Status
setRecovered(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: Whether the driver is recovered from the browser's tab list.
setRecovered(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Whether the driver is recovered from the browser's tab list.
setRecovered(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Whether the driver is recovered from the browser's tab list.
setReferrer(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The page referrer.
setReferrer(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setReferrer(java.lang.String) - function in ai.platon.pulsar.common.urls.ListenableUrl
setReferrer(java.lang.String) - function in ai.platon.pulsar.common.urls.UrlAware
setReferrer(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The url of the referrer page
setRefresh(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions
setRefresh(java.lang.Boolean) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: Sets refresh to the supplied value.
setRefreshHref(java.net.URL) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: Sets the refreshHref.
setRefreshTime(java.lang.Integer) - function in ai.platon.pulsar.crawl.parse.html.HTMLMetaTags: Sets the refreshTime.
setReGenerate(java.lang.Boolean) - function in ai.platon.pulsar.common.options.GenerateOptions
setReGenerateSeeds(java.lang.Boolean) - function in ai.platon.pulsar.common.options.GenerateOptions
setRegexMatchedLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setRemoteAddr(java.lang.String) - function in ai.platon.pulsar.common.urls.StatefulHyperlink
setRemoteAddr(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setRemoteAddr(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setReparseLinks(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: Re-parse links if the page has been parsed before.
setRequireAnchors(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The minimum number of anchors expected in the page, if it is less than that, it will need to be re-fetched.
setRequired(java.lang.Boolean) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setRequired(java.lang.Integer) - function in ai.platon.pulsar.crawl.parse.html.OpenMapFields
setRequireImages(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The minimum number of images expected in the page, if it is less than that, it will need to be re-fetched.
setRequireNotBlank(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The selector specified element should have a non-blank text, the task should be retried if the element's text content is empty or blank.
setRequireSize(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The minimum page size expected, if it is less than that, it will need to be re-fetched.
setReservedUrl(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.batch.IFetchEntry
setResource(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If true, fetch the url as a resource without browser rendering.
setResponse(ai.platon.pulsar.crawl.protocol.Response) - function in ai.platon.pulsar.crawl.fetch.FetchResult
setResponse(ai.platon.pulsar.crawl.protocol.Response) - function in ai.platon.pulsar.crawl.protocol.Protocol
setResponse(ai.platon.pulsar.crawl.protocol.Response) - function in ai.platon.pulsar.crawl.protocol.Protocol
setResponse(ai.platon.pulsar.crawl.protocol.Response) - function in ai.platon.pulsar.crawl.protocol.Protocol
setRestrictCss(java.lang.String) - function in ai.platon.pulsar.common.options.LinkOptions
setResume(java.lang.Boolean) - function in ai.platon.pulsar.common.options.FetchOptions
setRetryDelayPolicy(kotlin.jvm.functions.Function2) - function in ai.platon.pulsar.crawl.Crawler: Delay policy for retry tasks
setRetryDelayPolicy(kotlin.jvm.functions.Function2) - function in ai.platon.pulsar.crawl.impl.AbstractCrawler: Delay policy for retry tasks
setRetryDelayPolicy(kotlin.jvm.functions.Function2) - function in ai.platon.pulsar.crawl.impl.AbstractCrawler: Delay policy for retry tasks
setReused(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: Whether the driver is recovered and is reused to serve new tasks.
setReused(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Whether the driver is recovered and is reused to serve new tasks.
setReused(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Whether the driver is recovered and is reused to serve new tasks.
setRound(java.lang.Integer) - function in ai.platon.pulsar.common.collect.PeriodicalLocalFileHyperlinkCollector.Companion.Counters
setRound(java.lang.Integer) - function in ai.platon.pulsar.common.options.FetchOptions
setRound(java.lang.Integer) - function in ai.platon.pulsar.common.options.GenerateOptions
setRunningChromeProcesses(java.lang.Integer) - function in ai.platon.pulsar.crawl.CoreMetrics.Companion
setScore(java.lang.Integer) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setScoreVector(ai.platon.pulsar.common.ScoreVector) - function in ai.platon.pulsar.crawl.scoring.io.ScoreVectorWritable
setScriptTimeout(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The maximum time to perform javascript injected into the browser.
setScrollCount(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The number of times the page should be scrolled down after it has just been opened in a browser.
setScrollInterval(java.time.Duration) - function in ai.platon.pulsar.common.options.LoadOptions: The interval to scroll down.
setSearchUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setSeeds(java.util.List) - function in ai.platon.pulsar.common.options.InjectOptions
setSkipTruncated(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler
setSlashed(java.lang.Boolean) - function in ai.platon.pulsar.common.options.InjectOptions
setSpecifiedGlobalCache(ai.platon.pulsar.crawl.common.GlobalCache) - function in ai.platon.pulsar.crawl.common.GlobalCacheFactory
setStackTrace(kotlin.Array) - function in kotlin.Throwable
setStartTime(java.time.Instant) - function in ai.platon.pulsar.crawl.fetch.BatchStat
setStatus(java.lang.String) - function in ai.platon.pulsar.common.ReducerContext
setStatus(java.lang.Integer) - function in ai.platon.pulsar.common.urls.StatefulHyperlink
setStatus(java.lang.Integer) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setStatus(java.lang.Integer) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink
setStopped(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.driver.NavigateEntry: Indicate if the navigation is stopped.
setStoreContent(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: Store webpage content or not.
setStoreContent(java.lang.Boolean) - function in ai.platon.pulsar.common.options.LoadOptions: If false, do not persist the page content which is usually very large.
setStoreContent(java.lang.Boolean) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler
setStored(java.lang.Boolean) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setStrictDf(java.lang.Boolean) - function in ai.platon.pulsar.common.options.FetchOptions
setSuccessOK() - function in ai.platon.pulsar.persist.ParseStatus
setTaskId(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The task id, it's optional and can be used to distinguish tasks.
setTaskTime(java.time.Instant) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: The default task time.
setTaskTime(java.time.Instant) - function in ai.platon.pulsar.common.options.LoadOptions: The task time, we usually use task time to indicate the name of a task batch.
setTest(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptionDefaults: Set to be 0 if we are doing unit test or other test.
setTest(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: The test level, 0 to disable, we will talk more in test mode.
setText(java.lang.String) - function in ai.platon.pulsar.common.urls.Hyperlink
setText(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The anchor text of this hyperlink
setText(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The anchor text of this hyperlink
setTiebaUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setTimeouts(ai.platon.pulsar.browser.common.BrowserSettings) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
setTimeouts(ai.platon.pulsar.browser.common.BrowserSettings) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
setTimeoutsAsync(ai.platon.pulsar.browser.common.BrowserSettings) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
setTimeoutsAsync(ai.platon.pulsar.browser.common.BrowserSettings) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
setTimeoutsAsync(ai.platon.pulsar.browser.common.BrowserSettings) - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver
setTopLinks(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Specify how many links to extract for out pages.
setTopN(java.lang.Integer) - function in ai.platon.pulsar.common.options.GenerateOptions
setTopNAnchorGroups(java.lang.Integer) - function in ai.platon.pulsar.common.options.LoadOptions: Choose the top N anchor groups for further process.
setTotalNetworkIFsRecvBytes(java.lang.Long) - function in ai.platon.pulsar.crawl.CoreMetrics: The total all bytes received by the hardware last read from system
setTotalSuccessBytes(java.lang.Long) - function in ai.platon.pulsar.crawl.fetch.BatchStat
setType(java.lang.String) - function in ai.platon.pulsar.crawl.index.IndexerMapping.MappingField
setUncaughtExceptionHandler(java.lang.Thread.UncaughtExceptionHandler) - function in java.lang.Thread
setUnfilteredLinks(java.lang.Integer) - function in ai.platon.pulsar.common.collect.FatLinkExtractor.Companion.Counters
setUnknownUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setUnserializableValue(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.driver.JsEvaluation
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.common.IndexWriter
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.common.IndexingFilter
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.common.JobInitialized
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.component.GenerateComponent
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.fetch.batch.FetchMonitor
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.fetch.batch.TaskMonitor
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.fetch.batch.TaskScheduler
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.index.jit.indexer.JITIndexer
setup(ai.platon.pulsar.common.config.ImmutableConfig) - function in ai.platon.pulsar.crawl.parse.PageParser
setUrl(java.lang.String) - function in ai.platon.pulsar.common.urls.AbstractUrl
setUrl(java.lang.String) - function in ai.platon.pulsar.common.urls.ListenableUrl
setUrl(java.lang.String) - function in ai.platon.pulsar.common.urls.UrlAware
setUrl(java.lang.String) - function in ai.platon.pulsar.crawl.common.url.CompletableHyperlink: The url of this hyperlink
setUrl(java.lang.String) - function in ai.platon.pulsar.crawl.fetch.batch.JobFetchTask.Key
setUrl(java.lang.String) - function in ai.platon.pulsar.crawl.index.IndexDocument
setUrlContains(java.lang.String) - function in ai.platon.pulsar.common.options.LinkOptions
setUrlNormalizer(ai.platon.pulsar.common.urls.preprocess.UrlNormalizerPipeline) - function in ai.platon.pulsar.common.collect.HyperlinkCollector
setUrlNormalizer(ai.platon.pulsar.common.urls.preprocess.UrlNormalizerPipeline) - function in ai.platon.pulsar.common.collect.HyperlinkCollector
setUrlPool(ai.platon.pulsar.common.collect.UrlPool) - function in ai.platon.pulsar.crawl.common.GlobalCache: A url pool contains many url caches, the urls added to the pool will be processed in crawl loops.
setUrlPostfix(java.lang.String) - function in ai.platon.pulsar.common.options.LinkOptions
setUrlPrefix(java.lang.String) - function in ai.platon.pulsar.common.options.LinkOptions
setUrlRegex(java.lang.String) - function in ai.platon.pulsar.common.options.LinkOptions
setUrls(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setUrls(kotlin.sequences.Sequence) - function in ai.platon.pulsar.crawl.impl.StreamingCrawler: The url sequence
setUrlsFromSeed(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setUrlsTooLong(java.lang.Integer) - function in ai.platon.pulsar.crawl.fetch.UrlStat
setUsedMemory(java.lang.Long) - function in ai.platon.pulsar.crawl.CoreMetrics.Companion
setValue(java.lang.Integer,java.lang.Double) - function in ai.platon.pulsar.common.ScoreVector
setValue(java.lang.Integer,java.lang.Float) - function in ai.platon.pulsar.common.ScoreVector
setValue(java.lang.Integer,java.lang.Integer) - function in ai.platon.pulsar.common.ScoreVector
setValue(java.lang.Integer) - function in ai.platon.pulsar.common.ScoreVector
setValue(T,java.lang.Integer) - function in ai.platon.pulsar.common.metrics.AppMetricRegistry
setValue(java.lang.Enum,java.lang.Integer) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setValue(java.lang.Integer,java.lang.Enum,java.lang.Integer) - function in ai.platon.pulsar.common.metrics.EnumCounterRegistry
setValue(java.lang.Object) - function in ai.platon.pulsar.crawl.fetch.driver.JsEvaluation
setValue(ai.platon.pulsar.crawl.scoring.Name,java.lang.Float) - function in ai.platon.pulsar.crawl.scoring.NamedScoreVector
setValue(ai.platon.pulsar.crawl.scoring.Name,java.lang.Integer) - function in ai.platon.pulsar.crawl.scoring.NamedScoreVector
setValue(ai.platon.pulsar.crawl.scoring.Name,java.lang.Long) - function in ai.platon.pulsar.crawl.scoring.NamedScoreVector
setVariable(java.lang.String,java.lang.Object) - function in ai.platon.pulsar.session.AbstractPulsarSession: Set a variable into this session
setVariable(java.lang.String,java.lang.Object) - function in ai.platon.pulsar.session.AbstractPulsarSession: Set a variable into this session
setVariable(java.lang.String,java.lang.Object) - function in ai.platon.pulsar.session.PulsarSession: Set a variable into this session
setVerbose(java.lang.Integer) - function in ai.platon.pulsar.common.options.FetchOptions
setVerbose(java.lang.Integer) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
setVersion(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The load option version.
setWaitForElementTimeout(java.time.Duration) - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: The timeout to wait for some object.
setWaitForElementTimeout(java.time.Duration) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: The timeout to wait for some object.
setWaitForElementTimeout(java.time.Duration) - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: The timeout to wait for some object.
setWaitNonBlank(java.lang.String) - function in ai.platon.pulsar.common.options.LoadOptions: The selector specified element should have a non-blank text, the system should wait until the element is filled by a non-blank text, or until it times out.
setWeight(java.lang.Float) - function in ai.platon.pulsar.crawl.index.IndexDocument
setWeight(java.lang.Float) - function in ai.platon.pulsar.crawl.index.IndexField
setZkHostString(java.lang.String) - function in ai.platon.pulsar.common.options.FetchOptions
setZoneId(java.lang.String) - function in ai.platon.pulsar.common.options.deprecated.CrawlOptions
shouldFetch(java.lang.String,java.lang.String,ai.platon.pulsar.persist.WebPage) - function in ai.platon.pulsar.crawl.component.GenerateComponent: TODO : We may move some filters to hbase query filters directly TODO : Move to CrawlFilter
shouldFetch(ai.platon.pulsar.persist.WebPage,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: This method provides information whether the page is suitable for selection in the current fetchlist.
shouldFetch(ai.platon.pulsar.persist.WebPage,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.AbstractFetchSchedule: This method provides information whether the page is suitable for selection in the current fetchlist.
shouldFetch(ai.platon.pulsar.persist.WebPage,java.time.Instant) - function in ai.platon.pulsar.crawl.schedule.FetchSchedule: This method provides information whether the page is suitable for selection in the current fetchlist.
shouldFetchVividPage(java.lang.String,java.time.Duration,java.time.Instant) - function in ai.platon.pulsar.common.collect.FatLinkExtractor: TODO: the logic is different from the one in LoadComponent
shutdown() - function in ai.platon.pulsar.context.PulsarContexts
shutdownExecutorOnStop(java.lang.Boolean) - function in ai.platon.pulsar.common.metrics.CodahaleSlf4jReporter.Builder: Specifies whether or not, the executor (used for reporting) will be stopped with same time with reporter.
Signature - class in ai.platon.pulsar.crawl.signature
SignatureComparator - class in ai.platon.pulsar.crawl.signature
size() - function in ai.platon.pulsar.common.ScoreVector
sniffCharacterEncoding(kotlin.Array) - function in ai.platon.pulsar.common.EncodingDetector: Given a byte[] representing an html file of an unknown encoding, read out 'charset' parameter in the meta tag from the first CHUNK_SIZE bytes.
sniffEncoding(ai.platon.pulsar.persist.WebPage) - function in ai.platon.pulsar.common.EncodingDetector
split(java.lang.String) - function in ai.platon.pulsar.common.options.PulsarOptions.Companion: Split a command line into argument vector (argv).
spliterator() - function in kotlin.collections.Collection
spliterator() - function in kotlin.collections.Collection
SPONSORED - enum entry in ai.platon.pulsar.common.domain.DomainSuffix.Status
start() - function in ai.platon.pulsar.common.CrawlLoop
start() - function in ai.platon.pulsar.common.StartStopRunnable
start() - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
start(java.time.Duration,java.time.Duration) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
start(java.time.Duration,java.time.Duration,kotlin.jvm.functions.Function0) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
start(java.lang.Long,java.util.concurrent.TimeUnit) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
start(java.lang.Long,java.lang.Long,java.util.concurrent.TimeUnit) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
start(java.lang.Long,java.lang.Long,java.util.concurrent.TimeUnit,java.lang.Runnable) - function in ai.platon.pulsar.common.concurrent.GracefulScheduledExecutor
start() - function in ai.platon.pulsar.common.metrics.MetricsSystem
start() - function in ai.platon.pulsar.common.metrics.MetricsSystem
start() - function in ai.platon.pulsar.crawl.CoreMetrics
start() - function in ai.platon.pulsar.crawl.CrawlLoops
start() - function in ai.platon.pulsar.crawl.fetch.batch.FeedLoop
start() - function in ai.platon.pulsar.crawl.fetch.batch.FetchLoop
start(ai.platon.pulsar.common.ReducerContext) - function in ai.platon.pulsar.crawl.fetch.batch.FetchMonitor
start() - function in ai.platon.pulsar.crawl.impl.StreamingCrawlLoop
start(java.lang.Long,java.util.concurrent.TimeUnit) - function in com.codahale.metrics.ScheduledReporter
start(java.lang.Long,java.lang.Long,java.util.concurrent.TimeUnit) - function in com.codahale.metrics.ScheduledReporter
start() - function in java.lang.Thread
startsWith(java.lang.String) - function in ai.platon.pulsar.common.metrics.MetricFilters
STARTUP - enum entry in ai.platon.pulsar.common.domain.DomainSuffix.Status
startWork() - function in ai.platon.pulsar.crawl.fetch.FetchTask
startWork() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver: Mark the driver as working, so it can not be used to fetch another page.
startWork() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Mark the driver as working, so it can not be used to fetch another page.
startWork() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Mark the driver as working, so it can not be used to fetch another page.
StatefulListenableHyperlink - class in ai.platon.pulsar.crawl.common.url
StaticPulsarContext - class in ai.platon.pulsar.context.support
stFetched - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
stGone - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
stNotModified - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
stop() - function in ai.platon.pulsar.common.CrawlLoop
stop() - function in ai.platon.pulsar.common.StartStopRunnable
stop() - function in ai.platon.pulsar.crawl.CrawlLoops
stop() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Force the page stop all navigations and RELEASES all resources.
stop() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver: Force the page stop all navigations and RELEASES all resources.
stop() - function in ai.platon.pulsar.crawl.impl.StreamingCrawlLoop
stop() - function in com.codahale.metrics.ScheduledReporter
stop() - function in java.lang.Thread
stopAsync() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver: Force the page stop all navigations and RELEASES all resources.
stopAsync() - function in ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver: Force the page stop all navigations and RELEASES all resources.
stopAsync() - function in ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver: Force the page stop all navigations and RELEASES all resources.
stopLoading() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
stopLoading() - function in ai.platon.pulsar.crawl.fetch.driver.WebDriver
stream() - function in kotlin.collections.Collection
stream() - function in kotlin.collections.Collection
StreamingCrawler - class in ai.platon.pulsar.crawl.impl
StreamingCrawlLoop - class in ai.platon.pulsar.crawl.impl
stRedirPerm - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
stRedirTemp - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
stRetry - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
stUnfetched - enum entry in ai.platon.pulsar.common.metrics.CommonCounter
submit(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.context.PulsarContext: Submit a url, the url will be added to the task queue, and will be executed asynchronously.
submit(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Submit a url, the url will be added to the task queue, and will be executed asynchronously.
submit(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Submit a url, the url will be added to the task queue, and will be executed asynchronously.
submit(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, the url will be processed in the crawl loop later
submit(java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, the url will be processed in the crawl loop later
submit(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(ai.platon.pulsar.common.urls.UrlAware) - function in ai.platon.pulsar.session.PulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Submit a url to the URL pool, the url will be processed in the crawl loop later
submit(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submit(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Submit a url to the URL pool, and it will be processed in a crawl loop
submitAll(java.lang.Iterable) - function in ai.platon.pulsar.context.PulsarContext: Submit a batch of urls, the urls are added to the task queue, and will be executed asynchronously.
submitAll(java.lang.Iterable) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Submit a batch of urls, the urls are added to the task queue, and will be executed asynchronously.
submitAll(java.lang.Iterable) - function in ai.platon.pulsar.context.support.AbstractPulsarContext: Submit a batch of urls, the urls are added to the task queue, and will be executed asynchronously.
submitAll(java.util.Collection) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.util.Collection) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.util.Collection,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.util.Collection,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.util.Collection) - function in ai.platon.pulsar.session.PulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.util.Collection,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable) - function in ai.platon.pulsar.session.PulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitAll(java.lang.Iterable,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Submit the urls to the URL pool, the submitted urls will be processed in a crawl loop
submitForOutPages(ai.platon.pulsar.common.urls.UrlAware,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(ai.platon.pulsar.common.urls.UrlAware,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.AbstractPulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(ai.platon.pulsar.common.urls.UrlAware,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitForOutPages(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession: Load the portal page and submit the out links specified by the -outLink option to the URL pool.
submitOutPages(ai.platon.pulsar.common.urls.UrlAware,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(ai.platon.pulsar.common.urls.UrlAware,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(ai.platon.pulsar.common.urls.UrlAware,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(java.lang.String,ai.platon.pulsar.common.options.LoadOptions) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession
submitOutPages(java.lang.String,java.lang.String) - function in ai.platon.pulsar.session.PulsarSession
subscribeWebDriver() - function in ai.platon.pulsar.crawl.fetch.privacy.PrivacyContext
success(java.lang.Integer) - function in ai.platon.pulsar.crawl.parse.FilterResult.Companion
success(ai.platon.pulsar.dom.FeaturedDocument) - function in ai.platon.pulsar.crawl.parse.ParseResult.Companion
suspend() - function in java.lang.Thread
suspendListeners() - function in ai.platon.pulsar.common.event.Browser
suspendListeners(ai.platon.pulsar.crawl.fetch.driver.BrowserEvents) - function in ai.platon.pulsar.common.event.Browser
suspendListeners() - function in ai.platon.pulsar.common.event.EventEmitter
suspendListeners(ai.platon.pulsar.crawl.fetch.driver.BrowserEvents) - function in ai.platon.pulsar.common.event.EventEmitter
suspendListeners(ai.platon.pulsar.crawl.impl.CrawlEvents) - function in ai.platon.pulsar.common.event.EventEmitter
SystemDefaultPrivacyContextIdGenerator - class in ai.platon.pulsar.crawl.fetch.privacy

A B C D E F G H I J K L M N O P Q R S T U V W _