All Implemented Interfaces:

ai.platon.pulsar.context.PulsarContext , java.lang.AutoCloseable
```
public abstract class AbstractPulsarContext
 implements PulsarContext, AutoCloseable
                    
```
Main entry point for Pulsar functionality.
A PulsarContext can be used to inject, fetch, load, parse, store Web pages.

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public class`	`AbstractPulsarContext.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private final Integer`	`id`
`private final ImmutableConfig`	`unmodifiedConfig`
`private final CrawlUrlNormalizers`	`urlNormalizers`
`private final WebDb`	`webDb`
`private final GlobalCacheFactory`	`globalCacheFactory`
`private final InjectComponent`	`injectComponent`
`private final BatchFetchComponent`	`fetchComponent`
`private final ParseComponent`	`parseComponent`
`private final UpdateComponent`	`updateComponent`
`private final LoadComponent`	`loadComponent`
`private final CrawlLoops`	`crawlLoops`
`private final Long`	`startTime`
`private final Boolean`	`isActive`
`private final ConcurrentSkipListMap<Integer, PulsarSession>`	`sessions`
`private final AbstractApplicationContext`	`applicationContext`
`private final PulsarEnvironment`	`pulsarEnvironment`

Constructor Summary

Constructors
Constructor	Description
`AbstractPulsarContext(AbstractApplicationContext applicationContext, PulsarEnvironment pulsarEnvironment)`

Method Summary

Modifier and Type	Method	Description
`Integer`	`getId()`
`ImmutableConfig`	`getUnmodifiedConfig()`
`CrawlUrlNormalizers`	`getUrlNormalizers()`
`WebDb`	`getWebDb()`
`GlobalCacheFactory`	`getGlobalCacheFactory()`
`InjectComponent`	`getInjectComponent()`
`BatchFetchComponent`	`getFetchComponent()`
`ParseComponent`	`getParseComponent()`
`UpdateComponent`	`getUpdateComponent()`
`LoadComponent`	`getLoadComponent()`
`CrawlLoops`	`getCrawlLoops()`
`final Long`	`getStartTime()`	The start time
`final Boolean`	`getIsActive()`
`final ConcurrentSkipListMap<Integer, PulsarSession>`	`getSessions()`	All open sessions
`AbstractApplicationContext`	`getApplicationContext()`
`PulsarEnvironment`	`getPulsarEnvironment()`
`final <T extends Any> T`	`getBean(KClass<T> requiredType)`
`final <T extends Any> T`	`getBean()`
`final <T extends Any> T`	`getBeanOrNull(KClass<T> requiredType)`
`final <T extends Any> T`	`getBeanOrNull()`
`abstract AbstractPulsarSession`	`createSession()`
`Unit`	`closeSession(PulsarSession session)`
`Unit`	`registerClosable(AutoCloseable closable)`	Close objects when sessions closes
`final Unit`	`clearCaches()`
`NormUrl`	`normalize(String url, LoadOptions options, Boolean toItemOption)`	Normalize an url, the url can be one of the following: a configured url a base64 encoded url a base64 encoded configured url A url can be configured by appending arguments to the url, and it also can be used with a LoadOptions, If both tailing arguments and LoadOptions are present, the LoadOptions overrides the tailing arguments, but default values in LoadOptions are ignored.
`List<NormUrl>`	`normalize(Iterable<String> urls, LoadOptions options, Boolean toItemOption)`	Normalize urls, remove invalid urls
`NormUrl`	`normalize(UrlAware url, LoadOptions options, Boolean toItemOption)`	Normalize an url.
`List<NormUrl>`	`normalize(Collection<UrlAware> urls, LoadOptions options, Boolean toItemOption)`	Normalize urls, remove invalid urls
`NormUrl`	`normalizeOrNull(String url, LoadOptions options, Boolean toItemOption)`
`NormUrl`	`normalizeOrNull(UrlAware url, LoadOptions options, Boolean toItemOption)`
`WebPage`	`inject(String url)`	Inject an url
`WebPage`	`inject(NormUrl url)`
`WebPage`	`get(String url)`
`WebPage`	`getOrNull(String url)`
`Boolean`	`exists(String url)`
`CheckState`	`fetchState(WebPage page, LoadOptions options)`
`Iterator<WebPage>`	`scan(String urlPrefix)`
`Iterator<WebPage>`	`scan(String urlPrefix, Iterable<GWebPage.Field> fields)`
`Iterator<WebPage>`	`scan(String urlPrefix, Array<String> fields)`
`WebPage`	`load(String url, LoadOptions options)`	Load a url with specified options, see LoadOptions for all options
`WebPage`	`load(URL url, LoadOptions options)`	Load a url with specified options, see LoadOptions for all options
`WebPage`	`load(NormUrl url)`	Load a url, options can be specified following the url, see LoadOptions for all options
`WebPage`	`loadDeferred(NormUrl url)`
`Collection<WebPage>`	`loadAll(Iterable<String> urls, LoadOptions options)`	Load a batch of urls with the specified options.
`Collection<WebPage>`	`loadAll(Collection<NormUrl> urls, LoadOptions options)`
`FeaturedDocument`	`parse(WebPage page)`	Parse the WebPage using parseComponent
`Unit`	`persist(WebPage page)`
`Unit`	`delete(String url)`
`Unit`	`delete(WebPage page)`
`Unit`	`flush()`
`Unit`	`registerShutdownHook()`	Register a shutdown hook with the JVM runtime, closing this context on JVM shutdown unless it has already been closed at that time.
`Unit`	`close()`	Close this pulsar context, destroying all beans in its bean factory.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - AbstractPulsarContext
```
AbstractPulsarContext(AbstractApplicationContext applicationContext, PulsarEnvironment pulsarEnvironment)
```
- Method Detail
  - getId
```
 Integer getId()
```
  - getUnmodifiedConfig
```
 ImmutableConfig getUnmodifiedConfig()
```
  - getUrlNormalizers
```
 CrawlUrlNormalizers getUrlNormalizers()
```
  - getWebDb
```
 WebDb getWebDb()
```
  - getGlobalCacheFactory
```
 GlobalCacheFactory getGlobalCacheFactory()
```
  - getInjectComponent
```
 InjectComponent getInjectComponent()
```
  - getFetchComponent
```
 BatchFetchComponent getFetchComponent()
```
  - getParseComponent
```
 ParseComponent getParseComponent()
```
  - getUpdateComponent
```
 UpdateComponent getUpdateComponent()
```
  - getLoadComponent
```
 LoadComponent getLoadComponent()
```
  - getCrawlLoops
```
 CrawlLoops getCrawlLoops()
```
  - getStartTime
```
 final Long getStartTime()
```
    The start time
  - getIsActive
```
 final Boolean getIsActive()
```
  - getSessions
```
 final ConcurrentSkipListMap<Integer, PulsarSession> getSessions()
```
    All open sessions
  - getApplicationContext
```
 AbstractApplicationContext getApplicationContext()
```
  - getPulsarEnvironment
```
 PulsarEnvironment getPulsarEnvironment()
```
  - getBean
```
 final <T extends Any> T getBean(KClass<T> requiredType)
```
  - getBean
```
 final <T extends Any> T getBean()
```
  - getBeanOrNull
```
 final <T extends Any> T getBeanOrNull(KClass<T> requiredType)
```
  - getBeanOrNull
```
 final <T extends Any> T getBeanOrNull()
```
  - createSession
```
 abstract AbstractPulsarSession createSession()
```
  - closeSession
```
 Unit closeSession(PulsarSession session)
```
  - registerClosable
```
 Unit registerClosable(AutoCloseable closable)
```
    Close objects when sessions closes
  - clearCaches
```
 final Unit clearCaches()
```
  - normalize
```
 NormUrl normalize(String url, LoadOptions options, Boolean toItemOption)
```
    Normalize an url, the url can be one of the following:
    a configured url
    a base64 encoded url
    a base64 encoded configured url
    A url can be configured by appending arguments to the url, and it also can be used with a LoadOptions, If both tailing arguments and LoadOptions are present, the LoadOptions overrides the tailing arguments, but default values in LoadOptions are ignored.
  - normalize
```
 List<NormUrl> normalize(Iterable<String> urls, LoadOptions options, Boolean toItemOption)
```
    Normalize urls, remove invalid urls
    
    Parameters:
    
    urls - The urls to normalize
    
    options - The LoadOptions applied to each url
    
    toItemOption - If the LoadOptions is converted to item load options
  - normalize
```
 NormUrl normalize(UrlAware url, LoadOptions options, Boolean toItemOption)
```
    Normalize an url.
    If both url arguments and LoadOptions are present, the LoadOptions overrides the tailing arguments, but default values in LoadOptions are ignored.
  - normalize
```
 List<NormUrl> normalize(Collection<UrlAware> urls, LoadOptions options, Boolean toItemOption)
```
    Normalize urls, remove invalid urls
    
    Parameters:
    
    urls - The urls to normalize
    
    options - The LoadOptions applied to each url
    
    toItemOption - If the LoadOptions is converted to item load options
  - normalizeOrNull
```
 NormUrl normalizeOrNull(String url, LoadOptions options, Boolean toItemOption)
```
  - normalizeOrNull
```
 NormUrl normalizeOrNull(UrlAware url, LoadOptions options, Boolean toItemOption)
```
  - inject
```
 WebPage inject(String url)
```
    Inject an url
    
    Parameters:
    
    url - The url followed by config options
  - inject
```
 WebPage inject(NormUrl url)
```
  - get
```
 WebPage get(String url)
```
  - getOrNull
```
 WebPage getOrNull(String url)
```
  - exists
```
 Boolean exists(String url)
```
  - fetchState
```
 CheckState fetchState(WebPage page, LoadOptions options)
```
  - scan
```
 Iterator<WebPage> scan(String urlPrefix)
```
  - scan
```
 Iterator<WebPage> scan(String urlPrefix, Iterable<GWebPage.Field> fields)
```
  - scan
```
 Iterator<WebPage> scan(String urlPrefix, Array<String> fields)
```
  - load
```
 WebPage load(String url, LoadOptions options)
```
    Load a url with specified options, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
    
    options - The options
  - load
```
 WebPage load(URL url, LoadOptions options)
```
    Load a url with specified options, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
    
    options - The options
  - load
```
 WebPage load(NormUrl url)
```
    Load a url, options can be specified following the url, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
  - loadDeferred
```
 WebPage loadDeferred(NormUrl url)
```
  - loadAll
```
 Collection<WebPage> loadAll(Iterable<String> urls, LoadOptions options)
```
    Load a batch of urls with the specified options.
    If the option indicates prefer parallel, urls are fetched in a parallel manner whenever applicable. If the batch is too large, only a random part of the urls is fetched immediately, all the rest urls are put into a pending fetch list and will be fetched in background later.
    If a page does not exists neither in local storage nor at the given remote location, WebPage.NIL is returned
    
    Parameters:
    
    urls - The urls to load
    
    options - The options
  - loadAll
```
 Collection<WebPage> loadAll(Collection<NormUrl> urls, LoadOptions options)
```
  - parse
```
 FeaturedDocument parse(WebPage page)
```
    Parse the WebPage using parseComponent
  - persist
```
 Unit persist(WebPage page)
```
  - delete
```
 Unit delete(String url)
```
  - delete
```
 Unit delete(WebPage page)
```
  - flush
```
 Unit flush()
```
  - registerShutdownHook
```
 Unit registerShutdownHook()
```
    Register a shutdown hook with the JVM runtime, closing this context on JVM shutdown unless it has already been closed at that time.
    Delegates to doClose() for the actual closing procedure.
  - close
```
 Unit close()
```
    Close this pulsar context, destroying all beans in its bean factory.
    Delegates to doClose() for the actual closing procedure. Also removes a JVM shutdown hook, if registered, as it's not needed anymore.

Class AbstractPulsarContext

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

AbstractPulsarContext

Method Detail

getId

getUnmodifiedConfig

getUrlNormalizers

getWebDb

getGlobalCacheFactory

getInjectComponent

getFetchComponent

getParseComponent

getUpdateComponent

getLoadComponent

getCrawlLoops

getStartTime

getIsActive

getSessions

getApplicationContext

getPulsarEnvironment

getBean

getBean

getBeanOrNull

getBeanOrNull

createSession

closeSession

registerClosable

clearCaches

normalize

normalize

normalize

normalize

normalizeOrNull

normalizeOrNull

inject

inject

get

getOrNull

exists

fetchState

scan

scan

scan

load

load

load

loadDeferred

loadAll

loadAll

parse

persist

delete

delete

flush

registerShutdownHook

close