All Implemented Interfaces:

java.lang.AutoCloseable
```
public interface PulsarContext
 implements AutoCloseable
                    
```
Main entry point for Pulsar functionality.
A PulsarContext is used to inject, fetch, load, parse, store webpages.

Method Summary

Modifier and Type	Method	Description
`abstract <T extends Any> T`	`getBean(KClass<T> requiredType)`	Get a bean with the specified class, throws BeansException if the bean doesn't exist
`abstract <T extends Any> T`	`getBeanOrNull(KClass<T> requiredType)`	Get a bean with the specified class, returns null if the bean doesn't exist
`abstract PulsarSession`	`createSession()`
`abstract Unit`	`closeSession(PulsarSession session)`
`abstract Unit`	`registerClosable(AutoCloseable closable)`	Register close objects, the objects will be closed when the context closes
`abstract NormUrl`	`normalize(String url, LoadOptions options, Boolean toItemOption)`	Normalize an url, the url can be one of the following: a normal url a configured url a base64 encoded url a base64 encoded configured url An url can be configured by appending arguments to the url, and it also can be used with a LoadOptions, If both tailing arguments and LoadOptions are present, the LoadOptions overrides the tailing arguments, but default values in LoadOptions are ignored.
`abstract List<NormUrl>`	`normalize(Iterable<String> urls, LoadOptions options, Boolean toItemOption)`	Normalize urls, remove invalid ones
`abstract NormUrl`	`normalize(UrlAware url, LoadOptions options, Boolean toItemOption)`	Normalize an url.
`abstract List<NormUrl>`	`normalize(Collection<UrlAware> urls, LoadOptions options, Boolean toItemOption)`	Normalize urls, remove invalid ones
`abstract NormUrl`	`normalizeOrNull(String url, LoadOptions options, Boolean toItemOption)`
`abstract NormUrl`	`normalizeOrNull(UrlAware url, LoadOptions options, Boolean toItemOption)`
`abstract WebPage`	`inject(String url)`	Inject an url
`abstract WebPage`	`inject(NormUrl url)`	Inject an url
`abstract WebPage`	`get(String url)`	Get a webpage from the storage
`abstract WebPage`	`getOrNull(String url)`	Get a webpage from the storage
`abstract Boolean`	`exists(String url)`	Get a webpage from the storage
`abstract CheckState`	`fetchState(WebPage page, LoadOptions options)`	Check the fetch state of a webpage
`abstract Iterator<WebPage>`	`scan(String urlPrefix)`	Scan webpages in the storage whose url start with urlPrefix
`abstract Iterator<WebPage>`	`scan(String urlPrefix, Iterable<GWebPage.Field> fields)`
`abstract Iterator<WebPage>`	`scan(String urlPrefix, Array<String> fields)`
`abstract WebPage`	`load(String url, LoadOptions options)`	Load a url with specified options, see LoadOptions for all options
`abstract WebPage`	`load(URL url, LoadOptions options)`	Load a url with specified options, see LoadOptions for all options
`abstract WebPage`	`load(NormUrl url)`	Load a url, options can be specified following the url, see LoadOptions for all options
`abstract WebPage`	`loadDeferred(NormUrl url)`	Load a url, options can be specified following the url, see LoadOptions for all options
`abstract Collection<WebPage>`	`loadAll(Iterable<String> urls, LoadOptions options)`	Load a batch of urls with the specified options.
`abstract Collection<WebPage>`	`loadAll(Iterable<NormUrl> urls)`	Load a batch of urls with the specified options.
`abstract CompletableFuture<WebPage>`	`loadAsync(NormUrl url)`	Load a url asynchronously, the url is added to the crawl task queue, and will be executed asynchronously
`abstract List<CompletableFuture<WebPage>>`	`loadAllAsync(Iterable<NormUrl> urls)`	Load a batch of urls asynchronously, the urls are added to the crawl task queue, and will be executed asynchronously
`abstract PulsarContext`	`submit(UrlAware url)`	Submit a url, the url is added to the crawl task queue, and will be executed asynchronously
`abstract PulsarContext`	`submitAll(Iterable<UrlAware> urls)`	Submit a batch of urls, the urls are added to the crawl task queue, and will be executed asynchronously
`abstract FeaturedDocument`	`parse(WebPage page)`	Parse the WebPage using ParseComponent
`abstract Unit`	`persist(WebPage page)`	Persist the webpage into the backend storage immediately.
`abstract Unit`	`delete(String url)`	Delete the webpage from the backend storage
`abstract Unit`	`delete(WebPage page)`	Delete the webpage from the backend storage
`abstract Unit`	`flush()`	Flush the backend storage
`abstract Unit`	`await()`	Wait until all tasks are done.
`abstract Unit`	`registerShutdownHook()`	Register shutdown hook
`abstract Integer`	`getId()`	The context id
`abstract PulsarEnvironment`	`getPulsarEnvironment()`
`abstract ApplicationContext`	`getApplicationContext()`
`abstract ImmutableConfig`	`getUnmodifiedConfig()`	An immutable config is loaded from the config file at process startup, and never changes
`abstract UrlPool`	`getCrawlPool()`
`abstract CrawlLoops`	`getCrawlLoops()`

Methods inherited from class ai.platon.pulsar.context.PulsarContext
close
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Method Detail
  - getBean
```
 abstract <T extends Any> T getBean(KClass<T> requiredType)
```
    Get a bean with the specified class, throws BeansException if the bean doesn't exist
  - getBeanOrNull
```
 abstract <T extends Any> T getBeanOrNull(KClass<T> requiredType)
```
    Get a bean with the specified class, returns null if the bean doesn't exist
  - createSession
```
 abstract PulsarSession createSession()
```
  - closeSession
```
 abstract Unit closeSession(PulsarSession session)
```
  - registerClosable
```
 abstract Unit registerClosable(AutoCloseable closable)
```
    Register close objects, the objects will be closed when the context closes
  - normalize
```
 abstract NormUrl normalize(String url, LoadOptions options, Boolean toItemOption)
```
    Normalize an url, the url can be one of the following:
    a normal url
    a configured url
    a base64 encoded url
    a base64 encoded configured url
    An url can be configured by appending arguments to the url, and it also can be used with a LoadOptions, If both tailing arguments and LoadOptions are present, the LoadOptions overrides the tailing arguments, but default values in LoadOptions are ignored.
  - normalize
```
 abstract List<NormUrl> normalize(Iterable<String> urls, LoadOptions options, Boolean toItemOption)
```
    Normalize urls, remove invalid ones
    
    Parameters:
    
    urls - The urls to normalize
    
    options - The LoadOptions applied to each url
    
    toItemOption - If the LoadOptions is converted to item load options
  - normalize
```
 abstract NormUrl normalize(UrlAware url, LoadOptions options, Boolean toItemOption)
```
    Normalize an url.
    If both url arguments and LoadOptions are present, the LoadOptions overrides the tailing arguments, but default values in LoadOptions are ignored.
  - normalize
```
 abstract List<NormUrl> normalize(Collection<UrlAware> urls, LoadOptions options, Boolean toItemOption)
```
    Normalize urls, remove invalid ones
    
    Parameters:
    
    urls - The urls to normalize
    
    options - The LoadOptions applied to each url
    
    toItemOption - If the LoadOptions is converted to item load options
  - normalizeOrNull
```
 abstract NormUrl normalizeOrNull(String url, LoadOptions options, Boolean toItemOption)
```
  - normalizeOrNull
```
 abstract NormUrl normalizeOrNull(UrlAware url, LoadOptions options, Boolean toItemOption)
```
  - inject
```
 abstract WebPage inject(String url)
```
    Inject an url
    
    Parameters:
    
    url - The url followed by config options
  - inject
```
 abstract WebPage inject(NormUrl url)
```
    Inject an url
    
    Parameters:
    
    url - The url followed by config options
  - get
```
 abstract WebPage get(String url)
```
    Get a webpage from the storage
    
    Parameters:
    
    url - The url of the page to retrieve
  - getOrNull
```
 abstract WebPage getOrNull(String url)
```
    Get a webpage from the storage
    
    Parameters:
    
    url - The url of the page to retrieve
  - exists
```
 abstract Boolean exists(String url)
```
    Get a webpage from the storage
    
    Parameters:
    
    url - The url of the page to check
  - fetchState
```
 abstract CheckState fetchState(WebPage page, LoadOptions options)
```
    Check the fetch state of a webpage
    
    Parameters:
    
    page - The webpage to check
  - scan
```
 abstract Iterator<WebPage> scan(String urlPrefix)
```
    Scan webpages in the storage whose url start with urlPrefix
    
    Parameters:
    
    urlPrefix - The url prefix
  - scan
```
 abstract Iterator<WebPage> scan(String urlPrefix, Iterable<GWebPage.Field> fields)
```
  - scan
```
 abstract Iterator<WebPage> scan(String urlPrefix, Array<String> fields)
```
  - load
```
 abstract WebPage load(String url, LoadOptions options)
```
    Load a url with specified options, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
    
    options - The options
  - load
```
 abstract WebPage load(URL url, LoadOptions options)
```
    Load a url with specified options, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
    
    options - The options
  - load
```
 abstract WebPage load(NormUrl url)
```
    Load a url, options can be specified following the url, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
  - loadDeferred
```
 abstract WebPage loadDeferred(NormUrl url)
```
    Load a url, options can be specified following the url, see LoadOptions for all options
    
    Parameters:
    
    url - The url followed by options
  - loadAll
```
 abstract Collection<WebPage> loadAll(Iterable<String> urls, LoadOptions options)
```
    Load a batch of urls with the specified options.
    If the option indicates prefer parallel, urls are fetched in a parallel manner whenever applicable. If the batch is too large, only a random part of the urls is fetched immediately, all the rest urls are put into a pending fetch list and will be fetched in background later.
    If a page exists neither in local storage nor at the given remote location, WebPage.NIL is returned
    
    Parameters:
    
    urls - The urls to load
    
    options - The options
  - loadAll
```
 abstract Collection<WebPage> loadAll(Iterable<NormUrl> urls)
```
    Load a batch of urls with the specified options.
    If the option indicates prefer parallel, urls are fetched in a parallel manner whenever applicable. If the batch is too large, only a random part of the urls is fetched immediately, all the rest urls are put into a pending fetch list and will be fetched in background later.
    If a page exists neither in local storage nor at the given remote location, WebPage.NIL is returned
    
    Parameters:
    
    urls - The urls to load
  - loadAsync
```
 abstract CompletableFuture<WebPage> loadAsync(NormUrl url)
```
    Load a url asynchronously, the url is added to the crawl task queue, and will be executed asynchronously
  - loadAllAsync
```
 abstract List<CompletableFuture<WebPage>> loadAllAsync(Iterable<NormUrl> urls)
```
    Load a batch of urls asynchronously, the urls are added to the crawl task queue, and will be executed asynchronously
  - submit
```
 abstract PulsarContext submit(UrlAware url)
```
    Submit a url, the url is added to the crawl task queue, and will be executed asynchronously
  - submitAll
```
 abstract PulsarContext submitAll(Iterable<UrlAware> urls)
```
    Submit a batch of urls, the urls are added to the crawl task queue, and will be executed asynchronously
  - parse
```
 abstract FeaturedDocument parse(WebPage page)
```
    Parse the WebPage using ParseComponent
  - persist
```
 abstract Unit persist(WebPage page)
```
    Persist the webpage into the backend storage immediately. By default, the backend storage is the local file system, if mongodb is detected, the mongodb will be used as the backend storage.
  - delete
```
 abstract Unit delete(String url)
```
    Delete the webpage from the backend storage
  - delete
```
 abstract Unit delete(WebPage page)
```
    Delete the webpage from the backend storage
  - flush
```
 abstract Unit flush()
```
    Flush the backend storage
  - await
```
 abstract Unit await()
```
    Wait until all tasks are done.
  - registerShutdownHook
```
 abstract Unit registerShutdownHook()
```
    Register shutdown hook
  - getId
```
 abstract Integer getId()
```
    The context id
  - getPulsarEnvironment
```
 abstract PulsarEnvironment getPulsarEnvironment()
```
  - getApplicationContext
```
 abstract ApplicationContext getApplicationContext()
```
  - getUnmodifiedConfig
```
 abstract ImmutableConfig getUnmodifiedConfig()
```
    An immutable config is loaded from the config file at process startup, and never changes
  - getCrawlPool
```
 abstract UrlPool getCrawlPool()
```
  - getCrawlLoops
```
 abstract CrawlLoops getCrawlLoops()
```

Interface PulsarContext

Method Summary

Methods inherited from class ai.platon.pulsar.context.PulsarContext

Methods inherited from class java.lang.Object

Method Detail

getBean

getBeanOrNull

createSession

closeSession

registerClosable

normalize

normalize

normalize

normalize

normalizeOrNull

normalizeOrNull

inject

inject

get

getOrNull

exists

fetchState

scan

scan

scan

load

load

load

loadDeferred

loadAll

loadAll

loadAsync

loadAllAsync

submit

submitAll

parse

persist

delete

delete

flush

await

registerShutdownHook

getId

getPulsarEnvironment

getApplicationContext

getUnmodifiedConfig

getCrawlPool

getCrawlLoops