-
- All Implemented Interfaces:
-
ai.platon.pulsar.session.PulsarSession,java.lang.AutoCloseable
public abstract class AbstractPulsarSession implements PulsarSession
Created by vincent on 18-1-17. Copyright @ 2013-2017 Platon AI. All rights reserved
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classAbstractPulsarSession.Companion
-
Field Summary
Fields Modifier and Type Field Description private final ImmutableConfigunmodifiedConfigprivate final BeanFactorysessionBeanFactoryprivate final Stringdisplayprivate final BooleanisActiveprivate final GlobalCacheFactoryglobalCacheFactoryprivate final ConcurrentExpiringLRUCache<String, WebPage>pageCacheprivate final ConcurrentExpiringLRUCache<String, FeaturedDocument>documentCacheprivate final AbstractPulsarContextcontextprivate final VolatileConfigsessionConfigprivate final Integerid
-
Constructor Summary
Constructors Constructor Description AbstractPulsarSession(AbstractPulsarContext context, VolatileConfig sessionConfig, Integer id)
-
Method Summary
Modifier and Type Method Description ImmutableConfiggetUnmodifiedConfig()BeanFactorygetSessionBeanFactory()The session scoped bean factory StringgetDisplay()final BooleangetIsActive()GlobalCacheFactorygetGlobalCacheFactory()ConcurrentExpiringLRUCache<String, WebPage>getPageCache()ConcurrentExpiringLRUCache<String, FeaturedDocument>getDocumentCache()AbstractPulsarContextgetContext()The pulsar context VolatileConfiggetSessionConfig()The session scope volatile config, every setting is supposed to be changed at any time and any place IntegergetId()The session id. <ERROR CLASS>registerClosable(AutoCloseable closable)Close objects when the session is closing <ERROR CLASS>disableCache()LoadOptionsoptions(String args, PulsarEventHandler eventHandler)Create a new options, with a new volatile config Stringproperty(String name)Unitproperty(String name, String value)NormUrlnormalize(String url, String args)NormUrlnormalize(String url, LoadOptions options, Boolean toItemOption)List<NormUrl>normalize(Iterable<String> urls, LoadOptions options, Boolean toItemOption)NormUrlnormalize(UrlAware url, LoadOptions options, Boolean toItemOption)List<NormUrl>normalize(Collection<UrlAware> urls, LoadOptions options, Boolean toItemOption)NormUrlnormalizeOrNull(String url, LoadOptions options, Boolean toItemOption)NormUrlnormalizeOrNull(UrlAware url, LoadOptions options, Boolean toItemOption)WebPageinject(String url)Inject a url WebPageget(String url)Get a page from database if exists WebPagegetOrNull(String url)Get a page from database if exists Booleanexists(String url)Check if a page exists in the database CheckStatefetchState(WebPage page, LoadOptions options)Return the fetch state of the page WebPageopen(String url)Open a page with url WebPageload(String url, String args)Load a url with specified options WebPageload(String url, LoadOptions options)Load a url with specified options WebPageload(UrlAware url, String args)WebPageload(UrlAware url, LoadOptions options)WebPageload(NormUrl normUrl)WebPageloadDeferred(String url, LoadOptions options)WebPageloadDeferred(UrlAware url, String args)WebPageloadDeferred(UrlAware url, LoadOptions options)WebPageloadDeferred(NormUrl normUrl)List<WebPage>loadAll(Iterable<String> urls, LoadOptions options, Boolean toItemOption)Load all urls with specified options, this may cause a parallel fetching if required List<WebPage>loadAll(Iterable<NormUrl> normUrls)CompletableFuture<WebPage>loadAsync(NormUrl url)List<CompletableFuture<WebPage>>loadAllAsync(Iterable<NormUrl> urls)AbstractPulsarSessionsubmit(UrlAware url)AbstractPulsarSessionsubmitAll(Iterable<UrlAware> urls)List<WebPage>loadOutPages(String portalUrl, String args)Load all out pages in a portal page List<WebPage>loadOutPages(String portalUrl, LoadOptions options)Load all out pages in a portal page AbstractPulsarSessionsubmitOutPages(String portalUrl, String args)AbstractPulsarSessionsubmitOutPages(String portalUrl, LoadOptions options)List<CompletableFuture<WebPage>>loadOutPagesAsync(String portalUrl, LoadOptions options)WebPageloadResource(String url, String referer, String args)Load an url as a resource without browser rendering in the browser context WebPageloadResource(String url, String referer, LoadOptions opts)Load an url as a resource without browser rendering in the browser context FeaturedDocumentparse(WebPage page, Boolean noCache)Parse the Web page into DOM. FeaturedDocumentloadDocument(String url, String args)FeaturedDocumentloadDocument(String url, LoadOptions options)FeaturedDocumentloadDocument(NormUrl normUrl)Map<String, String>scrape(String url, String args, Iterable<String> fieldSelectors)Map<String, String>scrape(String url, String args, Map<String, String> fieldSelectors)List<Map<String, String>>scrape(String url, String args, String restrictSelector, Iterable<String> fieldSelectors)List<Map<String, String>>scrape(String url, String args, String restrictSelector, Map<String, String> fieldSelectors)List<Map<String, String>>scrapeOutPages(String portalUrl, String args, Iterable<String> fieldSelectors)List<Map<String, String>>scrapeOutPages(String portalUrl, String args, String restrictSelector, Iterable<String> fieldSelectors)List<Map<String, String>>scrapeOutPages(String portalUrl, String args, Map<String, String> fieldSelectors)List<Map<String, String>>scrapeOutPages(String portalUrl, String args, String restrictSelector, Map<String, String> fieldSelectors)ObjectgetVariable(String name)<ERROR CLASS>setVariable(String name, Object value)UnitputSessionBean(Object obj)final <T extends Any> TgetSessionBean()Unitdelete(String url)Unitflush()Booleanpersist(WebPage page)Pathexport(WebPage page, String ident)Pathexport(FeaturedDocument doc, String ident)PathexportTo(FeaturedDocument doc, Path path)Booleanequals(Object other)IntegerhashCode()StringtoString()Unitclose()-
-
Constructor Detail
-
AbstractPulsarSession
AbstractPulsarSession(AbstractPulsarContext context, VolatileConfig sessionConfig, Integer id)
-
-
Method Detail
-
getUnmodifiedConfig
ImmutableConfig getUnmodifiedConfig()
-
getSessionBeanFactory
BeanFactory getSessionBeanFactory()
The session scoped bean factory
-
getDisplay
String getDisplay()
-
getIsActive
final Boolean getIsActive()
-
getGlobalCacheFactory
GlobalCacheFactory getGlobalCacheFactory()
-
getPageCache
ConcurrentExpiringLRUCache<String, WebPage> getPageCache()
-
getDocumentCache
ConcurrentExpiringLRUCache<String, FeaturedDocument> getDocumentCache()
-
getContext
AbstractPulsarContext getContext()
The pulsar context
-
getSessionConfig
VolatileConfig getSessionConfig()
The session scope volatile config, every setting is supposed to be changed at any time and any place
-
getId
Integer getId()
The session id. Session id is expected to be set by the container, e.g. the h2 database runtime
-
registerClosable
<ERROR CLASS> registerClosable(AutoCloseable closable)
Close objects when the session is closing
-
disableCache
<ERROR CLASS> disableCache()
-
options
LoadOptions options(String args, PulsarEventHandler eventHandler)
Create a new options, with a new volatile config
-
normalize
NormUrl normalize(String url, LoadOptions options, Boolean toItemOption)
-
normalize
NormUrl normalize(UrlAware url, LoadOptions options, Boolean toItemOption)
-
normalize
List<NormUrl> normalize(Collection<UrlAware> urls, LoadOptions options, Boolean toItemOption)
-
normalizeOrNull
NormUrl normalizeOrNull(String url, LoadOptions options, Boolean toItemOption)
-
normalizeOrNull
NormUrl normalizeOrNull(UrlAware url, LoadOptions options, Boolean toItemOption)
-
getOrNull
WebPage getOrNull(String url)
Get a page from database if exists
- Parameters:
url- The url
-
fetchState
CheckState fetchState(WebPage page, LoadOptions options)
Return the fetch state of the page
- Parameters:
page- The webpageoptions- The load options
-
load
WebPage load(String url, String args)
Load a url with specified options
- Parameters:
url- The url to loadargs- The load args
-
load
WebPage load(String url, LoadOptions options)
Load a url with specified options
- Parameters:
url- The url to loadoptions- The load options
-
load
WebPage load(UrlAware url, LoadOptions options)
-
loadDeferred
WebPage loadDeferred(String url, LoadOptions options)
-
loadDeferred
WebPage loadDeferred(UrlAware url, String args)
-
loadDeferred
WebPage loadDeferred(UrlAware url, LoadOptions options)
-
loadDeferred
WebPage loadDeferred(NormUrl normUrl)
-
loadAll
List<WebPage> loadAll(Iterable<String> urls, LoadOptions options, Boolean toItemOption)
Load all urls with specified options, this may cause a parallel fetching if required
- Parameters:
urls- The urls to loadoptions- The load options for all urls
-
loadAsync
CompletableFuture<WebPage> loadAsync(NormUrl url)
-
loadAllAsync
List<CompletableFuture<WebPage>> loadAllAsync(Iterable<NormUrl> urls)
-
submit
AbstractPulsarSession submit(UrlAware url)
-
submitAll
AbstractPulsarSession submitAll(Iterable<UrlAware> urls)
-
loadOutPages
List<WebPage> loadOutPages(String portalUrl, String args)
Load all out pages in a portal page
- Parameters:
portalUrl- The portal url from where to load pagesargs- The load args
-
loadOutPages
List<WebPage> loadOutPages(String portalUrl, LoadOptions options)
Load all out pages in a portal page
- Parameters:
portalUrl- The portal url from where to load pagesoptions- The load options
-
submitOutPages
AbstractPulsarSession submitOutPages(String portalUrl, String args)
-
submitOutPages
AbstractPulsarSession submitOutPages(String portalUrl, LoadOptions options)
-
loadOutPagesAsync
List<CompletableFuture<WebPage>> loadOutPagesAsync(String portalUrl, LoadOptions options)
-
loadResource
WebPage loadResource(String url, String referer, String args)
Load an url as a resource without browser rendering in the browser context
- Parameters:
url- The url to loadargs- The load args
-
loadResource
WebPage loadResource(String url, String referer, LoadOptions opts)
Load an url as a resource without browser rendering in the browser context
- Parameters:
url- The url to loadopts- The load options
-
parse
FeaturedDocument parse(WebPage page, Boolean noCache)
Parse the Web page into DOM. If the Web page is not changed since last parse, use the last result if available
-
loadDocument
FeaturedDocument loadDocument(String url, String args)
-
loadDocument
FeaturedDocument loadDocument(String url, LoadOptions options)
-
loadDocument
FeaturedDocument loadDocument(NormUrl normUrl)
-
scrape
List<Map<String, String>> scrape(String url, String args, String restrictSelector, Iterable<String> fieldSelectors)
-
scrape
List<Map<String, String>> scrape(String url, String args, String restrictSelector, Map<String, String> fieldSelectors)
-
scrapeOutPages
@ExperimentalApi() List<Map<String, String>> scrapeOutPages(String portalUrl, String args, Iterable<String> fieldSelectors)
-
scrapeOutPages
@ExperimentalApi() List<Map<String, String>> scrapeOutPages(String portalUrl, String args, String restrictSelector, Iterable<String> fieldSelectors)
-
scrapeOutPages
@ExperimentalApi() List<Map<String, String>> scrapeOutPages(String portalUrl, String args, Map<String, String> fieldSelectors)
-
scrapeOutPages
@ExperimentalApi() List<Map<String, String>> scrapeOutPages(String portalUrl, String args, String restrictSelector, Map<String, String> fieldSelectors)
-
getVariable
Object getVariable(String name)
-
setVariable
<ERROR CLASS> setVariable(String name, Object value)
-
putSessionBean
Unit putSessionBean(Object obj)
-
getSessionBean
final <T extends Any> T getSessionBean()
-
-
-
-