All Implemented Interfaces:

java.lang.AutoCloseable
```
@Component() 
public final class LoadComponent
 implements AutoCloseable
                    
```
Created by vincent on 17-7-15. Copyright @ 2013-2017 Platon AI. All rights reserved
Load pages from storage or fetch from the Internet if it's not fetched or expired

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public class`	`LoadComponent.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private final GlobalCache`	`globalCache`
`private final ConcurrentExpiringLRUCache<String, WebPage>`	`pageCache`
`private final ConcurrentExpiringLRUCache<String, FeaturedDocument>`	`documentCache`
`private final Array<String>`	`metadataFields`
`private volatile String`	`lastPageReport`
`private final WebDb`	`webDb`
`private final GlobalCacheFactory`	`globalCacheFactory`
`private final BatchFetchComponent`	`fetchComponent`
`private final ParseComponent`	`parseComponent`
`private final UpdateComponent`	`updateComponent`
`private final AppStatusTracker`	`statusTracker`
`private final CrawlLoop`	`crawlLoop`
`private final ImmutableConfig`	`immutableConfig`

Constructor Summary

Constructors
Constructor	Description
`LoadComponent(WebDb webDb, GlobalCacheFactory globalCacheFactory, BatchFetchComponent fetchComponent, ParseComponent parseComponent, UpdateComponent updateComponent, ImmutableConfig immutableConfig)`
`LoadComponent(WebDb webDb, GlobalCacheFactory globalCacheFactory, BatchFetchComponent fetchComponent, ParseComponent parseComponent, UpdateComponent updateComponent, AppStatusTracker statusTracker, CrawlLoop crawlLoop, ImmutableConfig immutableConfig)`

Method Summary

Modifier and Type	Method	Description
`final GlobalCache`	`getGlobalCache()`
`final ConcurrentExpiringLRUCache<String, WebPage>`	`getPageCache()`
`final ConcurrentExpiringLRUCache<String, FeaturedDocument>`	`getDocumentCache()`
`final Array<String>`	`getMetadataFields()`
`final String`	`getLastPageReport()`
`final Unit`	`setLastPageReport(String lastPageReport)`
`final WebDb`	`getWebDb()`
`final GlobalCacheFactory`	`getGlobalCacheFactory()`
`final BatchFetchComponent`	`getFetchComponent()`
`final ParseComponent`	`getParseComponent()`
`final UpdateComponent`	`getUpdateComponent()`
`final AppStatusTracker`	`getStatusTracker()`
`final CrawlLoop`	`getCrawlLoop()`
`final ImmutableConfig`	`getImmutableConfig()`
`final CheckState`	`fetchState(WebPage page, LoadOptions options)`
`final WebPage`	`load(String configuredUrl)`	Load an url, options can be specified following the url, see LoadOptions for all options
`final WebPage`	`load(String url, String args)`	Load an url with specified options, see LoadOptions for all options
`final WebPage`	`load(String url, LoadOptions options)`	Load an url with specified options If there is no page in local storage nor at the given remote location, WebPage.NIL is returned
`final WebPage`	`load(URL url, LoadOptions options)`
`final WebPage`	`load(GHypeLink link, LoadOptions options)`	Load an url in GHypeLink format If there is no page in local storage nor at the given remote location, WebPage.NIL is returned
`final WebPage`	`load(NormUrl normUrl)`
`final WebPage`	`loadDeferred(NormUrl normUrl)`
`final WebPage`	`loadWithRetry(NormUrl normUrl)`
`final WebPage`	`loadWithRetryDeferred(NormUrl normUrl)`
`final Collection<WebPage>`	`loadAll(Iterable<NormUrl> normUrls, LoadOptions options)`
`final Unit`	`flush()`
`Unit`	`close()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LoadComponent

LoadComponent(WebDb webDb, GlobalCacheFactory globalCacheFactory, BatchFetchComponent fetchComponent, ParseComponent parseComponent, UpdateComponent updateComponent, ImmutableConfig immutableConfig)

LoadComponent

LoadComponent(WebDb webDb, GlobalCacheFactory globalCacheFactory, BatchFetchComponent fetchComponent, ParseComponent parseComponent, UpdateComponent updateComponent, AppStatusTracker statusTracker, CrawlLoop crawlLoop, ImmutableConfig immutableConfig)

Method Detail

getGlobalCache
```
 final GlobalCache getGlobalCache()
```

getPageCache

 final ConcurrentExpiringLRUCache<String, WebPage> getPageCache()

getDocumentCache

 final ConcurrentExpiringLRUCache<String, FeaturedDocument> getDocumentCache()

getMetadataFields

 final Array<String> getMetadataFields()

getLastPageReport
```
 final String getLastPageReport()
```

setLastPageReport

 final Unit setLastPageReport(String lastPageReport)

getWebDb
```
 final WebDb getWebDb()
```

getGlobalCacheFactory

 final GlobalCacheFactory getGlobalCacheFactory()

getFetchComponent

 final BatchFetchComponent getFetchComponent()

getParseComponent

 final ParseComponent getParseComponent()

getUpdateComponent

 final UpdateComponent getUpdateComponent()

getStatusTracker

 final AppStatusTracker getStatusTracker()

getCrawlLoop
```
 final CrawlLoop getCrawlLoop()
```

getImmutableConfig

 final ImmutableConfig getImmutableConfig()

fetchState

 final CheckState fetchState(WebPage page, LoadOptions options)

load
```
 final WebPage load(String configuredUrl)
```
Load an url, options can be specified following the url, see LoadOptions for all options

Parameters:

configuredUrl - The url followed by options

load
```
 final WebPage load(String url, String args)
```
Load an url with specified options, see LoadOptions for all options

Parameters:

url - The url to load

load
```
 final WebPage load(String url, LoadOptions options)
```
Load an url with specified options If there is no page in local storage nor at the given remote location, WebPage.NIL is returned

Parameters:

url - The url to load

options - The options

load

 final WebPage load(URL url, LoadOptions options)

load
```
 final WebPage load(GHypeLink link, LoadOptions options)
```
Load an url in GHypeLink format If there is no page in local storage nor at the given remote location, WebPage.NIL is returned

Parameters:

link - The url in GHypeLink format to load

options - The options

load
```
 final WebPage load(NormUrl normUrl)
```

loadDeferred

 final WebPage loadDeferred(NormUrl normUrl)

loadWithRetry

 final WebPage loadWithRetry(NormUrl normUrl)

loadWithRetryDeferred

 final WebPage loadWithRetryDeferred(NormUrl normUrl)

loadAll

 final Collection<WebPage> loadAll(Iterable<NormUrl> normUrls, LoadOptions options)

flush
```
 final Unit flush()
```

close
```
 Unit close()
```

Class LoadComponent

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

LoadComponent

LoadComponent

Method Detail

getGlobalCache

getPageCache

getDocumentCache

getMetadataFields

getLastPageReport

setLastPageReport

getWebDb

getGlobalCacheFactory

getFetchComponent

getParseComponent

getUpdateComponent

getStatusTracker

getCrawlLoop

getImmutableConfig

fetchState

load

load

load

load

load

load

loadDeferred

loadWithRetry

loadWithRetryDeferred

loadAll

flush

close