Class WebDriverPoolManager
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.config.Parameterized,java.lang.AutoCloseable
public class WebDriverPoolManager extends PreemptChannelSupport implements Parameterized, AutoCloseableThe web driver pool manager.
The web driver pool manager provides web drivers to run web page fetch tasks.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classWebDriverPoolManager.Companion
-
Field Summary
Fields Modifier and Type Field Description private final BooleanisActiveprivate final BooleanisReadyprivate final DurationidleTimeoutprivate final Map<BrowserId, LoadingWebDriverPool>workingDriverPoolsprivate final Map<BrowserId, LoadingWebDriverPool>retiredDriverPoolsprivate final Set<BrowserId>closedDriverPoolsprivate final InstantstartTimeprivate InstantlastActiveTimeprivate final DurationidleTimeprivate final BooleanisIdleprivate final <Error class: unknown class>numWaitingTasksprivate final <Error class: unknown class>numStandbyDriversprivate final <Error class: unknown class>numWorkingDriversprivate final <Error class: unknown class>numAvailableDriverSlotsprivate final <Error class: unknown class>numActiveDriversprivate final <Error class: unknown class>numDyingDriversprivate final IntegernumClosedDriversprivate final IntegermaxAllowedDyingDriversprivate final <Error class: unknown class>numResetprivate final <Error class: unknown class>numTimeoutprivate final <Error class: unknown class>gaugesprivate final BrowserManagerbrowserManagerprivate final BrowserFactorybrowserFactoryprivate final ImmutableConfigimmutableConfigprivate final BooleansuppressMetricsprivate final BooleanhasEventprivate final BooleanisNormalprivate final BooleanisPreemptedprivate final Stringnamepublic final static WebDriverPoolManager.CompanionCompanion
-
Constructor Summary
Constructors Constructor Description WebDriverPoolManager(BrowserManager browserManager, BrowserFactory browserFactory, ImmutableConfig immutableConfig, Boolean suppressMetrics)
-
Method Summary
Modifier and Type Method Description final BooleanisActive()final BooleanisReady()final DurationgetIdleTimeout()final Map<BrowserId, LoadingWebDriverPool>getWorkingDriverPools()final Map<BrowserId, LoadingWebDriverPool>getRetiredDriverPools()final Set<BrowserId>getClosedDriverPools()final InstantgetStartTime()final InstantgetLastActiveTime()final UnitsetLastActiveTime(Instant lastActiveTime)final DurationgetIdleTime()final BooleanisIdle()final <Error class: unknown class>getNumWaitingTasks()final <Error class: unknown class>getNumStandbyDrivers()final <Error class: unknown class>getNumWorkingDrivers()final <Error class: unknown class>getNumAvailableDriverSlots()final <Error class: unknown class>getNumActiveDrivers()final <Error class: unknown class>getNumDyingDrivers()final IntegergetNumClosedDrivers()final IntegergetMaxAllowedDyingDrivers()Maximum allowed number of dying drives, if it's exceeded, the oldest driver pool should be closed. final <Error class: unknown class>getNumReset()final <Error class: unknown class>getNumTimeout()final <Error class: unknown class>getGauges()final BrowserManagergetBrowserManager()final BrowserFactorygetBrowserFactory()final ImmutableConfiggetImmutableConfig()final BooleangetSuppressMetrics()final BooleanhasPossibility(BrowserId browserId)Check if the browser has possibility to serve tasks. final IntegerpromisedDriverCount(BrowserId browserId)Get the number of drivers which can serve tasks in the pool. final BooleanhasDriverPromise(BrowserId browserId)Check if the pool has at least one driver to serve. final BooleanisFullCapacity(BrowserId browserId)Check if the pool is on full capacity. final WebDriversubscribeDriver(BrowserId browserId)Subscribe a web driver in the pool specified by browserId, the other subscriber should not use the driver. final WebDriversubscribeDriver()Subscribe a web driver, the other subscriber should not use the driver. final <Error class: unknown class>isRetiredPool(BrowserId browserId)Check if a pool is retired. final FetchResultrun(BrowserId browserId, FetchTask task, SuspendFunction1<WebDriver, FetchResult> browseFun)reactor: tell me if you can do this job pro-actor: here is a job, tell me if you finished it final FetchResultrun(WebDriverTask task)Run the task. final BooleanisActivePool(BrowserId browserId)Check if a pool is retired. final WebDrivercancel(String url)Cancel the fetch task specified by url remotely. final WebDrivercancel(BrowserId browserId, String url)Cancel the fetch task specified by url remotely. final UnitcancelAll()Cancel all the fetch tasks, stop loading all pages. final UnitcancelAll(BrowserId browserId)Cancel all the fetch tasks, stop loading all pages. final LoadingWebDriverPoolcreateUnmanagedDriverPool(BrowserId browserId)Create a driver pool, but the driver pool is not added to workingDriverPools. final UnitcloseBrowserAccompaniedDriverPoolGracefully(BrowserId browserId, Duration timeToWait)Find the driver pool with browserId, cancel all running tasks in it, and close all drivers in it. final Unitmaintain(Boolean force)Maintain all the driver pools, check and report inconsistency, illness, idleness, etc., close bad pools if necessary. final StringbuildStatusString(BrowserId browserId, Boolean verbose)Take a snapshot about the state of the driver pools. final StringbuildStatusString(Boolean verbose)Take a snapshot about the state of the driver pools. Unitclose()Close the web driver pool manager. StringtoString()Return a string to represent the snapshot of the status. -
Methods inherited from class ai.platon.pulsar.common.concurrent.PreemptChannelSupport
formatPreemptChannelStatus, getHasEvent, getName, isNormal, isPreempted, preempt, releaseLocks, whenNormal, whenNormalDeferred -
Methods inherited from class ai.platon.pulsar.common.config.Parameterized
getParams -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Method Detail
-
isActive
final Boolean isActive()
-
isReady
final Boolean isReady()
-
getIdleTimeout
final Duration getIdleTimeout()
-
getWorkingDriverPools
final Map<BrowserId, LoadingWebDriverPool> getWorkingDriverPools()
-
getRetiredDriverPools
final Map<BrowserId, LoadingWebDriverPool> getRetiredDriverPools()
-
getClosedDriverPools
final Set<BrowserId> getClosedDriverPools()
-
getStartTime
final Instant getStartTime()
-
getLastActiveTime
final Instant getLastActiveTime()
-
setLastActiveTime
final Unit setLastActiveTime(Instant lastActiveTime)
-
getIdleTime
final Duration getIdleTime()
-
isIdle
final Boolean isIdle()
-
getNumWaitingTasks
final <Error class: unknown class> getNumWaitingTasks()
-
getNumStandbyDrivers
final <Error class: unknown class> getNumStandbyDrivers()
-
getNumWorkingDrivers
final <Error class: unknown class> getNumWorkingDrivers()
-
getNumAvailableDriverSlots
final <Error class: unknown class> getNumAvailableDriverSlots()
-
getNumActiveDrivers
final <Error class: unknown class> getNumActiveDrivers()
-
getNumDyingDrivers
final <Error class: unknown class> getNumDyingDrivers()
-
getNumClosedDrivers
final Integer getNumClosedDrivers()
-
getMaxAllowedDyingDrivers
final Integer getMaxAllowedDyingDrivers()
Maximum allowed number of dying drives, if it's exceeded, the oldest driver pool should be closed. Dying drivers are kept for diagnosis.
-
getNumReset
final <Error class: unknown class> getNumReset()
-
getNumTimeout
final <Error class: unknown class> getNumTimeout()
-
getGauges
final <Error class: unknown class> getGauges()
-
getBrowserManager
final BrowserManager getBrowserManager()
-
getBrowserFactory
final BrowserFactory getBrowserFactory()
-
getImmutableConfig
final ImmutableConfig getImmutableConfig()
-
getSuppressMetrics
final Boolean getSuppressMetrics()
-
hasPossibility
final Boolean hasPossibility(BrowserId browserId)
Check if the browser has possibility to serve tasks.
the browser is not in any pool
the browser is in the working pool
- Parameters:
browserId- the browser id
-
promisedDriverCount
final Integer promisedDriverCount(BrowserId browserId)
Get the number of drivers which can serve tasks in the pool.
-
hasDriverPromise
final Boolean hasDriverPromise(BrowserId browserId)
Check if the pool has at least one driver to serve.
-
isFullCapacity
final Boolean isFullCapacity(BrowserId browserId)
Check if the pool is on full capacity.
-
subscribeDriver
@Beta() final WebDriver subscribeDriver(BrowserId browserId)
Subscribe a web driver in the pool specified by browserId, the other subscriber should not use the driver.
-
subscribeDriver
@Beta() final WebDriver subscribeDriver()
Subscribe a web driver, the other subscriber should not use the driver.
-
isRetiredPool
final <Error class: unknown class> isRetiredPool(BrowserId browserId)
Check if a pool is retired.
-
run
final FetchResult run(BrowserId browserId, FetchTask task, SuspendFunction1<WebDriver, FetchResult> browseFun)
reactor: tell me if you can do this job pro-actor: here is a job, tell me if you finished it
- Returns:
The result of action, or null if timeout
-
run
final FetchResult run(WebDriverTask task)
Run the task.
-
isActivePool
final Boolean isActivePool(BrowserId browserId)
Check if a pool is retired.
-
cancel
final WebDriver cancel(String url)
Cancel the fetch task specified by url remotely. NOTE: A cancel request should run immediately not waiting for any browser task return.
-
cancel
final WebDriver cancel(BrowserId browserId, String url)
Cancel the fetch task specified by url remotely. NOTE: A cancel request should run immediately not waiting for any browser task return.
-
cancelAll
final Unit cancelAll(BrowserId browserId)
Cancel all the fetch tasks, stop loading all pages.
-
createUnmanagedDriverPool
final LoadingWebDriverPool createUnmanagedDriverPool(BrowserId browserId)
Create a driver pool, but the driver pool is not added to workingDriverPools.
-
closeBrowserAccompaniedDriverPoolGracefully
final Unit closeBrowserAccompaniedDriverPoolGracefully(BrowserId browserId, Duration timeToWait)
Find the driver pool with browserId, cancel all running tasks in it, and close all drivers in it. When we close a driver pool, all threads that are trying to get a driver should wait. When the driver pool is closed, all the drivers in it will be closed, and the browser associated with the driver pool will be closed as well.
-
maintain
final Unit maintain(Boolean force)
Maintain all the driver pools, check and report inconsistency, illness, idleness, etc., close bad pools if necessary.
If "takeDriverPoolSnapshot" is in file AppPaths.PATH_LOCAL_COMMAND, perform the action.
If the tmp dir is the default one, run the following command to take snapshot once: echo takeDriverPoolSnapshot /tmp/pulsar/pulsar-commands
-
buildStatusString
final String buildStatusString(BrowserId browserId, Boolean verbose)
Take a snapshot about the state of the driver pools.
-
buildStatusString
final String buildStatusString(Boolean verbose)
Take a snapshot about the state of the driver pools.
-
close
Unit close()
Close the web driver pool manager. All deferred tasks will be canceled, the pool of driver pools will be cleared and the browser manager will be closed.
It happens when the program exits.
The closing call stack:
PrivacyContextManager.close -> PrivacyContext.close -> WebDriverContext.close -> WebDriverPoolManager.close -> BrowserManager.close -> Browser.close -> WebDriver.close |-> LoadingWebDriverPool.close
-
toString
String toString()
Return a string to represent the snapshot of the status.
-
-
-
-