Class WebDriverPoolManager

  • All Implemented Interfaces:
    ai.platon.pulsar.common.config.Parameterized , java.lang.AutoCloseable

    
    public class WebDriverPoolManager
    extends PreemptChannelSupport implements Parameterized, AutoCloseable
                        

    The web driver pool manager.

    The web driver pool manager provides web drivers to run web page fetch tasks.

    • Constructor Detail

      • WebDriverPoolManager

        WebDriverPoolManager(BrowserManager browserManager, BrowserFactory browserFactory, ImmutableConfig immutableConfig, Boolean suppressMetrics)
    • Method Detail

      • isIdle

         final Boolean isIdle()
      • getMaxAllowedDyingDrivers

         final Integer getMaxAllowedDyingDrivers()

        Maximum allowed number of dying drives, if it's exceeded, the oldest driver pool should be closed. Dying drivers are kept for diagnosis.

      • getNumReset

         final <Error class: unknown class> getNumReset()
      • getNumTimeout

         final <Error class: unknown class> getNumTimeout()
      • getGauges

         final <Error class: unknown class> getGauges()
      • hasPossibility

         final Boolean hasPossibility(BrowserId browserId)

        Check if the browser has possibility to serve tasks.

        • the browser is not in any pool

        • the browser is in the working pool

        Parameters:
        browserId - the browser id
      • promisedDriverCount

         final Integer promisedDriverCount(BrowserId browserId)

        Get the number of drivers which can serve tasks in the pool.

      • hasDriverPromise

         final Boolean hasDriverPromise(BrowserId browserId)

        Check if the pool has at least one driver to serve.

      • isFullCapacity

         final Boolean isFullCapacity(BrowserId browserId)

        Check if the pool is on full capacity.

      • subscribeDriver

        @Beta() final WebDriver subscribeDriver(BrowserId browserId)

        Subscribe a web driver in the pool specified by browserId, the other subscriber should not use the driver.

      • subscribeDriver

        @Beta() final WebDriver subscribeDriver()

        Subscribe a web driver, the other subscriber should not use the driver.

      • isRetiredPool

         final <Error class: unknown class> isRetiredPool(BrowserId browserId)

        Check if a pool is retired.

      • run

         final FetchResult run(BrowserId browserId, FetchTask task, SuspendFunction1<WebDriver, FetchResult> browseFun)

        reactor: tell me if you can do this job pro-actor: here is a job, tell me if you finished it

        Returns:

        The result of action, or null if timeout

      • isActivePool

         final Boolean isActivePool(BrowserId browserId)

        Check if a pool is retired.

      • cancel

         final WebDriver cancel(String url)

        Cancel the fetch task specified by url remotely. NOTE: A cancel request should run immediately not waiting for any browser task return.

      • cancel

         final WebDriver cancel(BrowserId browserId, String url)

        Cancel the fetch task specified by url remotely. NOTE: A cancel request should run immediately not waiting for any browser task return.

      • cancelAll

         final Unit cancelAll()

        Cancel all the fetch tasks, stop loading all pages.

      • cancelAll

         final Unit cancelAll(BrowserId browserId)

        Cancel all the fetch tasks, stop loading all pages.

      • closeBrowserAccompaniedDriverPoolGracefully

         final Unit closeBrowserAccompaniedDriverPoolGracefully(BrowserId browserId, Duration timeToWait)

        Find the driver pool with browserId, cancel all running tasks in it, and close all drivers in it. When we close a driver pool, all threads that are trying to get a driver should wait. When the driver pool is closed, all the drivers in it will be closed, and the browser associated with the driver pool will be closed as well.

      • maintain

         final Unit maintain(Boolean force)

        Maintain all the driver pools, check and report inconsistency, illness, idleness, etc., close bad pools if necessary.

        If "takeDriverPoolSnapshot" is in file AppPaths.PATH_LOCAL_COMMAND, perform the action.

        If the tmp dir is the default one, run the following command to take snapshot once: echo takeDriverPoolSnapshot /tmp/pulsar/pulsar-commands

      • buildStatusString

         final String buildStatusString(BrowserId browserId, Boolean verbose)

        Take a snapshot about the state of the driver pools.

      • buildStatusString

         final String buildStatusString(Boolean verbose)

        Take a snapshot about the state of the driver pools.

      • close

         Unit close()

        Close the web driver pool manager. All deferred tasks will be canceled, the pool of driver pools will be cleared and the browser manager will be closed.

        It happens when the program exits.

        The closing call stack:

        PrivacyContextManager.close -> PrivacyContext.close -> WebDriverContext.close -> WebDriverPoolManager.close -> BrowserManager.close -> Browser.close -> WebDriver.close |-> LoadingWebDriverPool.close

      • toString

         String toString()

        Return a string to represent the snapshot of the status.