Interface PrivacyContext

  • All Implemented Interfaces:
    java.lang.AutoCloseable

    
    public interface PrivacyContext
     implements AutoCloseable
                        

    A privacy context is a unique context of a privacy agent to the target website, it will be closed once it is leaked.

    One of the biggest difficulties in web scraping tasks is the bot stealth.

    For web scraping tasks, the website should have no idea whether a visit is from a human being or a bot. Once a page visit is suspected by the website, which we call a privacy leak, the privacy context has to be dropped, and PulsarRPA will visit the page in another privacy context.

    • Constructor Detail

    • Method Detail

      • promisedWebDriverCount

         abstract Integer promisedWebDriverCount()

        The promised workers (free web drivers) count.

        The implementation has to tell the caller how many workers (free web drivers) it can provide.

        The number of workers can change immediately after reading, so the caller only has promises but no guarantees.

        Returns:

        the number of workers promised.

      • hasWebDriverPromise

         abstract Boolean hasWebDriverPromise()

        Check if the privacy context promises at least one worker to provide.

      • open

         abstract FetchResult open(String url)

        Open a page in the privacy context.

        Parameters:
        url - The URL to open.
        Returns:

        The fetch result.

      • open

         abstract FetchResult open(String url, LoadOptions options)

        Open a page in the privacy context.

        Parameters:
        url - The URL to open.
        options - The load options to use.
        Returns:

        The fetch result.

      • dismiss

         abstract Unit dismiss()

        Dismiss the privacy context.

        This method is used to dismiss the privacy context. It should be called when the privacy context is no longer needed.

      • maintain

         abstract Unit maintain()

        Maintain the privacy context.

        This method is used to maintain the privacy context. It should be called periodically to keep the privacy context alive.

      • buildReport

         abstract String buildReport()

        Build the privacy context report.

        This method is used to build the privacy context report. It should be called periodically to keep the privacy context alive.

      • isFullCapacity

         abstract Boolean isFullCapacity()

        Check whether the privacy context is at full capacity. If the privacy context is indeed at full capacity, it should not be used for processing new tasks, and the underlying services may potentially refuse to provide service.

        A privacy context is running at full capacity when the underlying webdriver pool is full capacity, so the webdriver pool can not provide a webdriver for new tasks.

        Note that if a driver pool is retired or closed, it's not full capacity.

        Returns:

        True if the privacy context is running at full capacity, false otherwise.

      • isUnderLoaded

         abstract Boolean isUnderLoaded()

        Check whether the privacy context is running under loaded.

      • isIdle

         abstract Boolean isIdle()

        Check whether the privacy context is idle.

      • isRetired

         abstract Boolean isRetired()

        Check whether the privacy context is retired.

        This property indicates whether the privacy context has been marked as retired. It returns the value of the retired field, which is a boolean indicating the retirement status.

        Returns:

        true if the privacy context is retired, false otherwise.

      • isLeaked

         abstract Boolean isLeaked()

        Check whether the privacy context is leaked.

        This property indicates whether the privacy context has been marked as leaked. It returns the value of the leaked field, which is a boolean indicating the leakage status.

        Returns:

        true if the privacy context is leaked, false otherwise.

      • isGood

         abstract Boolean isGood()

        Check whether the privacy context is good.

        A good privacy context has to meet the following requirements:

        • the fetch speed is good

      • isActive

         abstract Boolean isActive()

        Check whether the privacy context is active.

        TODO: check and distinct from isReady in use cases.

        An active privacy context has to meet the following requirements:

        • not closed

        • not leaked

        • not retired

        Note: this flag does not guarantee consistency, and can change immediately after it's read

      • isClosed

         abstract Boolean isClosed()

        Check whether the privacy context is closed.

      • isReady

         abstract Boolean isReady()

        A ready privacy context is ready to serve tasks.

        A ready privacy context has to meet the following requirements:

        • not closed

        • not leaked

        • requirement removed not idle

        • not retired

        • if there is a proxy, the proxy has to be ready

        • the associated driver pool promises to provide an available driver, ether one of the following:

        Note: this flag does not guarantee consistency, and can change immediately after it's read