Interface PrivacyContext
-
- All Implemented Interfaces:
-
java.lang.AutoCloseable
public interface PrivacyContext implements AutoCloseable
A privacy context is a unique context of a privacy agent to the target website, it will be closed once it is leaked.
One of the biggest difficulties in web scraping tasks is the bot stealth.
For web scraping tasks, the website should have no idea whether a visit is from a human being or a bot. Once a page visit is suspected by the website, which we call a privacy leak, the privacy context has to be dropped, and PulsarRPA will visit the page in another privacy context.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description public classPrivacyContext.Companion
-
Method Summary
Modifier and Type Method Description abstract StringbuildStatusString()Build the privacy context status string. abstract IntegerpromisedWebDriverCount()The promised workers (free web drivers) count. abstract BooleanhasWebDriverPromise()Check if the privacy context promises at least one worker to provide. abstract FetchResultopen(String url)Open a page in the privacy context. abstract FetchResultopen(String url, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)Open a page in the privacy context. abstract FetchResultopen(String url, LoadOptions options)Open a page in the privacy context. abstract FetchResultrun(FetchTask task, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)Run a task in the privacy context. abstract FetchResultdoRun(FetchTask task, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)Run a task in the privacy context. abstract Unitdismiss()Dismiss the privacy context. abstract Unitmaintain()Maintain the privacy context. abstract StringbuildReport()Build the privacy context report. abstract PrivacyAgentIdgetId()The privacy context id. abstract PrivacyAgentgetPrivacyAgent()The associated privacy agent. abstract BooleanisFullCapacity()Check whether the privacy context is at full capacity. abstract BooleanisUnderLoaded()Check whether the privacy context is running under loaded. abstract BooleanisIdle()Check whether the privacy context is idle. abstract BooleanisRetired()Check whether the privacy context is retired. abstract BooleanisLeaked()Check whether the privacy context is leaked. abstract BooleanisGood()Check whether the privacy context is good. abstract BooleanisActive()Check whether the privacy context is active. abstract BooleanisClosed()Check whether the privacy context is closed. abstract BooleanisReady()A ready privacy context is ready to serve tasks. abstract FloatgetFailureRate()The failure rate of the privacy context. abstract BooleanisHighFailureRate()Check whether the failure rate is high. abstract DurationgetIdleTime()The idle time of the privacy context. abstract DurationgetElapsedTime()The elapsed time of the privacy context. abstract StringgetDisplay()A readable privacy context display. abstract StringgetReadableState()Get the readable privacy context state. -
-
Method Detail
-
buildStatusString
abstract String buildStatusString()
Build the privacy context status string.
-
promisedWebDriverCount
abstract Integer promisedWebDriverCount()
The promised workers (free web drivers) count.
The implementation has to tell the caller how many workers (free web drivers) it can provide.
The number of workers can change immediately after reading, so the caller only has promises but no guarantees.
- Returns:
the number of workers promised.
-
hasWebDriverPromise
abstract Boolean hasWebDriverPromise()
Check if the privacy context promises at least one worker to provide.
-
open
abstract FetchResult open(String url)
Open a page in the privacy context.
- Parameters:
url- The URL to open.- Returns:
The fetch result.
-
open
abstract FetchResult open(String url, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)
Open a page in the privacy context.
- Parameters:
url- The URL to open.fetchFun- The fetch function to use.- Returns:
The fetch result.
-
open
abstract FetchResult open(String url, LoadOptions options)
Open a page in the privacy context.
- Parameters:
url- The URL to open.options- The load options to use.- Returns:
The fetch result.
-
run
abstract FetchResult run(FetchTask task, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)
Run a task in the privacy context.
- Parameters:
task- The task to run.fetchFun- The fetch function to use.- Returns:
The fetch result.
-
doRun
abstract FetchResult doRun(FetchTask task, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)
Run a task in the privacy context.
- Parameters:
task- The task to run.fetchFun- The fetch function to use.- Returns:
The fetch result.
-
dismiss
abstract Unit dismiss()
Dismiss the privacy context.
This method is used to dismiss the privacy context. It should be called when the privacy context is no longer needed.
-
maintain
abstract Unit maintain()
Maintain the privacy context.
This method is used to maintain the privacy context. It should be called periodically to keep the privacy context alive.
-
buildReport
abstract String buildReport()
Build the privacy context report.
This method is used to build the privacy context report. It should be called periodically to keep the privacy context alive.
-
getId
abstract PrivacyAgentId getId()
The privacy context id.
-
getPrivacyAgent
abstract PrivacyAgent getPrivacyAgent()
The associated privacy agent.
-
isFullCapacity
abstract Boolean isFullCapacity()
Check whether the privacy context is at full capacity. If the privacy context is indeed at full capacity, it should not be used for processing new tasks, and the underlying services may potentially refuse to provide service.
A privacy context is running at full capacity when the underlying webdriver pool is full capacity, so the webdriver pool can not provide a webdriver for new tasks.
Note that if a driver pool is retired or closed, it's not full capacity.
- Returns:
True if the privacy context is running at full capacity, false otherwise.
-
isUnderLoaded
abstract Boolean isUnderLoaded()
Check whether the privacy context is running under loaded.
-
isRetired
abstract Boolean isRetired()
Check whether the privacy context is retired.
This property indicates whether the privacy context has been marked as retired. It returns the value of the
retiredfield, which is a boolean indicating the retirement status.- Returns:
trueif the privacy context is retired,falseotherwise.
-
isLeaked
abstract Boolean isLeaked()
Check whether the privacy context is leaked.
This property indicates whether the privacy context has been marked as leaked. It returns the value of the
leakedfield, which is a boolean indicating the leakage status.- Returns:
trueif the privacy context is leaked,falseotherwise.
-
isGood
abstract Boolean isGood()
Check whether the privacy context is good.
A good privacy context has to meet the following requirements:
the fetch speed is good
-
isActive
abstract Boolean isActive()
Check whether the privacy context is active.
TODO: check and distinct from isReady in use cases.
An active privacy context has to meet the following requirements:
not closed
not leaked
not retired
Note: this flag does not guarantee consistency, and can change immediately after it's read
-
isReady
abstract Boolean isReady()
A ready privacy context is ready to serve tasks.
A ready privacy context has to meet the following requirements:
not closed
not leaked
requirement removed not idle
not retired
if there is a proxy, the proxy has to be ready
the associated driver pool promises to provide an available driver, ether one of the following:
Note: this flag does not guarantee consistency, and can change immediately after it's read
-
getFailureRate
abstract Float getFailureRate()
The failure rate of the privacy context.
-
isHighFailureRate
abstract Boolean isHighFailureRate()
Check whether the failure rate is high.
-
getIdleTime
abstract Duration getIdleTime()
The idle time of the privacy context.
-
getElapsedTime
abstract Duration getElapsedTime()
The elapsed time of the privacy context.
-
getDisplay
abstract String getDisplay()
A readable privacy context display.
-
getReadableState
abstract String getReadableState()
Get the readable privacy context state.
-
-
-
-