Class AbstractPrivacyManager
-
- All Implemented Interfaces:
-
ai.platon.pulsar.skeleton.crawl.fetch.privacy.PrivacyManager,java.lang.AutoCloseable
public abstract class AbstractPrivacyManager implements PrivacyManager
Manages the lifecycle of privacy contexts, including permanent and temporary contexts. Permanent contexts have a long lifecycle and are never deleted, while temporary contexts are short-lived and discarded if a privacy leak is detected.
-
-
Field Summary
Fields Modifier and Type Field Description private final ConcurrentHashMap<PrivacyAgent, PrivacyContext>permanentContextsprivate final ConcurrentHashMap<PrivacyAgent, PrivacyContext>temporaryContextsprivate final Map<PrivacyAgent, PrivacyContext>activeContextsprivate final ConcurrentLinkedDeque<PrivacyContext>zombieContextsprivate final ConcurrentLinkedDeque<PrivacyContext>deadContextsprivate final ObjectcontextLifeCycleMonitorprivate final PrivacyAgentGeneratorprivacyAgentGeneratorprivate final BooleanisClosedprivate final BooleanisActiveprivate final ImmutableConfigconf
-
Constructor Summary
Constructors Constructor Description AbstractPrivacyManager(ImmutableConfig conf)
-
Method Summary
Modifier and Type Method Description final ConcurrentHashMap<PrivacyAgent, PrivacyContext>getPermanentContexts()Permanent contexts have a long lifecycle and are never deleted by the system. final ConcurrentHashMap<PrivacyAgent, PrivacyContext>getTemporaryContexts()Temporary contexts have a short lifecycle and are discarded if a privacy leak is detected. final Map<PrivacyAgent, PrivacyContext>getActiveContexts()final ConcurrentLinkedDeque<PrivacyContext>getZombieContexts()Zombie contexts are contexts that are no longer active but have not yet been fully closed. final ConcurrentLinkedDeque<PrivacyContext>getDeadContexts()Dead contexts are contexts that have been fully closed and their resources released. final ObjectgetContextLifeCycleMonitor()Monitor object used to synchronize operations on contexts. PrivacyAgentGeneratorgetPrivacyAgentGenerator()BooleanisClosed()Indicates whether the privacy manager is closed. BooleanisActive()Indicates whether the privacy manager is active. ImmutableConfiggetConf()abstract FetchResultrun(FetchTask task, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)Runs a fetch task within a privacy context. abstract PrivacyContexttryGetNextReadyPrivacyContext(WebPage page, Fingerprint fingerprint, FetchTask task)Attempts to get the next ready privacy context for a given page, fingerprint, and task. abstract PrivacyContexttryGetNextReadyPrivacyContext(Fingerprint fingerprint)Attempts to get the next ready privacy context for a given fingerprint. abstract PrivacyContexttryGetNextUnderLoadedPrivacyContext(WebPage page, Fingerprint fingerprint, FetchTask task)Attempts to get the next under-loaded privacy context for a given page, fingerprint, and task. abstract PrivacyContextgetOrCreate(PrivacyAgent privacyAgent)Gets or creates a privacy context for the given privacy agent. abstract PrivacyContextcreateUnmanagedContext(PrivacyAgent privacyAgent)Creates an unmanaged privacy context for the given privacy agent. PrivacyContextcreateUnmanagedContext(PrivacyAgent privacyAgent, WebDriverFetcher fetcher)Creates an unmanaged privacy context for the given privacy agent and fetcher. StringbuildStatusString()Builds a status string summarizing the current state of active contexts. Unitmaintain(Boolean force)Performs maintenance tasks on the privacy manager. Unitclose(PrivacyContext privacyContext)Closes a given privacy context, moving it from the active list to the zombie list. Unitclose()Closes the privacy manager, including all active contexts. Unitreset(String reason)Resets the privacy environment by closing all privacy contexts. -
-
Method Detail
-
getPermanentContexts
final ConcurrentHashMap<PrivacyAgent, PrivacyContext> getPermanentContexts()
Permanent contexts have a long lifecycle and are never deleted by the system. Predefined privacy agents for permanent contexts include:
PrivacyAgent.SYSTEM_DEFAULT
PrivacyAgent.PROTOTYPE
PrivacyAgent.DEFAULT
PrivacyAgent.NEXT_SEQUENTIAL
-
getTemporaryContexts
final ConcurrentHashMap<PrivacyAgent, PrivacyContext> getTemporaryContexts()
Temporary contexts have a short lifecycle and are discarded if a privacy leak is detected. A new context is created to replace the leaked one.
-
getActiveContexts
final Map<PrivacyAgent, PrivacyContext> getActiveContexts()
-
getZombieContexts
final ConcurrentLinkedDeque<PrivacyContext> getZombieContexts()
Zombie contexts are contexts that are no longer active but have not yet been fully closed.
-
getDeadContexts
final ConcurrentLinkedDeque<PrivacyContext> getDeadContexts()
Dead contexts are contexts that have been fully closed and their resources released.
-
getContextLifeCycleMonitor
final Object getContextLifeCycleMonitor()
Monitor object used to synchronize operations on contexts.
-
getPrivacyAgentGenerator
PrivacyAgentGenerator getPrivacyAgentGenerator()
-
getConf
ImmutableConfig getConf()
-
run
abstract FetchResult run(FetchTask task, SuspendFunction2<FetchTask, WebDriver, FetchResult> fetchFun)
Runs a fetch task within a privacy context.
- Parameters:
task- The fetch task to execute.fetchFun- The function to execute the fetch task.- Returns:
The result of the fetch task.
-
tryGetNextReadyPrivacyContext
abstract PrivacyContext tryGetNextReadyPrivacyContext(WebPage page, Fingerprint fingerprint, FetchTask task)
Attempts to get the next ready privacy context for a given page, fingerprint, and task.
- Parameters:
page- The web page associated with the context.fingerprint- The fingerprint used to identify the context.task- The fetch task associated with the context.- Returns:
The next ready privacy context.
-
tryGetNextReadyPrivacyContext
abstract PrivacyContext tryGetNextReadyPrivacyContext(Fingerprint fingerprint)
Attempts to get the next ready privacy context for a given fingerprint.
- Parameters:
fingerprint- The fingerprint used to identify the context.- Returns:
The next ready privacy context.
-
tryGetNextUnderLoadedPrivacyContext
abstract PrivacyContext tryGetNextUnderLoadedPrivacyContext(WebPage page, Fingerprint fingerprint, FetchTask task)
Attempts to get the next under-loaded privacy context for a given page, fingerprint, and task.
- Parameters:
page- The web page associated with the context.fingerprint- The fingerprint used to identify the context.task- The fetch task associated with the context.- Returns:
The next under-loaded privacy context, or null if none is available.
-
getOrCreate
abstract PrivacyContext getOrCreate(PrivacyAgent privacyAgent)
Gets or creates a privacy context for the given privacy agent.
- Parameters:
privacyAgent- The privacy agent associated with the context.- Returns:
The privacy context.
-
createUnmanagedContext
abstract PrivacyContext createUnmanagedContext(PrivacyAgent privacyAgent)
Creates an unmanaged privacy context for the given privacy agent.
- Parameters:
privacyAgent- The privacy agent associated with the context.- Returns:
The unmanaged privacy context.
-
createUnmanagedContext
PrivacyContext createUnmanagedContext(PrivacyAgent privacyAgent, WebDriverFetcher fetcher)
Creates an unmanaged privacy context for the given privacy agent and fetcher.
- Parameters:
privacyAgent- The privacy agent associated with the context.fetcher- The web driver fetcher used to create the context.- Returns:
The unmanaged privacy context.
-
buildStatusString
String buildStatusString()
Builds a status string summarizing the current state of active contexts.
- Returns:
A string representation of the active contexts' status.
-
maintain
Unit maintain(Boolean force)
Performs maintenance tasks on the privacy manager.
- Parameters:
force- If true, forces maintenance tasks to run.
-
close
Unit close(PrivacyContext privacyContext)
Closes a given privacy context, moving it from the active list to the zombie list.
- Parameters:
privacyContext- The privacy context to close.
-
-
-
-