All Implemented Interfaces:: ai.platon.pulsar.crawl.fetch.driver.JvmWebDriver , ai.platon.pulsar.crawl.fetch.driver.WebDriver , java.io.Closeable , java.lang.AutoCloseable , kotlin.Comparable


public abstract class AbstractWebDriver
extends AbstractJvmWebDriver implements Comparable<AbstractWebDriver>, WebDriver, JvmWebDriver

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`public class`	`AbstractWebDriver.Companion`

Field Summary

Fields
Modifier and Type	Field	Description
`private Duration`	`idleTimeout`
`private Duration`	`waitForElementTimeout`
`private final String`	`name`
`private final Function1<String, Long>`	`delayPolicy`
`private NavigateEntry`	`navigateEntry`
`private final List<NavigateEntry>`	`navigateHistory`
`private final Boolean`	`supportJavascript`
`private final Boolean`	`isMockedPageSource`
`private Boolean`	`isRecovered`
`private Boolean`	`isReused`
`private final AtomicReference<WebDriver.State>`	`state`
`private Instant`	`lastActiveTime`
`private final Boolean`	`isInit`
`private final Boolean`	`isReady`
`private final Boolean`	`isFree`
`private final Boolean`	`isWorking`
`private final Boolean`	`isRetired`
`private final Boolean`	`isQuit`
`private final Boolean`	`isCanceled`
`private final Boolean`	`isCrashed`
`private final String`	`sessionId`
`private final Browser`	`browser`
`private final Integer`	`id`
`private final BrowserType`	`browserType`
`private final Boolean`	`isIdle`

Constructor Summary

Constructors
Constructor	Description
`AbstractWebDriver(Browser browser, Integer id)`

Method Summary

Modifier and Type	Method	Description
`Duration`	`getIdleTimeout()`	The idle timeout.
`Unit`	`setIdleTimeout(Duration idleTimeout)`	The idle timeout.
`Duration`	`getWaitForElementTimeout()`	The timeout to wait for some object.
`Unit`	`setWaitForElementTimeout(Duration waitForElementTimeout)`	The timeout to wait for some object.
`String`	`getName()`
`Function1<String, Long>`	`getDelayPolicy()`
`NavigateEntry`	`getNavigateEntry()`	The current navigation entry.
`Unit`	`setNavigateEntry(NavigateEntry navigateEntry)`	The current navigation entry.
`List<NavigateEntry>`	`getNavigateHistory()`	The navigation history.
`Boolean`	`getSupportJavascript()`	Indicates whether the driver supports javascript.
`Boolean`	`getIsMockedPageSource()`	Indicate whether the page source is mocked.
`Boolean`	`getIsRecovered()`	Whether the driver is recovered from the browser's tab list.
`Unit`	`setIsRecovered(Boolean isRecovered)`	Whether the driver is recovered from the browser's tab list.
`Boolean`	`getIsReused()`	Whether the driver is recovered and is reused to serve new tasks.
`Unit`	`setIsReused(Boolean isReused)`	Whether the driver is recovered and is reused to serve new tasks.
`AtomicReference<WebDriver.State>`	`getState()`	The driver status.
`Instant`	`getLastActiveTime()`	The time of the last action.
`Unit`	`setLastActiveTime(Instant lastActiveTime)`	The time of the last action.
`Boolean`	`getIsInit()`
`Boolean`	`getIsReady()`
`Boolean`	`getIsFree()`
`Boolean`	`getIsWorking()`
`Boolean`	`getIsRetired()`
`Boolean`	`getIsQuit()`
`Boolean`	`getIsCanceled()`
`Boolean`	`getIsCrashed()`
`String`	`getSessionId()`
`Browser`	`getBrowser()`	The browser of the driver.
`Integer`	`getId()`	The unique driver id.
`abstract BrowserType`	`getBrowserType()`	The browser type.
`Boolean`	`getIsIdle()`
`Unit`	`free()`	Mark the driver as free, so it can be used to fetch a new page.
`Unit`	`startWork()`	Mark the driver as working, so it can not be used to fetch another page.
`Unit`	`retire()`	Mark the driver as retired, so it can not be used to fetch any page, and should be quit as soon as possible.
`Unit`	`cancel()`	Mark the driver as canceled, so the fetch process should return as soon as possible, and the fetch result should be dropped.
`JvmWebDriver`	`jvm()`	Returns a JvmWebDriver to support other JVM languages, such as java, clojure, scala, and so on, which had difficulty handling kotlin suspend methods.
`Unit`	`navigateTo(String url)`	Returns the main resource response.
`Long`	`waitForSelector(String selector, Long timeoutMillis)`	Returns when element specified by selector satisfies {@code state} option.
`Long`	`waitForSelector(String selector)`	Returns when element specified by selector satisfies {@code state} option.
`Long`	`waitForNavigation()`
`Long`	`waitForNavigation(Long timeoutMillis)`
`Object`	`evaluateSilently(String expression)`	Executes JavaScript in the context of the currently selected frame or window.
`Boolean`	`isVisible(String selector)`
`Boolean`	`isChecked(String selector)`
`Unit`	`scrollDown(Integer count)`
`Unit`	`scrollUp(Integer count)`
`Unit`	`scrollToTop()`
`Unit`	`scrollToBottom()`
`Unit`	`scrollToMiddle(Float ratio)`
`String`	`clickNthAnchor(Integer n, String rootSelector)`
`String`	`outerHTML(String selector)`
`String`	`firstText(String selector)`
`List<String>`	`allTexts(String selector)`
`String`	`firstAttr(String selector, String attrName)`
`List<String>`	`allAttrs(String selector, String attrName)`
`Unit`	`clickTextMatches(String selector, String pattern, Integer count)`
`Unit`	`clickMatches(String selector, String attrName, String pattern, Integer count)`
`Unit`	`check(String selector)`
`Unit`	`uncheck(String selector)`
`Connection`	`newSession()`	Create a new session with the same context of the browser: headers, cookies, proxy, etc.
`Connection.Response`	`loadResource(String url)`	Load url as a resource without browser rendering, with the last page's context, which means, the same headers and cookies.
`Unit`	`quit()`	Quit the browser instance
`Boolean`	`equals(Object other)`
`Integer`	`hashCode()`
`Integer`	`compareTo(AbstractWebDriver other)`
`String`	`toString()`

Methods inherited from class ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver
addBlockedURLs, addInitScript, addProbabilityBlockedURLs, awaitTermination, boundingBox, bringToFront, captureScreenshot, captureScreenshot, click, clickAsync, clickMatches, clickMatchesAsync, clickablePoint, currentUrl, dragAndDrop, evaluate, evaluateDetail, exists, focus, getCookies, isHidden, mainRequestCookies, mainRequestHeaders, mouseWheelDown, mouseWheelUp, moveMouseTo, moveMouseTo, navigateTo, pageSource, pause, scrollTo, setTimeouts, stop, stopLoading, terminate, type, visible, waitForNavigation, waitForPage, waitForSelector
Methods inherited from class ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver
addBlockedURLsAsync, addInitScriptAsync, allAttrsAsync, allTextsAsync, boundingBoxAsync, bringToFrontAsync, captureScreenshotAsync, captureScreenshotAsync, checkAsync, clickAsync, clickMatchesAsync, clickMatchesAsync, clickNthAnchorAsync, clickablePointAsync, currentUrlAsync, dragAndDropAsync, evaluateAsync, evaluateSilentlyAsync, existsAsync, firstAttrAsync, firstTextAsync, getCookiesAsync, isCheckedAsync, isHiddenAsync, isVisibleAsync, loadResourceAsync, mainRequestCookiesAsync, mainRequestHeadersAsync, mouseWheelDownAsync, mouseWheelUpAsync, moveMouseToAsync, navigateToAsync, navigateToAsync, newSessionAsync, outerHTMLAsync, pageSourceAsync, pauseAsync, scrollDownAsync, scrollToAsync, scrollToBottomAsync, scrollToMiddleAsync, scrollToTopAsync, scrollUpAsync, setTimeoutsAsync, stopAsync, terminateAsync, typeAsync, uncheckAsync, visibleAsync, waitForNavigationAsync, waitForNavigationAsync, waitForNavigationAsync, waitForSelectorAsync, waitForSelectorAsync, waitForSelectorAsync
Methods inherited from class ai.platon.pulsar.crawl.fetch.driver.WebDriver
close
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - AbstractWebDriver
```
AbstractWebDriver(Browser browser, Integer id)
```
- Method Detail
  - getIdleTimeout
```
 Duration getIdleTimeout()
```
    The idle timeout.
  - setIdleTimeout
```
 Unit setIdleTimeout(Duration idleTimeout)
```
    The idle timeout.
  - getWaitForElementTimeout
```
 Duration getWaitForElementTimeout()
```
    The timeout to wait for some object.
  - setWaitForElementTimeout
```
 Unit setWaitForElementTimeout(Duration waitForElementTimeout)
```
    The timeout to wait for some object.
  - getName
```
 String getName()
```
  - getDelayPolicy
```
 Function1<String, Long> getDelayPolicy()
```
  - getNavigateEntry
```
 NavigateEntry getNavigateEntry()
```
    The current navigation entry.
  - setNavigateEntry
```
 Unit setNavigateEntry(NavigateEntry navigateEntry)
```
    The current navigation entry.
  - getNavigateHistory
```
 List<NavigateEntry> getNavigateHistory()
```
    The navigation history.
  - getSupportJavascript
```
 Boolean getSupportJavascript()
```
    Indicates whether the driver supports javascript. Web drivers such as MockDriver do not support javascript.
  - getIsMockedPageSource
```
 Boolean getIsMockedPageSource()
```
    Indicate whether the page source is mocked.
  - getIsRecovered
```
 Boolean getIsRecovered()
```
    Whether the driver is recovered from the browser's tab list.
  - setIsRecovered
```
 Unit setIsRecovered(Boolean isRecovered)
```
    Whether the driver is recovered from the browser's tab list.
  - getIsReused
```
 Boolean getIsReused()
```
    Whether the driver is recovered and is reused to serve new tasks.
  - setIsReused
```
 Unit setIsReused(Boolean isReused)
```
    Whether the driver is recovered and is reused to serve new tasks.
  - getState
```
 AtomicReference<WebDriver.State> getState()
```
    The driver status.
  - getLastActiveTime
```
 Instant getLastActiveTime()
```
    The time of the last action.
  - setLastActiveTime
```
 Unit setLastActiveTime(Instant lastActiveTime)
```
    The time of the last action.
  - getIsInit
```
 Boolean getIsInit()
```
  - getIsReady
```
 Boolean getIsReady()
```
  - getIsFree
```
 Boolean getIsFree()
```
  - getIsWorking
```
 Boolean getIsWorking()
```
  - getIsRetired
```
 Boolean getIsRetired()
```
  - getIsQuit
```
 Boolean getIsQuit()
```
  - getIsCanceled
```
 Boolean getIsCanceled()
```
  - getIsCrashed
```
 Boolean getIsCrashed()
```
  - getSessionId
```
 String getSessionId()
```
  - getBrowser
```
 Browser getBrowser()
```
    The browser of the driver. The browser defines methods and events to manipulate a real browser.
  - getId
```
 Integer getId()
```
    The unique driver id.
  - getBrowserType
```
 abstract BrowserType getBrowserType()
```
    The browser type.
  - getIsIdle
```
 Boolean getIsIdle()
```
  - free
```
 Unit free()
```
    Mark the driver as free, so it can be used to fetch a new page.
  - startWork
```
 Unit startWork()
```
    Mark the driver as working, so it can not be used to fetch another page.
  - retire
```
 Unit retire()
```
    Mark the driver as retired, so it can not be used to fetch any page, and should be quit as soon as possible.
  - cancel
```
 Unit cancel()
```
    Mark the driver as canceled, so the fetch process should return as soon as possible, and the fetch result should be dropped.
  - jvm
```
 JvmWebDriver jvm()
```
    Returns a JvmWebDriver to support other JVM languages, such as java, clojure, scala, and so on, which had difficulty handling kotlin suspend methods.
  - navigateTo
```
 Unit navigateTo(String url)
```
    Returns the main resource response. In case of multiple redirects, the navigation will resolve with the first non-redirect response.
    
    Parameters:
    
    url - URL to navigate page to.
  - waitForSelector
```
 Long waitForSelector(String selector, Long timeoutMillis)
```
    Returns when element specified by selector satisfies {@code state} option. Returns the time remaining until timeout.
  - waitForSelector
```
 Long waitForSelector(String selector)
```
    Returns when element specified by selector satisfies {@code state} option.
  - waitForNavigation
```
 Long waitForNavigation()
```
  - waitForNavigation
```
 Long waitForNavigation(Long timeoutMillis)
```
  - evaluateSilently
```
 Object evaluateSilently(String expression)
```
    Executes JavaScript in the context of the currently selected frame or window. The script fragment provided will be executed as the body of an anonymous function.
    All possible exceptions are suppressed and do not throw.
    
    Parameters:
    
    expression - Javascript expression to evaluate
  - isVisible
```
 Boolean isVisible(String selector)
```
  - isChecked
```
 Boolean isChecked(String selector)
```
  - scrollDown
```
 Unit scrollDown(Integer count)
```
  - scrollUp
```
 Unit scrollUp(Integer count)
```
  - scrollToTop
```
 Unit scrollToTop()
```
  - scrollToBottom
```
 Unit scrollToBottom()
```
  - scrollToMiddle
```
 Unit scrollToMiddle(Float ratio)
```
  - clickNthAnchor
```
 String clickNthAnchor(Integer n, String rootSelector)
```
  - outerHTML
```
 String outerHTML(String selector)
```
  - firstText
```
 String firstText(String selector)
```
  - allTexts
```
 List<String> allTexts(String selector)
```
  - firstAttr
```
 String firstAttr(String selector, String attrName)
```
  - allAttrs
```
 List<String> allAttrs(String selector, String attrName)
```
  - clickTextMatches
```
 Unit clickTextMatches(String selector, String pattern, Integer count)
```
  - clickMatches
```
 Unit clickMatches(String selector, String attrName, String pattern, Integer count)
```
  - check
```
 Unit check(String selector)
```
  - uncheck
```
 Unit uncheck(String selector)
```
  - newSession
```
 Connection newSession()
```
    Create a new session with the same context of the browser: headers, cookies, proxy, etc. The browser should be initialized by opening a page before the session is created.
  - loadResource
```
 Connection.Response loadResource(String url)
```
    Load url as a resource without browser rendering, with the last page's context, which means, the same headers and cookies.
  - quit
```
 Unit quit()
```
    Quit the browser instance
  - equals
```
 Boolean equals(Object other)
```
  - hashCode
```
 Integer hashCode()
```
  - compareTo
```
 Integer compareTo(AbstractWebDriver other)
```
  - toString
```
 String toString()
```

Class AbstractWebDriver

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class ai.platon.pulsar.crawl.fetch.driver.AbstractJvmWebDriver

Methods inherited from class ai.platon.pulsar.crawl.fetch.driver.AbstractWebDriver

Methods inherited from class ai.platon.pulsar.crawl.fetch.driver.WebDriver

Methods inherited from class java.lang.Object

Constructor Detail

AbstractWebDriver

Method Detail

getIdleTimeout

setIdleTimeout

getWaitForElementTimeout

setWaitForElementTimeout

getName

getDelayPolicy

getNavigateEntry

setNavigateEntry

getNavigateHistory

getSupportJavascript

getIsMockedPageSource

getIsRecovered

setIsRecovered

getIsReused

setIsReused

getState

getLastActiveTime

setLastActiveTime

getIsInit

getIsReady

getIsFree

getIsWorking

getIsRetired

getIsQuit

getIsCanceled

getIsCrashed

getSessionId

getBrowser

getId

getBrowserType

getIsIdle

free

startWork

retire

cancel

jvm

navigateTo

waitForSelector

waitForSelector

waitForNavigation

waitForNavigation

evaluateSilently

isVisible

isChecked

scrollDown

scrollUp

scrollToTop

scrollToBottom

scrollToMiddle

clickNthAnchor

outerHTML

firstText

allTexts

firstAttr

allAttrs

clickTextMatches

clickMatches

check

uncheck

newSession

loadResource

quit

equals

hashCode

compareTo

toString