Package ai.platon.pulsar.skeleton.crawl.fetch.driver

Interface WebDriver

  • All Implemented Interfaces:
    java.io.Closeable , java.lang.AutoCloseable 
    
public interface WebDriver
 implements Closeable

    WebDriver defines a concise interface to visit and manipulate webpages.

    The webpage is rendered to a Document Object Model (DOM) in a real browser, and the interface provides methods to control the browser, select textContent and attributes of Elements, and interact with the webpage.

    All actions and behaviors are optimized to mimic real people as closely as possible, such as scrolling, clicking, typing text, dragging and dropping, etc.

    The term document here refers to a Document Object Model (DOM) within a browser.

    The methods in this interface fall into three categories:

    • Control of the browser itself

    • Selection of textContent and attributes of Elements

    • Interact with the webpage

    Key methods:

    • navigateTo: navigate to a URL.

    • currentUrl: get the current URL displayed in the address bar.

    • scrollDown: scroll down on a webpage to fully load the page. Most modern webpages support lazy loading using ajax tech, where the page content only starts to load when it is scrolled into view.

    • pageSource: retrieve the source code of the webpage.

    For each document, there are several properties that represent the URL of the document:

    • driver.currentUrl(): Returns the URL displayed in the address bar, it can be either navigated or not.

    • driver.url(), document.URL: Returns the URL of the document.

    • driver.documentURI(), document.documentURI: Returns the URI of the document.

    • driver.baseURI(), document.baseURI: Returns the base URI of the document.

    • document.location: Represents the location (URL) of the current page and allows you to manipulate the URL.

    In the Document Object Model (DOM), the relationship between document.URL, document.documentURI, document.location, and the URL displayed in the browser's address bar is as follows:

    • driver.currentUrl():

    • driver.url(), document.URL:

    • driver.documentURI(), document.documentURI:

    • driver.baseURI(), document.baseURI:

    • document.location:

    In summary, document.URL and document.documentURI are read-only properties that reflect the current URL of the document, while document.location is a read-write property that not only reflects the current URL but also allows you to navigate to a new one. The URL displayed in the address bar is a user-facing representation of the current document's URL, which is usually in sync with document.location.

    In addition to the above properties, The method driver.referrer() returns the document's referrer. The document.referrer property returns the URI of the page that linked to the current page. If the user navigated directly to the page (e.g., via a bookmark), the value is an empty string. Inside an <iframe>, the referrer is initially set to the same value as the href of the parent window's Window.location.

    The following example demonstrates how to use the WebDriver interface to visit a webpage and interact with it:

    fun visit() {
    val url = "https://twitter.com/home"
    val args = "-refresh"
    val options = session.options(args)

    options.eventHandlers.browseEventHandlers.onDocumentSteady.addLast { page, driver ->
        interact(page, driver)
    }

    session.load(url, options)
}

// interact with the page
private suspend fun interact(driver: WebDriver) {
    val selector = "input[placeholder*=搜索], input[placeholder*=Search]"
    driver.waitForSelector(selector)
    driver.fill(selector, "Facebook")
    driver.press(selector, "Space")
    "Email".forEach { driver.press(selector, "$it") }
    driver.press(selector, "Enter")
}

    • Constructor Detail

    • Method Detail

      • jvm

         abstract JvmWebDriver jvm()

        Returns a JvmWebDriver to support other JVM languages, such as java, clojure, scala, and so on, the other JVM languages might have difficulty to handle kotlin suspend methods.

      • addInitScript

         abstract Unit addInitScript(String script)

        Adds a script which would be evaluated in one of the following scenarios:

        • Whenever the page is navigated.

        The script is evaluated after the document was created but before any of its scripts were run. This is useful to amend the JavaScript environment, e.g. to seed Math.random.

        This function should be invoked before a navigation, usually in an onWillNavigate event handler.

        Parameters:
        script - Javascript source code to add.

      • addBlockedURLs

         abstract Unit addBlockedURLs(List<String> urlPatterns)

        Blocks resource URLs from loading.

        Parameters:
        urlPatterns - URL patterns to block.

      • addProbabilityBlockedURLs

         abstract Unit addProbabilityBlockedURLs(List<String> urlPatterns)

        Block resource URL loading with a certain probability.

        Parameters:
        urlPatterns - Regular expressions of URLs to block.

      • setTimeouts

         abstract Unit setTimeouts(BrowserSettings browserSettings)

        TODO: NOT IMPLEMENTED

      • open

         Unit open(String url)

        Opens the specified URL in the web driver.

        This function navigates the web driver to the provided URL and waits for the navigation to complete. It is a suspend function, meaning it can be used within coroutines for asynchronous execution.

        Example usage:

        driver.open("https://www.example.com")
        Parameters:
        url - The URL to which the web driver should navigate.

      • navigateTo

         abstract Unit navigateTo(String url)

        Navigates current page to the given URL.

        driver.navigateTo("https://www.example.com")
driver.waitForNavigation()
        Parameters:
        url - URL to navigate page to.

      • navigateTo

         abstract Unit navigateTo(NavigateEntry entry)

        Navigates current page to the given URL.

        val entry = NavigateEntry("https://www.example.com?timestamp=11712067353", pageUrl = "https://www.example.com")
driver.navigateTo(entry)
driver.waitForNavigation()
        Parameters:
        entry - NavigateEntry to navigate page to.

      • currentUrl

         abstract String currentUrl()

        Returns a string representing the current URL that the browser is looking at. The current url is always the main frame's document.documentURI if the browser succeed to return it, and is displayed in the browser's address bar.

        If the browser failed to return a proper url, returns the passed in url to navigate, just like a real user enter a url in the address bar but the browser failed to load the page.

        NOTE: the url can be non-standard, for example:

        about:blank chrome://newtab chrome://settings

        Returns:

        A string containing the URL of the document, or the passed in url to navigate.

      • url

         abstract String url()

        The URL read-only property of the Document interface returns the document location as a string.

        This property equals to javascript document.URL. The document.URL property returns the same value. The document.documentURI property can be used on any document types, while the document.URL property can only be used on HTML documents.

        Returns:

        A string containing the URL of the document.

      • documentURI

         abstract String documentURI()

        Returns the document location as a string.

        This property equals to javascript document.documentURI.

        The document.URL property which returns the same value. The document.documentURI property can be used on any document types, while the document.URL property can only be used on HTML documents.

        Returns:

        The document's documentURI.

      • baseURI

         abstract String baseURI()

        Returns the document's baseURI.

        The baseURI is a property of Node, it's the absolute base URL of the document containing the node. A baseURI is used to resolve relative URLs.

        The base URL is determined as follows:

        • By default, the base URL is the location of the document (as determined by window.location).

        • If the document has an <base> element, its href attribute is used.

        Returns:

        The document's baseURI.

      • referrer

         abstract String referrer()

        The referrer property returns the URI of the page that linked to this page.

        The value is an empty string if the user navigated to the page directly (not through a link, but, for example, by using a bookmark).

        Inside an <iframe>, the referrer will initially be set to the same value as the href of the parent window's Window.location.

        Returns:

        The document's referrer.

      • pageSource

         abstract String pageSource()

        Returns the source of the last loaded page. If the page has been modified after loading (for example, by Javascript) there is no guarantee that the returned text is that of the modified page.

        TODO: distinguish pageSource and outerHTML

        val pageSource = driver.pageSource()
        Returns:

        The source of the current page

      • chat

         abstract ModelResponse chat(String prompt, String selector)

        Chat with the AI model about the specified element.

        Parameters:
        prompt - The prompt to chat with
        selector - The selector to find the element
        Returns:

        The response from the model

      • instruct

         abstract InstructionResult instruct(String prompt)

        Instructs the webdriver to perform a series of actions based on the given prompt. This function converts the prompt into a sequence of webdriver actions, which are then executed.

        Parameters:
        prompt - The textual prompt that describes the actions to be performed by the webdriver.
        Returns:

        The response from the model, though in this implementation, the return value is not explicitly used.

      • getCookies

         abstract List<Map<String, String>> getCookies()

        Returns the cookies of the current page.

        val cookies = driver.getCookies()
        Returns:

        The cookies of the current page.

      • deleteCookies

         abstract Unit deleteCookies(String name)

        Deletes browser cookies with matching name.

        driver.deleteCookies("name")
        Parameters:
        name - Name of the cookies to remove.

      • deleteCookies

         abstract Unit deleteCookies(String name, String url, String domain, String path)

        Deletes browser cookies with matching name and url or domain/path pair.

        driver.deleteCookies("name", "https://www.example.com")
        Parameters:
        name - Name of the cookies to remove.
        url - If specified, deletes all the cookies with the given name where domain and path match provided URL.
        domain - If specified, deletes only cookies with the exact domain.
        path - If specified, deletes only cookies with the exact path.

      • clearBrowserCookies

         abstract Unit clearBrowserCookies()

        Clears browser cookies.

        driver.clearBrowserCookies()

        TODO: consider only use driver.browser.clearCookies()

      • waitForSelector

         Duration waitForSelector(String selector)

        Wait until the element identified by the selector becomes present in the DOM or timeout.

        val remainingTime = driver.waitForSelector("h2.title")
        Parameters:
        selector - The selector of the element to wait for.
        Returns:

        The remaining time until timeout when the element becomes present.

      • waitForSelector

         Long waitForSelector(String selector, Long timeoutMillis)

        Wait until the element identified by the selector becomes present in the DOM or timeout.

        val remainingTime = driver.waitForSelector("h2.title", 30000)
        Parameters:
        timeoutMillis - The maximum time to wait for the element to become present.
        Returns:

        The remaining time until timeout when the element becomes present.

      • waitForSelector

         Duration waitForSelector(String selector, Duration timeout)

        Wait for the element identified by the selector to become present in the DOM, or until timeout.

        val remainingTime = driver.waitForSelector("h2.title", Duration.ofSeconds(30))
        Parameters:
        timeout - The maximum time to wait for the element to become present.
        Returns:

        The remaining time until timeout when the element becomes present.

      • waitForSelector

         abstract Duration waitForSelector(String selector, SuspendFunction0<Unit> action)

        Wait for the element identified by the selector to become present in the DOM, or until timeout. This method periodically checks for the existence of the element. If the element is not found during a check, the action will be executed, such as scrolling the page down.

        val remainingTime = driver.waitForSelector("h2.title") {
 driver.scrollDown()
}
        Parameters:
        action - The action to execute when the element is not found.

      • waitForSelector

         Long waitForSelector(String selector, Long timeoutMillis, SuspendFunction0<Unit> action)

        Wait for the element identified by the selector to become present in the DOM, or until timeout. This method periodically checks for the existence of the element. If the element is not found during a check, the action will be executed, such as scrolling the page down.

        val remainingTime = driver.waitForSelector("h2.title", 30000) {
 driver.scrollDown()
}
        Parameters:
        timeoutMillis - The maximum time to wait for the element to become present.
        action - The action to execute when the element is not found.
        Returns:

        The remaining time until timeout when the element becomes present.

      • waitForSelector

         abstract Duration waitForSelector(String selector, Duration timeout, SuspendFunction0<Unit> action)

        Wait for the element identified by the selector to become present in the DOM, or until timeout. This method periodically checks for the existence of the element. If the element is not found during a check, the action will be executed, such as scrolling the page down.

        val remainingTime = driver.waitForSelector("h2.title", Duration.ofSeconds(30)) {
 driver.scrollDown()
}
        Parameters:
        timeout - The maximum time to wait for the element to become present.
        action - The action to execute when the element is not found.
        Returns:

        The remaining time until timeout when the element becomes present.

      • waitForNavigation

         abstract Duration waitForNavigation(String oldUrl)

        Wait until the current url changes or timeout.

        val url = "https://www.example.com"
driver.navigateTo(url)
var remainingTime = driver.waitForNavigation()
if (remainingTime 0) {
  driver.click("a[href='/next']")
  remainingTime = driver.waitForNavigation(url)
}

      • waitForNavigation

         Long waitForNavigation(String oldUrl, Long timeoutMillis)

        Wait until the current url changes or timeout.

        val url = "https://www.example.com"
driver.navigateTo(url)
var remainingTime = driver.waitForNavigation(1000)
if (remainingTime 0) {
  driver.click("a[href='/next']")
  remainingTime = driver.waitForNavigation(url, 1000)
}
        Parameters:
        timeoutMillis - The maximum time to wait for the url to change.

      • waitForNavigation

         abstract Duration waitForNavigation(String oldUrl, Duration timeout)

        Wait until the current url changes or timeout.

        val timeout = Duration.ofSeconds(30)
val url = "https://www.example.com"
driver.navigateTo(url)
var remainingTime = driver.waitForNavigation(timeout)
if (remainingTime 0) {
  driver.click("a[href='/next']")
  remainingTime = driver.waitForNavigation(url, timeout)
}
        Parameters:
        timeout - The maximum time to wait for the url to change.

      • waitForPage

         abstract WebDriver waitForPage(String url, Duration timeout)

        Await navigation to the specified URL page or timeout if necessary.

        val newDriver = driver.waitForPage("https://www.example.com", Duration.ofSeconds(30))
        Parameters:
        url - The URL to navigate to.
        Returns:

        The remaining time until timeout when the predicate returns true.

      • waitUntil

         abstract Duration waitUntil(SuspendFunction0<Boolean> predicate)

        Wait until the predicate returns true.

        val remainingTime = driver.waitUntil {
   driver.exists("h2.title")
}
        Parameters:
        predicate - The predicate to check.
        Returns:

        The remaining time until timeout when the predicate returns true.

      • waitUntil

         Long waitUntil(Long timeoutMillis, SuspendFunction0<Boolean> predicate)

        Wait until the predicate returns true.

        val remainingTime = driver.waitUntil(10000) {
  driver.exists("h2.title")
}
        Parameters:
        timeoutMillis - The maximum time to wait for the predicate to return true.
        predicate - The predicate to check.
        Returns:

        The remaining time until timeout when the predicate returns true.

      • waitUntil

         abstract Duration waitUntil(Duration timeout, SuspendFunction0<Boolean> predicate)

        Wait until the predicate returns true.

        val remainingTime = driver.waitUntil(Duration.ofSeconds(10)) {
   driver.exists("h2.title")
}
        Parameters:
        timeout - The maximum time to wait for the predicate to return true.
        predicate - The predicate to check.
        Returns:

        The remaining time until timeout when the predicate returns true.

      • exists

         abstract Boolean exists(String selector)

        Returns whether the element exists.

        driver.exists("h2.title")
        Parameters:
        selector -

        • The selector of the element to check.

        Returns:

        Whether the element exists.

      • isHidden

         Boolean isHidden(String selector)

        Returns whether the element is hidden.

        driver.isHidden("input[name='q']")
        Parameters:
        selector -

        • The selector of the element to check.

        Returns:

        Whether the element is hidden.

      • isVisible

         abstract Boolean isVisible(String selector)

        Returns whether the element is visible.

        driver.isVisible("input[name='q']")
        Parameters:
        selector -

        • The selector of the element to check.

        Returns:

        Whether the element is visible.

      • visible

         Boolean visible(String selector)

        Returns whether the element is visible.

        driver.visible("input[name='q']")
        Parameters:
        selector -

        • The selector of the element to check.

        Returns:

        Whether the element is visible.

      • isChecked

         abstract Boolean isChecked(String selector)

        Returns whether the element is checked.

        driver.isChecked("input[name='agree']")
        Parameters:
        selector -

        • The selector of the element to check.

        Returns:

        Whether the element is checked.

      • bringToFront

         abstract Unit bringToFront()

        Brings the browser window to the front.

        driver.bringToFront()

      • focus

         abstract Unit focus(String selector)

        This method fetches an element with selector and focuses it. If there's no element matching selector, nothing to do.

        driver.focus("input[name='q']")
        Parameters:
        selector -

        • A selector of an element to focus. If there are multiple elements satisfying the selector, the first will be focused.

      • type

         abstract Unit type(String selector, String text)

        This method emulates inserting text that doesn't come from a key press.

        driver.type("input[name='q']", "Hello, World!")
        Parameters:
        selector -

        • A selector of an element to focus. If there are multiple elements satisfying the selector, the first will be focused.

        text - The text to insert.

      • fill

         abstract Unit fill(String selector, String text)

        This method emulates inserting text that doesn't come from a key press.

        Unlike type, this method clears the existing value before typing.

        driver.fill("input[name='q']", "Hello, World!")
        Parameters:
        selector -

        • A selector of an element to focus, and then fill text into it. If there are multiple elements satisfying the selector, the first will be focused.

        text - The text to fill.

      • press

         abstract Unit press(String selector, String key)

        Shortcut for keyboard down and keyboard up.

        The key is specified as a string, which can be a single character, a key name, or a combination of both. For example, 'a', 'A', 'KeyA', 'Enter', 'Shift+A', and 'Control+Shift+Tab' are all valid keys.

        driver.press("input[name='q']", "Enter")
        Parameters:
        selector -

        • A selector of an element to focus, and then press a key. If there are multiple elements satisfying the selector, the first will be focused.

        key -

      • click

         abstract Unit click(String selector, Integer count)

        This method clicks an element with selector and focuses it. If there's no element matching selector, nothing to do.

        driver.click("button[type='submit']")
        Parameters:
        selector -

        • A selector of an element to focus. If there are multiple elements satisfying the selector, the first will be focused.

        count - The number of times to click.

      • clickTextMatches

         abstract Unit clickTextMatches(String selector, String pattern, Integer count)

        This method clicks an element with selector whose text content matches pattern, and then focuses it. If there's no element matching selector, or the element's text content doesn't match pattern, nothing to do.

        driver.clickTextMatches("button", "submit")
        Parameters:
        selector -

        • A selector of an element to focus. If there are multiple elements satisfying the selector, the first will be focused.

        pattern - The pattern to match the text content.
        count - The number of times to click.

      • clickMatches

         abstract Unit clickMatches(String selector, String attrName, String pattern, Integer count)

        This method clicks an element with selector whose attribute name is attrName and value matches pattern, and then focuses it. If there's no element matching selector, or the element has no attribute attrName, or the element's attribute value doesn't match pattern, nothing to do.

        driver.clickAttributeMatches("button", "type", "submit")
        Parameters:
        selector -

        • A selector of an element to focus. If there are multiple elements satisfying the selector, the first will be focused.

        attrName - The attribute name to match.
        pattern - The pattern to match the text content.
        count - The number of times to click.

      • clickNthAnchor

         abstract String clickNthAnchor(Integer n, String rootSelector)

        Clicks the nth anchor element in the DOM.

        This function searches for all anchor (<a>) elements within the specified root element, and clicks the nth anchor element (0-based index). If the anchor element exists, it returns the href attribute of the clicked anchor element. If the element does not exist, it returns null.

        driver.clickNthAnchor(100, "body")
        Parameters:
        n - The index of the anchor element to click (0-based).
        rootSelector - The CSS selector of the root element to search within (default is "body").
        Returns:

        The href attribute of the clicked anchor element, or null if the element does not exist.

      • check

         abstract Unit check(String selector)

        This method check an element with selector. If there's no element matching selector, nothing to do.

        Parameters:
        selector -

        • A selector of an element to check. If there are multiple elements satisfying the selector, the first will be checked.

      • uncheck

         abstract Unit uncheck(String selector)

        This method uncheck an element with selector. If there's no element matching selector, nothing to do.

        Parameters:
        selector -

        • A selector of an element to uncheck. If there are multiple elements satisfying the selector, the first will be focused.

      • scrollTo

         abstract Unit scrollTo(String selector)

        This method fetches an element with selector, scrolls it into view if needed. If there's no element matching selector, the method does nothing.

        driver.scrollTo("h2.title")
        Parameters:
        selector -

        • A selector to search for element to scroll to. If there are multiple elements satisfying the selector, the first will be selected.

      • scrollDown

         abstract Unit scrollDown(Integer count)

        The current page frame scrolls down for count times.

        driver.scrollDown(3)
        Parameters:
        count - The times to scroll down.

      • scrollUp

         abstract Unit scrollUp(Integer count)

        The current page frame scrolls up for count times.

        driver.scrollUp(3)
        Parameters:
        count - The times to scroll up.

      • scrollToTop

         abstract Unit scrollToTop()

        The current page frame scrolls to the top.

        driver.scrollToTop()

      • scrollToBottom

         abstract Unit scrollToBottom()

        The current page frame scrolls to the bottom.

        driver.scrollToBottom()

      • scrollToMiddle

         abstract Unit scrollToMiddle(Double ratio)

        The current page frame scrolls to the middle.

        driver.scrollToMiddle(0.2)
driver.scrollToMiddle(0.5)
driver.scrollToMiddle(0.8)
        Parameters:
        ratio - The ratio of the page to scroll to, 0.0 means the top, 1.0 means the bottom.

      • scrollToScreen

         abstract Unit scrollToScreen(Double screenNumber)

        The current page frame scrolls to the middle.

        driver.scrollToScreen(0.0)
driver.scrollToScreen(0.5)
driver.scrollToScreen(1.5)
driver.scrollToScreen(3.0)
        Parameters:
        screenNumber - The screen number of the page to scroll to (0-based).

      • mouseWheelDown

         abstract Unit mouseWheelDown(Integer count, Double deltaX, Double deltaY, Long delayMillis)

        The mouse wheels down for count times.

        driver.mouseWheelDown(3)
        Parameters:
        count - The times to wheel down.
        deltaX - The distance to wheel horizontally.
        deltaY - The distance to wheel vertically.
        delayMillis - The delay time in milliseconds.

      • mouseWheelUp

         abstract Unit mouseWheelUp(Integer count, Double deltaX, Double deltaY, Long delayMillis)

        The mouse wheels up for count times.

        driver.mouseWheelUp(3)
        Parameters:
        count - The times to wheel up.
        deltaX - The distance to wheel horizontally.
        deltaY - The distance to wheel vertically.
        delayMillis - The delay time in milliseconds.

      • moveMouseTo

         abstract Unit moveMouseTo(Double x, Double y)

        The mouse moves to the position specified by x and y.

        driver.moveMouseTo(100.0, 200.0)
        Parameters:
        x - The x coordinate to move to.
        y - The y coordinate to move to.

      • moveMouseTo

         abstract Unit moveMouseTo(String selector, Integer deltaX, Integer deltaY)

        The mouse moves to the element with selector.

        driver.moveMouseTo("h2.title")
        Parameters:
        deltaX - The distance to the left of the element.
        deltaY - The distance to the top of the element.

      • dragAndDrop

         abstract Unit dragAndDrop(String selector, Integer deltaX, Integer deltaY)

        Performs a drag, dragenter, dragover, and drop in sequence.

        Parameters:
        selector -

        • selector of the element to drag from.

        deltaX - The distance to drag horizontally.
        deltaY - The distance to drag vertically.

      • outerHTML

         abstract String outerHTML()

        Returns the document's HTML markup.

        val html = driver.outerHTML()

        If the document does not exist, returns null.

        Returns:

        The HTML markup of the document.

      • outerHTML

         abstract String outerHTML(String selector)

        Returns the node's HTML markup, the node is located by selector.

        If the node does not exist, returns null.

        val html = driver.outerHTML("h2.title")
        Parameters:
        selector - The selector to locate the node.
        Returns:

        The HTML markup of the node.

      • selectFirstTextOrNull

         abstract String selectFirstTextOrNull(String selector)

        Returns the node's text content, the node is located by selector.

        If the node does not exist, returns null.

        val text = driver.selectFirstTextOrNull("h2.title")
        Parameters:
        selector - The selector to locate the node.
        Returns:

        The text content of the node.

      • selectTextAll

         abstract List<String> selectTextAll(String selector)

        Returns a list of text contents of all the elements matching the specified selector within the page.

        If no elements match the selector, returns an empty list.

        val texts = driver.selectTextAll("h2")
        Parameters:
        selector - The selector to locate the nodes.
        Returns:

        The text contents of the nodes.

      • selectFirstAttributeOrNull

         abstract String selectFirstAttributeOrNull(String selector, String attrName)

        Returns the node's attribute value, the node is located by selector, the attribute is attrName.

        If the node does not exist, or the attribute does not exist, returns null.

        val classes = driver.selectFirstAttributeOrNull("h2.title", "class")
        Parameters:
        selector - The selector to locate the node.
        attrName - The attribute name to retrieve.
        Returns:

        The attribute value of the node.

      • selectAttributes

         abstract Map<String, String> selectAttributes(String selector)

        Returns the node's attribute values, the node is located by selector.

        If the node do not exist, or the attribute does not exist, returns an empty list.

        val classes = driver.selectAttributes("h2.title", "class")
        Parameters:
        selector - The selector to locate the nodes.
        Returns:

        The attribute pairs of the nodes.

      • selectAttributeAll

         abstract List<String> selectAttributeAll(String selector, String attrName, Integer start, Integer limit)

        Returns the nodes' attribute values, the nodes are located by selector, the attribute is attrName.

        If the nodes do not exist, or the attribute does not exist, returns an empty list.

        val classes = driver.selectAttributeAll("h2.title", "class")
        Parameters:
        selector - The selector to locate the nodes.
        attrName - The attribute name to retrieve.
        start - The offset of the first node to select.
        limit - The maximum number of nodes to select.
        Returns:

        The attribute values of the nodes.

      • setAttribute

         abstract Unit setAttribute(String selector, String attrName, String attrValue)

        Set the attribute of an element located by selector.

        driver.setAttribute("h2.title", "class", "header")
        Parameters:
        selector - The CSS query to select an element.
        attrName - The attribute name to set.
        attrValue - The attribute value to set.

      • setAttributeAll

         abstract Unit setAttributeAll(String selector, String attrName, String attrValue)

        Set the attribute of all elements matching the CSS query.

        driver.setAttributeAll("h2.title", "class", "header")
        Parameters:
        selector - The CSS query to select elements.
        attrName - The attribute name to set.
        attrValue - The attribute value to set.

      • selectFirstPropertyValueOrNull

         abstract String selectFirstPropertyValueOrNull(String selector, String propName)

        Returns the node's property value, the node is located by selector, the property is attrName.

        If the node does not exist, or the property does not exist, returns null.

        val classes = driver.selectFirstPropertyOrNull("input#input", "value")
        Parameters:
        selector - The selector to locate the node.
        propName - The property name to retrieve.
        Returns:

        The property value of the node.

      • selectPropertyValueAll

         abstract List<String> selectPropertyValueAll(String selector, String propName, Integer start, Integer limit)

        Returns the nodes' property values, the nodes are located by selector, the property is propName.

        If the nodes do not exist, or the property does not exist, returns an empty list.

        val classes = driver.selectPropertyAll("input#input", "value")
        Parameters:
        selector - The selector to locate the nodes.
        propName - The property name to retrieve.
        start - The offset of the first node to select.
        limit - The maximum number of nodes to select.
        Returns:

        The property values of the nodes.

      • setProperty

         abstract Unit setProperty(String selector, String propName, String propValue)

        Set the property of an element located by selector.

        driver.setProperty("input#input", "value")
        Parameters:
        selector - The CSS query to select an element.
        propName - The property name to set.
        propValue - The property value to set.

      • setPropertyAll

         abstract Unit setPropertyAll(String selector, String propName, String propValue)

        Set the property of all elements matching the CSS query.

        driver.setPropertyAll("input#input", "value")
        Parameters:
        selector - The CSS query to select elements.
        propName - The property name to set.
        propValue - The property value to set.

      • selectHyperlinks

         abstract List<Hyperlink> selectHyperlinks(String selector, Integer offset, Integer limit)

        Find hyperlinks in elements matching the CSS query.

        val hyperlinks = driver.selectHyperlinks("a.product-link")
        Parameters:
        selector - The CSS query to select elements.
        offset - The offset of the first element to select.
        limit - The maximum number of elements to select.
        Returns:

        The hyperlinks in the elements.

      • selectAnchors

         abstract List<GeoAnchor> selectAnchors(String selector, Integer offset, Integer limit)

        Find anchor elements matching the CSS query.

        val anchors = driver.selectAnchors("a.product-link")
        Parameters:
        selector - The CSS query to select elements.
        offset - The offset of the first element to select.
        limit - The maximum number of elements to select.
        Returns:

        The anchors.

      • selectImages

         abstract List<String> selectImages(String selector, Integer offset, Integer limit)

        Find image elements matching the CSS query.

        val images = driver.selectImages("img.product-image")
        Parameters:
        selector - The CSS query to select elements.
        offset - The offset of the first element to select.
        limit - The maximum number of elements to select.
        Returns:

        The image URLs.

      • evaluate

         abstract Object evaluate(String expression)

        Executes JavaScript in the context of the currently selected frame or window. If the result is not Javascript object, it is not returned.

        val title = driver.evaluate("document.title")

        Multi-line JavaScript code:

        val code = """
() => {
  const a = 10;
  const b = 20;
  return a * b;
}
""".trimIndent()

val result = driver.evaluate(code)

        • Wrap the code in an IIFE (Immediately Invoked Function Expression) to return a value.

        • Escape line breaks with \n.

        Parameters:
        expression - Javascript expression to evaluate
        Returns:

        Remote object value in case of primitive values or null.

      • evaluate

         abstract <T extends Any> T evaluate(String expression, T defaultValue)

        Executes JavaScript in the context of the currently selected frame or window. If the result is not Javascript object, it is not returned.

        val title = driver.evaluate("document.title", "Untitled")
        Parameters:
        expression - Javascript expression to evaluate
        Returns:

        Remote object value in case of primitive values or null.

      • evaluateDetail

        @Beta() abstract JsEvaluation evaluateDetail(String expression)

        Executes JavaScript in the context of the currently selected frame or window. If the result is not Javascript object, it is not returned.

        val title = driver.evaluate("document.title")
        Parameters:
        expression - Javascript expression to evaluate
        Returns:

        expression result

      • evaluateValue

        @Beta() abstract Object evaluateValue(String expression)

        Executes JavaScript in the context of the currently selected frame or window. If the result is a Javascript object, it is returned as a JSON object.

        val title = driver.evaluate("document.title")

        To execute multi-line JavaScript code:

        val code = """
() => {
  const a = 10;
  const b = 20;
  return a * b;
}
""".trimIndent()

val result = driver.evaluate(code)

        • Wrap the code in an IIFE (Immediately Invoked Function Expression) to return a value.

        • Escape line breaks with \n.

        Parameters:
        expression - Javascript expression to evaluate
        Returns:

        Remote object value in case of primitive values or JSON values (if it was requested).

      • evaluateValue

        @Beta() abstract <T extends Any> T evaluateValue(String expression, T defaultValue)

        Executes JavaScript in the context of the currently selected frame or window. If the result is a Javascript object, it is returned as a JSON object.

        val title = driver.evaluate("document.title", "Untitled")
        Parameters:
        expression - Javascript expression to evaluate
        Returns:

        Remote object value in case of primitive values or JSON values (if it was requested).

      • evaluateValueDetail

        @Beta() abstract JsEvaluation evaluateValueDetail(String expression)

        Executes JavaScript in the context of the currently selected frame or window. If the result is a Javascript object, it is returned as a JSON object.

        val title = driver.evaluate("document.title")
        Parameters:
        expression - Javascript expression to evaluate
        Returns:

        expression result

      • captureScreenshot

         abstract String captureScreenshot()

        This method scrolls element into view if needed, and then ake a screenshot of the element.

        val screenshot = driver.captureScreenshot()
val bytes = Base64.getDecoder().decode(screenshot)
val path = AppPaths.TMP_DIR.resolve("screenshot.jpg")
AppFiles.saveTo(bytes, path, true)
        Returns:

        The screenshot of the element in base64 format.

      • captureScreenshot

         abstract String captureScreenshot(String selector)

        This method scrolls element into view if needed, and then ake a screenshot of the element.

        val screenshot = driver.captureScreenshot("h2.title")
val bytes = Base64.getDecoder().decode(screenshot)
val path = AppPaths.TMP_DIR.resolve("screenshot.jpg")
AppFiles.saveTo(bytes, path, true)
        Parameters:
        selector - The selector of the element to capture.
        Returns:

        The screenshot of the element in base64 format.

      • captureScreenshot

         abstract String captureScreenshot(RectD rect)

        This method scrolls element into view if needed, and then ake a screenshot of the element.

        Parameters:
        rect - The rectangle of the element to capture.
        Returns:

        The screenshot of the element in base64 format.

      • clickablePoint

         abstract PointD clickablePoint(String selector)

        Calculate the clickable point of an element located by selector. If the element does not exist, or is not clickable, returns null.

        Parameters:
        selector - The selector of the element to calculate the clickable point.
        Returns:

        The clickable point of the element.

      • boundingBox

         abstract RectD boundingBox(String selector)

        Return the bounding box of an element located by selector. If the element does not exist, returns null.

        Parameters:
        selector - The selector of the element to calculate the bounding box.
        Returns:

        The bounding box of the element.

      • newJsoupSession

         abstract Connection newJsoupSession()

        Create a new Jsoup session with the last page's context, which means, the same headers and cookies.

        Returns:

        The Jsoup session.

      • loadJsoupResource

         abstract Connection.Response loadJsoupResource(String url)

        Load the url as a resource with Jsoup rather than browser rendering, with the last page's context, which means, the same headers and cookies.

        Parameters:
        url - The URL to load.
        Returns:

        The Jsoup response.

      • loadResource

         abstract NetworkResourceResponse loadResource(String url)

        Load the url as a resource without browser rendering, with the last page's context, which means, the same headers and cookies.

        Parameters:
        url - The URL to load.
        Returns:

        The network resource response.

      • delay

         Unit delay(Long millis)

        Delay for a given amount of time.

        Parameters:
        millis - The amount of time to delay, in milliseconds.

      • delay

         Unit delay(Duration duration)

        Delay for a given amount of time.

        Parameters:
        duration - The amount of time to delay.

      • delay

         Unit delay(Duration duration)

        Delay for a given amount of time.

        Parameters:
        duration - The amount of time to delay.

      • pause

         abstract Unit pause()

        Force the page pauses all navigations and PENDING resource fetches. If the page loading pauses, the user can still interact with the page, and therefore resources can continue to load.

      • stop

         abstract Unit stop()

        Force the page stop all navigations and RELEASES all resources. Interaction with the stop page results in undefined behavior and the results should not be trusted.

        If a web driver stops, it can later be used to visit other pages.

      • getBrowser

         abstract Browser getBrowser()

        The browser of the driver. The browser defines methods and events to manipulate a real browser.

      • getFrames

         abstract List<WebDriver> getFrames()

        Web pages for the page open from the current page, via window.open(), link click, form submission, etc.

        NOT IMPLEMENTED

      • getOutgoingPages

         abstract Set<WebDriver> getOutgoingPages()

        Web pages for the page open from the current page, via window.open(), link click, form submission, etc.

      • getBrowserType

         abstract BrowserType getBrowserType()

        The browser type. BrowserType.PULSAR_CHROME is the only supported browser type currently.

      • getDelayPolicy

         abstract Map<String, IntRange> getDelayPolicy()

        The delay policy of the driver. The delay policy is a map of delay ranges in milliseconds for different actions.

        The delay policy is used to simulate human behaviors, such as typing, clicking, scrolling, etc.

        delayPolicy["click"] == 500..1000

      • getTimeoutPolicy

         abstract Map<String, Duration> getTimeoutPolicy()

        The timeout policy of the driver. The timeout policy is a map of timeout durations for different actions.

        The timeout policy is used to set the maximum time to wait for an action to complete.

        timeoutPolicy["click"] == Duration.ofSeconds(30)