Constructor Detail

• AbstractCrawlEventHandlers

  AbstractCrawlEventHandlers(UrlAwareEventHandler onWillLoad, UrlAwareWebPageEventHandler onLoaded)

  Parameters:

  onLoaded - The URL that was processed

Method Detail

• getOnWillLoad

   UrlAwareEventHandler getOnWillLoad()

  Fires when the URL is about to be loaded in the Main loop.

  This is the first event in the page lifecycle. Use this handler to:

  • Filter URLs (return null to skip)

  • Modify URL properties before loading

  • Log or track browser progress

  (UrlAware) -> UrlAware?

  onWillLoad.addLast { url: UrlAware ->
      if (isBlacklisted(url.url)) null else url
  }

  Returns:

  The URL to continue processing, or null to skip this URL

• getOnLoaded

   UrlAwareWebPageEventHandler getOnLoaded()

  Fires when the URL has been loaded in the Main loop.

  This is the last event in the page lifecycle. Use this handler to:

  • Process loaded page content

  • Handle load failures (page will be null)

  • Collect statistics or metrics

  (UrlAware, WebPage?) -> Any?

  onLoaded.addLast { url, page ->
      if (page != null) {
          saveToDatabase(page)
      } else {
          logFailure(url)
      }
  }

• chain

   CrawlEventHandlers chain(CrawlEventHandlers other)

  Chains another browser event handler to the tail of this one.

  Chained handlers execute in order: this handler's callbacks run first, then the other handler's callbacks.

  Parameters:

  other - The browser event handlers to chain

  Returns:

  This handler instance for fluent chaining