All Implemented Interfaces:: ai.platon.pulsar.common.event.EventEmitter , ai.platon.pulsar.skeleton.crawl.Crawler , java.lang.AutoCloseable


public abstract class AbstractCrawler
extends AbstractEventEmitter<CrawlEvents> implements Crawler

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description

Field Summary

Fields
Modifier and Type	Field	Description
`private final Integer`	`id`
`private final String`	`name`
`private Function2<Integer, UrlAware, Duration>`	`retryDelayPolicy`
`private final Boolean`	`isActive`
`private final PulsarSession`	`session`
`private final Boolean`	`autoClose`
`private Function1<Throwable, Unit>`	`eventExceptionHandler`
`private final Map<CrawlEvents, List<Function<Object>>>`	`listeners`

Constructor Summary

Constructors
Constructor	Description
`AbstractCrawler(PulsarSession session, Boolean autoClose)`

Enum Constant Summary

Enum Constants
Enum Constant	Description

Method Summary

Modifier and Type	Method	Description
`Integer`	`getId()`	The crawl id
`String`	`getName()`	The crawl name
`Function2<Integer, UrlAware, Duration>`	`getRetryDelayPolicy()`	Delay policy for retry tasks
`Unit`	`setRetryDelayPolicy(Function2<Integer, UrlAware, Duration> retryDelayPolicy)`	Delay policy for retry tasks
`Boolean`	`isActive()`
`final PulsarSession`	`getSession()`
`final Boolean`	`getAutoClose()`
`Unit`	`pause()`
`Unit`	`resume()`
`Unit`	`report()`
`Unit`	`onWillLoad(UrlAware url)`
`Unit`	`onLoad(UrlAware url)`
`Unit`	`onLoaded(UrlAware url, WebPage page)`
`Unit`	`close()`

Methods inherited from class ai.platon.pulsar.common.event.AbstractEventEmitter
count, emit, emit, emit, emit, emit1, emit1, emit1, emit1, getEventExceptionHandler, getListeners, hasListeners, listeners, listeners, off, off, off, off, off, off1, off1, off1, off1, off1, on, on, on, on, on1, on1, on1, on1, once, once, once, once1, once1, once1, setEventExceptionHandler
Methods inherited from class ai.platon.pulsar.skeleton.crawl.Crawler
await
Methods inherited from class ai.platon.pulsar.common.event.EventEmitter
hasNormalListeners, hasSuspendListeners, normalListeners, normalListeners, suspendListeners, suspendListeners
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractCrawler

AbstractCrawler(PulsarSession session, Boolean autoClose)

Method Detail

getId
```
 Integer getId()
```
The crawl id

getName
```
 String getName()
```
The crawl name

getRetryDelayPolicy

 Function2<Integer, UrlAware, Duration> getRetryDelayPolicy()

Delay policy for retry tasks

setRetryDelayPolicy

 Unit setRetryDelayPolicy(Function2<Integer, UrlAware, Duration> retryDelayPolicy)

Delay policy for retry tasks

isActive
```
 Boolean isActive()
```

getSession
```
 final PulsarSession getSession()
```

getAutoClose
```
 final Boolean getAutoClose()
```

pause
```
 Unit pause()
```

resume
```
 Unit resume()
```

report
```
 Unit report()
```

onWillLoad
```
 Unit onWillLoad(UrlAware url)
```

onLoad
```
 Unit onLoad(UrlAware url)
```

onLoaded

 Unit onLoaded(UrlAware url, WebPage page)

close
```
 Unit close()
```

Class AbstractCrawler

Nested Class Summary

Field Summary

Constructor Summary

Enum Constant Summary

Method Summary

Methods inherited from class ai.platon.pulsar.common.event.AbstractEventEmitter

Methods inherited from class ai.platon.pulsar.skeleton.crawl.Crawler

Methods inherited from class ai.platon.pulsar.common.event.EventEmitter

Methods inherited from class java.lang.Object

Constructor Detail

AbstractCrawler

Method Detail

getId

getName

getRetryDelayPolicy

setRetryDelayPolicy

isActive

getSession

getAutoClose

pause

resume

report

onWillLoad

onLoad

onLoaded

close