Package ai.platon.pulsar.crawl.fetch
See: Description
-
Class Summary Class Description CoreMetrics FeedLoop This class feeds the fetchMonitor with input items, and re-fills them as items are consumed by FetcherThread-s. FetchLoop This class picks items from queues and fetches the pages FetchMonitor Created by vincent on 16-9-24. UrlStat Created by vincent on 16-10-15. BatchStat FetchTask FetchResult Key JobFetchTask This class described the item to be fetched. LazyFetchTaskManager TaskMonitor Tasks Monitor TaskPool This class handles FetchTasks which come from the same host ID (be it a proto/hostname or proto/IP pair). Status TaskScheduler TaskSchedulers -
Interface Summary Interface Description IFetchEntry