Class ChainedUrlNormalizer
-
- All Implemented Interfaces:
-
ai.platon.pulsar.common.urls.preprocess.UrlNormalizer,ai.platon.pulsar.skeleton.crawl.filter.ScopedUrlNormalizer,kotlin.Function,kotlin.Function1
public final class ChainedUrlNormalizer extends AbstractScopedUrlNormalizer
-
-
Field Summary
Fields Modifier and Type Field Description private IntegermaxLoopsprivate final MultiValuedMap<String, ScopedUrlNormalizer>urlNormalizers
-
Constructor Summary
Constructors Constructor Description ChainedUrlNormalizer()
-
Method Summary
Modifier and Type Method Description final IntegergetMaxLoops()final UnitsetMaxLoops(Integer maxLoops)final MultiValuedMap<String, ScopedUrlNormalizer>getUrlNormalizers()final Unitadd(ScopedUrlNormalizer normalizer, String scope)final Collection<ScopedUrlNormalizer>get(String scope)BooleanisRelevant(String url, String scope)Stringnormalize(String url, String scope)Normalize StringtoString()-
Methods inherited from class ai.platon.pulsar.skeleton.crawl.filter.AbstractScopedUrlNormalizer
invoke -
Methods inherited from class ai.platon.pulsar.skeleton.crawl.filter.ScopedUrlNormalizer
valid -
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
-
Method Detail
-
getMaxLoops
final Integer getMaxLoops()
-
setMaxLoops
final Unit setMaxLoops(Integer maxLoops)
-
getUrlNormalizers
final MultiValuedMap<String, ScopedUrlNormalizer> getUrlNormalizers()
-
add
final Unit add(ScopedUrlNormalizer normalizer, String scope)
-
get
final Collection<ScopedUrlNormalizer> get(String scope)
-
isRelevant
Boolean isRelevant(String url, String scope)
-
normalize
String normalize(String url, String scope)
Normalize
- Parameters:
url- The URL string to normalize.- Returns:
A normalized String, using the given
scope
-
-
-
-