org.sindice.siren.analysis

Class AnyURIAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

org.sindice.siren.analysis.AnyURIAnalyzer

All Implemented Interfaces:

Closeable, AutoCloseable

public class AnyURIAnalyzer
extends org.apache.lucene.analysis.Analyzer

Analyzer designed to deal with any kind of URIs and perform some post-processing on URIs.

The URI normalisation can be configured using setUriNormalisation(URINormalisation). You can disable it, activate it only on URI local name, or on the full URI. However, URI normalisation on the full URI is costly in term of CPU at indexing time, and can double the size of the index, since each URI is duplicated by n tokens. By default, the URI normalisation is disabled.

See Also:

URINormalisationFilter, URILocalnameFilter

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AnyURIAnalyzer.URINormalisation Types of URI normalisation

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.Analyzer.GlobalReuseStrategy, org.apache.lucene.analysis.Analyzer.PerFieldReuseStrategy, org.apache.lucene.analysis.Analyzer.ReuseStrategy, org.apache.lucene.analysis.Analyzer.TokenStreamComponents

Field Summary

Fields

Modifier and Type	Field and Description
static org.apache.lucene.analysis.util.CharArraySet	STOP_WORDS_SET An unmodifiable set containing some common English words that are usually not useful for searching.

Constructor Summary

Constructors

Constructor and Description
AnyURIAnalyzer(org.apache.lucene.util.Version version)
AnyURIAnalyzer(org.apache.lucene.util.Version version, org.apache.lucene.analysis.util.CharArraySet stopWords)
AnyURIAnalyzer(org.apache.lucene.util.Version version, File stopwords)
AnyURIAnalyzer(org.apache.lucene.util.Version version, Reader stopWords)
AnyURIAnalyzer(org.apache.lucene.util.Version version, String[] stopWords)

Method Summary

Methods

Modifier and Type	Method and Description
protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents	createComponents(String fieldName, Reader reader)
void	setUriNormalisation(AnyURIAnalyzer.URINormalisation n)

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getOffsetGap, getPositionIncrementGap, initReader, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STOP_WORDS_SET

public static final org.apache.lucene.analysis.util.CharArraySet STOP_WORDS_SET

An unmodifiable set containing some common English words that are usually not useful for searching.

Constructor Detail

AnyURIAnalyzer

public AnyURIAnalyzer(org.apache.lucene.util.Version version)

AnyURIAnalyzer

public AnyURIAnalyzer(org.apache.lucene.util.Version version,
              org.apache.lucene.analysis.util.CharArraySet stopWords)

AnyURIAnalyzer

public AnyURIAnalyzer(org.apache.lucene.util.Version version,
              String[] stopWords)

AnyURIAnalyzer

public AnyURIAnalyzer(org.apache.lucene.util.Version version,
              File stopwords)
               throws IOException

Throws:

IOException

AnyURIAnalyzer

public AnyURIAnalyzer(org.apache.lucene.util.Version version,
              Reader stopWords)
               throws IOException

Throws:

IOException

Method Detail

setUriNormalisation

public void setUriNormalisation(AnyURIAnalyzer.URINormalisation n)

createComponents

protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents createComponents(String fieldName,
                                                                         Reader reader)

Specified by:

createComponents in class org.apache.lucene.analysis.Analyzer