org.sindice.siren.index.codecs.block

Class AForBlockCompressor

java.lang.Object

org.sindice.siren.index.codecs.block.BlockCompressor

org.sindice.siren.index.codecs.block.AForBlockCompressor

public class AForBlockCompressor
extends BlockCompressor

Implementation of BlockCompressor based on the Adaptive Frame Of Reference algorithm.

Adaptive Frame Of Reference (AFOR) attempts to retain the best of FOR (Frame Of Reference), i.e., a very efficient compression and decompression algorithm using highly-optimised routines, while providing a better tolerance against outliers and therefore achieving a higher compression ratio. Compared to PFOR, AFOR does not rely on the encoding of exceptions in the presence of outliers. Instead, AFOR partitions a block into multiple frames of variable length, the partition and the length of the frames being chosen appropriately in order to adapt the encoding to the value distribution.

For more information about AFOR, please refer to the journal article Searching web data: An entity retrieval and high-performance indexing model.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.sindice.siren.index.codecs.block.AForFrameCompressor.FrameCompressor[]	compressors
protected int	HEADER_SIZE Size of header in buffer
protected static int	MAX_FRAME_SIZE
protected static int	MIN_FRAME_SIZE

Constructor Summary

Constructors

Constructor and Description
AForBlockCompressor()

Method Summary

Methods

Modifier and Type	Method and Description
void	compress(org.apache.lucene.util.IntsRef input, org.apache.lucene.util.BytesRef output) Compress the specified list of integers into a byte array.
int	getWindowSize() Return the window size over the IntsRef input buffer used during compression.
int	maxCompressedSize(int arraySize) The maximum size in bytes of a compressed block of values

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HEADER_SIZE

protected final int HEADER_SIZE

Size of header in buffer

See Also:

Constant Field Values

compressors

protected final org.sindice.siren.index.codecs.block.AForFrameCompressor.FrameCompressor[] compressors

MAX_FRAME_SIZE

protected static final int MAX_FRAME_SIZE

See Also:

Constant Field Values

MIN_FRAME_SIZE

protected static final int MIN_FRAME_SIZE

See Also:

Constant Field Values

Constructor Detail

AForBlockCompressor

public AForBlockCompressor()

Method Detail

compress

public void compress(org.apache.lucene.util.IntsRef input,
            org.apache.lucene.util.BytesRef output)

Description copied from class: BlockCompressor

Compress the specified list of integers into a byte array.

Specified by:

compress in class BlockCompressor

maxCompressedSize

public int maxCompressedSize(int arraySize)

Description copied from class: BlockCompressor

The maximum size in bytes of a compressed block of values

Specified by:

maxCompressedSize in class BlockCompressor

getWindowSize

public int getWindowSize()

Description copied from class: BlockCompressor

Return the window size over the IntsRef input buffer used during compression.

This is to avoid ArrayIndexOutOfBoundsException if the compressor is working on a buffer which size does not match a multiple of the window size.

For example, AForBlockCompressor is performing compression using a window of 32 integers. In the case of short postings list with 5 integers, the instantiated input buffer must be a multiple of window size, i.e., 32 in this case. If the postings list would contain 33 integers instead, then the instantiated input buffer should be 64.

Specified by:

getWindowSize in class BlockCompressor