Package it.unimi.dsi.fastutil.longs

Class LongMappedBigList

java.lang.Object

java.util.AbstractCollection<Long>

it.unimi.dsi.fastutil.longs.AbstractLongCollection

it.unimi.dsi.fastutil.longs.AbstractLongBigList

it.unimi.dsi.fastutil.longs.LongMappedBigList

All Implemented Interfaces:

BigList<Long>, LongBigList, LongCollection, LongIterable, LongStack, Size64, Stack<Long>, Comparable<BigList<? extends Long>>, Iterable<Long>, Collection<Long>

public class LongMappedBigList
extends AbstractLongBigList

A bridge between byte buffers and type-specific big lists.

Java's memory-mapping facilities have the severe limitation of mapping at most Integer.MAX_VALUE bytes, as they expose the content of a file using a MappedByteBuffer. This class can expose a file of primitive types of arbitrary length as a BigList that is actually based on an array of MappedByteBuffers, each mapping a chunk of CHUNK_SIZE longs.

Mapping can happen with a specified byte order: saving and mapping data in native order using methods from BinIO will enhance performance significantly.

Instances of this class are not thread safe, but the copy() method provides a lightweight duplicate that can be read independently by another thread. Only chunks that are actually used will be duplicated lazily. If you are modifiying the content of list, however, you will need to provide external synchronization.

Author:

Sebastiano Vigna

Nested Class Summary

Nested classes/interfaces inherited from class it.unimi.dsi.fastutil.longs.AbstractLongBigList

AbstractLongBigList.LongRandomAccessSubList, AbstractLongBigList.LongSubList

Field Summary

Fields

Modifier and Type	Field	Description
static final long	CHUNK_SIZE	The size in elements of a chunk created by map(FileChannel, ByteOrder, FileChannel.MapMode).
static int	LOG2_BITS	Deprecated. Use LOG2_BYTES.
static int	LOG2_BYTES	The logarithm of the number of bytes of the primitive type of this list.

Method Summary

Modifier and Type	Method	Description
LongMappedBigList	copy()	Returns a lightweight duplicate that can be read independently by another thread.
void	getElements(long from, long[] a, int offset, int length)	Copies (hopefully quickly) elements of this type-specific big list into the given array.
long	getLong(long index)	Returns the element at the specified position.
static LongMappedBigList	map(FileChannel fileChannel)	Creates a new mapped big list by read-only mapping a given file channel using the standard Java (i.e., DataOutput) byte order (ByteOrder.BIG_ENDIAN).
static LongMappedBigList	map(FileChannel fileChannel, ByteOrder byteOrder)	Creates a new mapped big list by read-only mapping a given file channel.
static LongMappedBigList	map(FileChannel fileChannel, ByteOrder byteOrder, FileChannel.MapMode mapMode)	Creates a new mapped big list by mapping a given file channel.
long	set(long index, long value)	Replaces the element at the specified position in this big list with the specified element (optional operation).
long	size64()	Returns the size of this data structure as a long.

Methods inherited from class it.unimi.dsi.fastutil.longs.AbstractLongBigList

add, add, add, addAll, addAll, addAll, addAll, addElements, addElements, clear, compareTo, contains, equals, forEach, get, getElements, hashCode, indexOf, indexOf, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, peek, peekLong, pop, popLong, push, push, rem, remove, removeElements, removeLong, set, setElements, size, size, subList, top, topLong, toString

Methods inherited from class it.unimi.dsi.fastutil.longs.AbstractLongCollection

add, contains, containsAll, containsAll, forEach, remove, removeAll, removeAll, removeIf, retainAll, retainAll, toArray, toLongArray, toLongArray

Methods inherited from class java.util.AbstractCollection

isEmpty, toArray, toArray

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Collection

containsAll, isEmpty, removeAll, retainAll, toArray, toArray, toArray

Methods inherited from interface it.unimi.dsi.fastutil.longs.LongBigList

addAll, addAll, addAll, addAll, setElements, setElements, spliterator

Methods inherited from interface it.unimi.dsi.fastutil.longs.LongCollection

add, contains, containsAll, longIterator, longParallelStream, longSpliterator, longStream, parallelStream, remove, removeAll, removeIf, removeIf, removeIf, retainAll, stream, toArray, toLongArray, toLongArray

Methods inherited from interface it.unimi.dsi.fastutil.longs.LongIterable

forEach, forEach

Methods inherited from interface it.unimi.dsi.fastutil.Stack

isEmpty

Field Details

LOG2_BYTES

public static int LOG2_BYTES

The logarithm of the number of bytes of the primitive type of this list.

LOG2_BITS

@Deprecated
public static int LOG2_BITS

Deprecated.

Use LOG2_BYTES.

CHUNK_SIZE

public static final long CHUNK_SIZE

The size in elements of a chunk created by map(FileChannel, ByteOrder, FileChannel.MapMode).

Method Details

map

public static LongMappedBigList map(FileChannel fileChannel)
                             throws IOException

Creates a new mapped big list by read-only mapping a given file channel using the standard Java (i.e., DataOutput) byte order (ByteOrder.BIG_ENDIAN).

Parameters:

fileChannel - the file channel that will be mapped.

Returns:

a new read-only mapped big list over the contents of fileChannel.

Throws:

IOException

See Also:

• map(FileChannel, ByteOrder, MapMode)

map

public static LongMappedBigList map(FileChannel fileChannel,
 ByteOrder byteOrder)
                             throws IOException

Creates a new mapped big list by read-only mapping a given file channel.

Parameters:

fileChannel - the file channel that will be mapped.

byteOrder - a prescribed byte order.

Returns:

a new read-only mapped big list over the contents of fileChannel.

Throws:

IOException

See Also:

• map(FileChannel, ByteOrder, MapMode)

map

public static LongMappedBigList map(FileChannel fileChannel,
 ByteOrder byteOrder,
 FileChannel.MapMode mapMode)
                             throws IOException

Creates a new mapped big list by mapping a given file channel.

Parameters:

fileChannel - the file channel that will be mapped.

byteOrder - a prescribed byte order.

mapMode - the mapping mode: usually FileChannel.MapMode.READ_ONLY, but if intend to make the list mutable, you have to pass FileChannel.MapMode.READ_WRITE.

Returns:

a new mapped big list over the contents of fileChannel.

Throws:

IOException

copy

public LongMappedBigList copy()

Returns a lightweight duplicate that can be read independently by another thread.

Only chunks that are actually used will be duplicated lazily.

Returns:

a lightweight duplicate that can be read independently by another thread.

getLong

public long getLong(long index)

Description copied from interface: LongBigList

Returns the element at the specified position.

See Also:

• BigList.get(long)

getElements

public void getElements(long from,
 long[] a,
 int offset,
 int length)

Description copied from interface: LongBigList

Copies (hopefully quickly) elements of this type-specific big list into the given array.

Parameters:

from - the start index (inclusive).

a - the destination array.

offset - the offset into the destination array where to store the first element copied.

length - the number of elements to be copied.

set

public long set(long index,
 long value)

Description copied from class: AbstractLongBigList

Replaces the element at the specified position in this big list with the specified element (optional operation).

Specified by:

set in interface LongBigList

Overrides:

set in class AbstractLongBigList

See Also:

• BigList.set(long,Object)

size64

public long size64()

Description copied from interface: Size64

Returns the size of this data structure as a long.

Returns:

the size of this data structure.