Internal API¶
These classes are only intended for internal use in ndindex. They shouldn’t relied on as they may be removed or changed.
Note that the documentation for methods on ndindex classes will sometimes link
to this page because the methods are defined on the on the
ImmutableObject
, NDIndex
, or ArrayIndex
base
classes. These classes are not designed to be used directly. Such methods are
present on all ndindex classes, which are what should be
actually be constructed. Remember that the primary entry-point API for
constructing ndindex index classes is the ndindex()
function.
Base Classes¶
- class ndindex.ndindex.ImmutableObject(*args, **kwargs)[source]¶
Base class for immutable objects.
Subclasses of this class are immutable objects. They all have the
.args
attribute, which gives the full necessary data to recreate the class, via,type(obj)(*obj.args) == obj
Note: subclasses that specifically represent indices should subclass
NDIndex
instead.All classes that subclass
ImmutableObject
should define the_typecheck
method._typecheck(self, *args)
should do type checking and basic type canonicalization, and either return a tuple of the new arguments for the class or raise an exception. Type checking means it should raise exceptions for input types that are never semantically meaningful for numpy arrays, for example, floating point indices, using the same exceptions as numpy where possible. Basic type canonicalization means, for instance, converting integers intoint
usingoperator.index()
. All other canonicalization should be done in thereduce()
method. TheImmutableObject
base constructor will automatically set.args
to the arguments returned by this method. Classes should always be able to recreate themselves with.args
, i.e.,type(obj)(*obj.args) == obj
should always hold.See also
- args¶
idx.args
contains the arguments needed to createidx
.For an ndindex object
idx
,idx.args
is always a tuple such thattype(idx)(*idx.args) == idx
For
Tuple
indices, the elements of.args
are themselves ndindex types. For other types,.args
contains raw Python types. Note that.args
contains NumPy arrays forIntegerArray
andBooleanArray
types, so one should always do equality testing or hashing on the ndindex type itself, not its.args
.
- class ndindex.ndindex.NDIndex(*args, **kwargs)[source]¶
Represents an index into an nd-array (i.e., a numpy array).
This is a base class for all ndindex types. All types that subclass this class should redefine the following methods
_typecheck(self, *args)
. See the docstring ofImmutableObject
.raw
(a @property method) should return the raw index that can be passed as an index to a numpy array.
In addition other methods on this should be re-defined as necessary. Some methods have a default implementation on this class, which is sufficient for some subclasses.
The methods
__init__
and__eq__
should not be overridden. Equality (and hashability) onNDIndex
subclasses is determined by equality of types and.args
. Equivalent indices should not attempt to redefine equality. Rather they should define canonicalization viareduce()
.__hash__
is defined so that the hash matches the hash of.raw
. If.raw
is unhashable,__hash__
should be overridden to usehash(self.args)
.See also
- args¶
idx.args
contains the arguments needed to createidx
.For an ndindex object
idx
,idx.args
is always a tuple such thattype(idx)(*idx.args) == idx
For
Tuple
indices, the elements of.args
are themselves ndindex types. For other types,.args
contains raw Python types. Note that.args
contains NumPy arrays forIntegerArray
andBooleanArray
types, so one should always do equality testing or hashing on the ndindex type itself, not its.args
.
- as_subindex(index)[source]¶
i.as_subindex(j)
produces an indexk
such thata[j][k]
gives all of the elements ofa[j]
that are also ina[i]
.If
a[j]
is a subset ofa[i]
, thena[j][k] == a[i]
. Otherwise,a[j][k] == a[i & j]
, wherei & j
is the intersection ofi
andj
, that is, the elements ofa
that are indexed by bothi
andj
.For example, in the below diagram,
i
andj
index a subset of the arraya
.k = i.as_subindex(j)
is an index ona[j]
that gives the subset ofa[j]
also included ina[i]
:+------------ self ------------+ | | ------------------- a ----------------------- | | +------------- index -------------+ | | +- self.as_subindex(index) -+
i.as_subindex(j)
is currently only implemented whenj
is a slice with positive steps and nonnegative start and stop, or a Tuple of the same. To use it with slices with negative start or stop, callreduce()
with a shape first.as_subindex
can be seen as the left-inverse of composition, that is, ifa[i] = a[j][k]
, thenk = i.as_subindex(j)
, so thatk "=" (j^-1)[i]
(this only works as a true inverse ifj
is a subset ofi
).Note that due to symmetry,
a[j][i.as_subindex(j)]
anda[i][j.as_subindex(i)]
will give the same subarrays ofa
, which will be the array of elements indexed by botha[i]
anda[j]
.i.as_subindex(j)
may raiseValueError
in the case that the indicesi
andj
do not intersect at all.Examples
An example usage of
as_subindex
is to split an index up into subindices of chunks of an array. For example, say a 1-D arraya
is chunked up into chunks of sizeN
, so thata[0:N]
,a[N:2*N]
,[2*N:3*N]
, etc. are stored separately. Then an indexa[i]
can be reindexed onto the chunks viai.as_subindex(Slice(0, N))
,i.as_subindex(Slice(N, 2*N))
, etc.>>> from ndindex import Slice >>> i = Slice(5, 15) >>> j1 = Slice(0, 10) >>> j2 = Slice(10, 20) >>> a = list(range(20)) >>> a[i.raw] [5, 6, 7, 8, 9, 10, 11, 12, 13, 14] >>> a[j1.raw] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> a[j2.raw] [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> k1 = i.as_subindex(j1) >>> k1 Slice(5, 10, 1) >>> k2 = i.as_subindex(j2) >>> k2 Slice(0, 5, 1) >>> a[j1.raw][k1.raw] [5, 6, 7, 8, 9] >>> a[j2.raw][k2.raw] [10, 11, 12, 13, 14]
See also
ndindex.ChunkSize.as_subchunks
a high-level iterator that efficiently gives only those chunks that intersect with a given index
- broadcast_arrays()[source]¶
Broadcast all the array indices in self to a common shape and convert boolean array indices into integer array indices.
The resulting index is equivalent in all contexts where the original index is allowed. However, it is possible for the original index to give an IndexError but for the new index to not, since integer array indices have less stringent shape requirements than boolean array indices. There are also some instances for empty indices (
isempty
is True) where bounds would be checked before broadcasting but not after.Any
BooleanArray
indices are converted toIntegerArray
indices. Furthermore, if there areBooleanArray
orIntegerArray
indices, then anyInteger
indices are also converted into scalarIntegerArray
indices and broadcast. Furthermore, if there are multiple boolean scalar indices (True
orFalse
), they are combined into a single one.Note that array broadcastability is checked in the
Tuple
constructor, so this method will not raise any exceptions.This is part of what is performed by
expand
, but unlikeexpand
, this method does not do any other manipulations, and it does not require a shape.>>> from ndindex import Tuple >>> idx = Tuple([[False], [True], [True]], [[4], [5], [5]], -1) >>> print(idx.broadcast_arrays()) Tuple(IntegerArray([[1 2] [1 2] [1 2]]), IntegerArray([[0 0] [0 0] [0 0]]), IntegerArray([[4 4] [5 5] [5 5]]), IntegerArray([[-1 -1] [-1 -1] [-1 -1]]))
See also
- expand(shape)[source]¶
Expand a Tuple index on an array of shape
shape
An expanded index is as explicit as possible. Unlike
reduce
, which tries to simplify an index and remove redundancies,expand()
typically makes an index larger.If
self
is invalid for the given shape, anIndexError
is raised. Otherwise, the returned index satisfies the following:It is always a
Tuple
.All the elements of the
Tuple
are recursivelyreduced
.The length of the
.args
is equal to the length of the shape plus the number ofNewaxis
indices inself
plus 1 if there is a scalarBooleanArray
(True
orFalse
).The resulting
Tuple
has noellipses
. If there are axes that would be matched by an ellipsis or an implicit ellipsis at the end of the tuple,Slice(0, n, 1)
indices are inserted, wheren
is the corresponding axis of theshape
.Any array indices in
self
are broadcast together. Ifself
contains array indices (IntegerArray
orBooleanArray
), then anyInteger
indices are converted intoIntegerArray
indices of shape()
and broadcast. Note that broadcasting is done in a memory efficient way so that even if the broadcasted shape is large it will not take up more memory than the original.Scalar
BooleanArray
arguments (True
orFalse
) are combined into a single term (the same as withTuple.reduce()
).Non-scalar
BooleanArray
s are all converted into equivalentIntegerArray
s vianonzero()
and broadcasted.
>>> from ndindex import Tuple, Slice >>> Slice(None).expand((2, 3)) Tuple(slice(0, 2, 1), slice(0, 3, 1))
>>> idx = Tuple(slice(0, 10), ..., None, -3) >>> idx.expand((5, 3)) Tuple(slice(0, 5, 1), None, 0) >>> idx.expand((1, 2, 3)) Tuple(slice(0, 1, 1), slice(0, 2, 1), None, 0) >>> idx.expand((5,)) Traceback (most recent call last): ... IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed >>> idx.expand((5, 2)) Traceback (most recent call last): ... IndexError: index -3 is out of bounds for axis 1 with size 2
>>> idx = Tuple(..., [0, 1], -1) >>> idx.expand((1, 2, 3)) Tuple(slice(0, 1, 1), [0, 1], [2, 2])
See also
- isempty(shape=None)[source]¶
Returns whether self always indexes an empty array
An empty array is an array whose shape contains at least one 0. Note that scalars (arrays with shape
()
) are not considered empty.shape
can beNone
(the default), or an array shape. If it isNone
, isempty() will returnTrue
whenself
is always empty for any array shape. However, if it givesFalse
, it could still give an empty array for some array shapes, but not all. If you know the shape of the array that will be indexed, useidx.isempty(shape)
and the result will be correct for arrays of shapeshape
. Ifshape
is given andself
would raise anIndexError
on an array of shapeshape
,isempty()
also raisesIndexError
.>>> from ndindex import Tuple, Slice >>> Tuple(0, slice(0, 1)).isempty() False >>> Tuple(0, slice(0, 0)).isempty() True >>> Slice(5, 10).isempty() False >>> Slice(5, 10).isempty(4) True
See also
- isvalid(shape)[source]¶
Check whether a given index is valid on an array of a given shape.
Returns
True
if an array of shapeshape
can be indexed byself
andFalse
if it would raiseIndexError
.>>> from ndindex import ndindex >>> ndindex(3).isvalid((4,)) True >>> ndindex(3).isvalid((2,)) False
Note that some indices can never be valid and will raise a
IndexError
orTypeError
if you attempt to construct them.>>> ndindex((..., 0, ...)) Traceback (most recent call last): ... IndexError: an index can only have a single ellipsis ('...') >>> ndindex(slice(True)) Traceback (most recent call last): ... TypeError: 'bool' object cannot be interpreted as an integer
See also
- newshape(shape)[source]¶
Returns the shape of
a[idx.raw]
, assuminga
has shapeshape
.shape
should be a tuple of ints, or an int, which is equivalent to a 1-D shape.Raises
IndexError
ifself
would be invalid for an array of shapeshape
.>>> from ndindex import Slice, Integer, Tuple >>> shape = (6, 7, 8) >>> Integer(1).newshape(shape) (7, 8) >>> Integer(10).newshape(shape) Traceback (most recent call last): ... IndexError: index 10 is out of bounds for axis 0 with size 6 >>> Slice(2, 5).newshape(shape) (3, 7, 8) >>> Tuple(0, ..., Slice(1, 3)).newshape(shape) (7, 2)
See also
- property raw¶
Return the equivalent of
self
that can be used as an indexNumPy does not allow custom objects to be used as indices, with the exception of integer indices, so to use an ndindex object as an index, it is necessary to use
raw
.>>> from ndindex import Slice >>> import numpy as np >>> a = np.arange(5) >>> s = Slice(2, 4) >>> a[s] Traceback (most recent call last): ... IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices >>> a[s.raw] array([2, 3])
- reduce(shape=None, *, negative_int=False)[source]¶
Simplify an index given that it will be applied to an array of a given shape.
If
shape
is None (the default), the index will be canonicalized as much as possible while still staying equivalent for all array shapes that it does not raise IndexError for.Either returns a new index type, which is equivalent on arrays of shape
shape
, or raises IndexError if the index would give an index error (for instance, out of bounds integer index or too many indices for array).>>> from ndindex import Slice, Integer >>> Slice(0, 10).reduce((5,)) Slice(0, 5, 1) >>> Integer(10).reduce((5,)) Traceback (most recent call last): ... IndexError: index 10 is out of bounds for axis 0 with size 5
For single axis indices such as Slice and Tuple,
reduce
takes an optionalaxis
argument to specify the axis, defaulting to 0.
- selected_indices(shape, axis=0)[source]¶
Return an iterator over all indices that are selected by
self
on an array of shapeshape
.The result is a set of indices
i
such that[a[i] for i in idx.selected_indices(a.shape)]
is all the elements ofa[idx]
. The indices are all iterated over in C (i.e., row major) order.>>> from ndindex import Slice, Tuple >>> idx = Slice(5, 10) >>> list(idx.selected_indices(20)) [Integer(5), Integer(6), Integer(7), Integer(8), Integer(9)] >>> idx = Tuple(Slice(5, 10), Slice(0, 2)) >>> list(idx.selected_indices((20, 3))) [Tuple(5, 0), Tuple(5, 1), Tuple(6, 0), Tuple(6, 1), Tuple(7, 0), Tuple(7, 1), Tuple(8, 0), Tuple(8, 1), Tuple(9, 0), Tuple(9, 1)]
To correspond these indices to the elements of
a[idx]
, you can useiter_indices(idx.newshape(shape))
, since both iterators iterate the indices in C order.>>> from ndindex import iter_indices >>> idx = Tuple(Slice(3, 5), Slice(0, 2)) >>> shape = (5, 5) >>> import numpy as np >>> a = np.arange(25).reshape(shape) >>> for a_idx, (new_idx,) in zip( ... idx.selected_indices(shape), ... iter_indices(idx.newshape(shape))): ... print(a_idx, new_idx, a[a_idx.raw], a[idx.raw][new_idx.raw]) Tuple(3, 0) Tuple(0, 0) 15 15 Tuple(3, 1) Tuple(0, 1) 16 16 Tuple(4, 0) Tuple(1, 0) 20 20 Tuple(4, 1) Tuple(1, 1) 21 21
See also
ndindex.iter_indices
An iterator of indices to select every element for arrays of a given shape.
ndindex.ChunkSize.as_subchunks
A high-level iterator that efficiently gives only those chunks that intersect with a given index
- class ndindex.array.ArrayIndex(idx, shape=None, _copy=True)[source]¶
Superclass for array indices
This class should not be instantiated directly. Rather, use one of its subclasses,
IntegerArray
orBooleanArray
.To subclass this, define the
dtype
attribute, as well as all the usual ndindex methods.- dtype Subclasses should redefine this¶
- property array¶
Return the NumPy array of self.
This is the same as
self.args[0]
.>>> from ndindex import IntegerArray, BooleanArray >>> IntegerArray([0, 1]).array array([0, 1]) >>> BooleanArray([False, True]).array array([False, True])
- isvalid(shape, _axis=0)[source]¶
Check whether a given index is valid on an array of a given shape.
Returns
True
if an array of shapeshape
can be indexed byself
andFalse
if it would raiseIndexError
.>>> from ndindex import ndindex >>> ndindex(3).isvalid((4,)) True >>> ndindex(3).isvalid((2,)) False
Note that some indices can never be valid and will raise a
IndexError
orTypeError
if you attempt to construct them.>>> ndindex((..., 0, ...)) Traceback (most recent call last): ... IndexError: an index can only have a single ellipsis ('...') >>> ndindex(slice(True)) Traceback (most recent call last): ... TypeError: 'bool' object cannot be interpreted as an integer
See also
- property ndim¶
Return the number of dimensions of the array of self.
This is the same as
self.array.ndim
. Note that this is not the same as the number of dimensions of an array that is indexed byself
. Uselen
onnewshape()
to get that.>>> from ndindex import IntegerArray, BooleanArray >>> IntegerArray([[0], [1]]).ndim 2 >>> BooleanArray([[False], [True]]).ndim 2
- property raw¶
Return the equivalent of
self
that can be used as an indexNumPy does not allow custom objects to be used as indices, with the exception of integer indices, so to use an ndindex object as an index, it is necessary to use
raw
.>>> from ndindex import Slice >>> import numpy as np >>> a = np.arange(5) >>> s = Slice(2, 4) >>> a[s] Traceback (most recent call last): ... IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices >>> a[s.raw] array([2, 3])
- property shape¶
Return the shape of the array of self.
This is the same as
self.array.shape
. Note that this is not the same as the shape of an array that is indexed byself
. Usenewshape()
to get that.>>> from ndindex import IntegerArray, BooleanArray >>> IntegerArray([[0], [1]]).shape (2, 1) >>> BooleanArray([[False], [True]]).shape (2, 1)
- property size¶
Return the number of elements of the array of self.
This is the same as
self.array.size
. Note that this is not the same as the number of elements of an array that is indexed byself
. Usenp.prod
onnewshape()
to get that.>>> from ndindex import IntegerArray, BooleanArray >>> IntegerArray([[0], [1]]).size 2 >>> BooleanArray([[False], [True]]).size 2
Other Internal Functions¶
- class ndindex.slice.default[source]¶
A default keyword argument value.
Used as the default value for keyword arguments where
None
is also a meaningful value but not the default.
- ndindex.ndindex.operator_index(idx)[source]¶
Convert
idx
into an integer index using__index__()
or raiseTypeError
.This is the same as
operator.index()
except it disallows boolean types.This is a slight break in NumPy compatibility, as NumPy allows bools in some contexts where
__index__()
is used, for instance, in slices. It does disallow it in others, such as in shapes. The main motivation for disallowing bools entirely is 1)numpy.bool_.__index__()
is deprecated (currently it matches the built-inbool.__index__()
and returns the object unchanged, but prints a deprecation warning), and 2) for raw indices, booleans and0
/1
are completely different, i.e.,a[True]
is not the same asa[1]
.>>> from ndindex.ndindex import operator_index >>> operator_index(1) 1 >>> operator_index(1.0) Traceback (most recent call last): ... TypeError: 'float' object cannot be interpreted as an integer >>> operator_index(True) Traceback (most recent call last): ... TypeError: 'bool' object cannot be interpreted as an integer
- ndindex.shapetools.asshape(shape, axis=None, *, allow_int=True, allow_negative=False)[source]¶
Cast
shape
as a valid NumPy shape.The input can be an integer
n
(ifallow_int=True
), which is equivalent to(n,)
, or a tuple of integers.If the
axis
argument is provided, anIndexError
is raised if it is out of bounds for the shape.The resulting shape is always a tuple of nonnegative integers. If
allow_negative=True
, negative integers are also allowed.All ndindex functions that take a shape input should use:
shape = asshape(shape)
or:
shape = asshape(shape, axis=axis)
- ndindex.shapetools.ncycles(iterable, n)[source]¶
Iterate
iterable
repeatedn
times.This is based on a recipe from the Python itertools docs, but improved to give a repr, and to denest when it can. This makes debugging
iter_indices()
easier.This is only intended for internal usage.
>>> from ndindex.shapetools import ncycles >>> ncycles(range(3), 2) ncycles(range(0, 3), 2) >>> list(_) [0, 1, 2, 0, 1, 2] >>> ncycles(ncycles(range(3), 3), 2) ncycles(range(0, 3), 6)
- ndindex.shapetools.associated_axis(broadcasted_shape, i, skip_axes)[source]¶
Return the associated element of
broadcasted_shape
corresponding toshape[i]
givenskip_axes
. If there is not such element (i.e., it’s out of bounds), returns None.This function makes implicit assumptions about its input and is only designed for internal use.
- ndindex.shapetools.remove_indices(x, idxes)[source]¶
Return
x
with the indicesidxes
removed.This function is only intended for internal usage.
- ndindex.shapetools.unremove_indices(x, idxes, *, val=None)[source]¶
Insert
val
inx
so that it appears atidxes
.Note that idxes must be either all negative or all nonnegative.
This function is only intended for internal usage.
- ndindex.shapetools.normalize_skip_axes(shapes, skip_axes)[source]¶
Return a canonical form of
skip_axes
corresponding toshapes
.A canonical form of
skip_axes
is a list of tuples of integers, one for each shape inshapes
, which are a unique set of axes for each corresponding shape.If
skip_axes
is an integer, this is basically[(skip_axes,) for s in shapes]
. Ifskip_axes
is a tuple, it is like[skip_axes for s in shapes]
.The
skip_axes
must always refer to unique axes in each shape.The returned
skip_axes
will always be negative integers and will be sorted.This function is only intended for internal usage.