`hikari.iterators`#

Lazy iterators for data that requires repeated API calls to retrieve.

For consumers of this API, the only class you need to worry about is hikari.iterators.LazyIterator. Everything else is internal detail only exposed for people who wish to extend this API further!

AnotherValueT `module-attribute` #

AnotherValueT = TypeVar('AnotherValueT')

Type-hint of the type of a value by a mapped lazy iterator.

ValueT `module-attribute` #

ValueT = TypeVar('ValueT')

Type-hint of the type of the value returned by a lazy iterator.

All #

All(conditions: Collection[Callable[[ValueT], bool]])

Bases: Generic[ValueT]

Helper that wraps predicates and invokes them together.

Calling this object will pass the input item to each item, returning True only when all wrapped predicates return True when called with the given item.

For example...

if w(foo) and x(foo) and y(foo) and z(foo):
    ...

is equivalent to

condition = All([w, x, y, z])

if condition(foo):
    ...

This behaves like a lazy wrapper implementation of the all builtin.

Note

Like the rest of the standard library, this is a short-circuiting operation. This means that if a predicate returns False, no predicates after this are invoked, as the result is already known. In this sense, they are invoked in-order.

Warning

You should not generally need to use this outside of extending the iterators API in this library!

Operators

this(value : ValueT) -> bool: Return True if all conditions return True when invoked with the given value.
~this: Return a condition that, when invoked with the value, returns False if all conditions were True in this object.

PARAMETER	DESCRIPTION
`conditions`	The predicates to wrap. TYPE: `Collection[Callable[[ValueT], bool]]`

conditions `instance-attribute` #

conditions: Collection[Callable[[ValueT], bool]] = conditions

Collection of the condition callbacks within this.

AttrComparator #

AttrComparator(attr_name: str, expected_value: Any, cast: Optional[Callable[[ValueT], Any]] = None)

Bases: Generic[ValueT]

A comparator that compares the result of a call with something else.

This uses the spel module internally.

PARAMETER	DESCRIPTION
`attr_name`	The attribute name. Can be prepended with a `.` optionally. If the attribute name ends with a `()`, then the call is invoked rather than treated as a property (useful for methods like `str.isupper`, for example). TYPE: `str`
`expected_value`	The expected value. TYPE: `Any`
`cast`	Optional cast to perform on the input value when being called before comparing it to the expected value but after accessing the attribute. TYPE: `Optional[Callable[[ValueT], Any]]` DEFAULT: `None`

BufferedLazyIterator #

BufferedLazyIterator()

Bases: Generic[ValueT], LazyIterator[ValueT], ABC

A special kind of lazy iterator that is used by internal components.

The purpose of this is to provide an interface to lazily deserialize collections of payloads received from paginated API endpoints such as GET /channels/{channel_id}/messages, which will return a certain number of messages at a time on a low level. This class provides the base interface for handling lazily decoding each item in those responses and returning them in the expected format when iterating across this object.

Implementations are expected to provide a _next_chunk private method which when awaited returns a lazy generator of each deserialized object to later yield. This will be iterated across lazily by this implementation, thus reducing the amount of work needed if only a few objects out of, say, 100, need to be deserialized.

This _next_chunk should return None once the end of all items has been reached.

An example would look like the following:

async def some_http_call(i): ...


class SomeEndpointLazyIterator(BufferedLazyIterator[SomeObject]):
    def __init__(self):
        super().__init__()
        self._i = 0

    def _next_chunk(
        self,
    ) -> typing.Optional[typing.Generator[ValueT, None, None]]:
        raw_items = await some_http_call(self._i)
        self._i += 1

        if not raw_items:
            return None

        generator = (SomeObject(raw_item) for raw_item in raw_items)
        return generator

FlatLazyIterator #

FlatLazyIterator(values: Iterable[ValueT])

Bases: Generic[ValueT], LazyIterator[ValueT]

A lazy iterator that has all items in-memory and ready.

This can be iterated across as a normal iterator, or as an async iterator.

LazyIterator #

Bases: Generic[ValueT], ABC

A set of results that are fetched asynchronously from the API as needed.

This is a typing.AsyncIterable and typing.AsyncIterator with several additional helpful methods provided for convenience.

Examples:

You can use this in multiple ways.

As an async iterable:

>>> async for item in paginated_results:
...    process(item)

As an eagerly retrieved set of results (performs all API calls at once, which may be slow for large sets of data):

>>> results = await paginated_results
>>> # ... which is equivalent to this...
>>> results = [item async for item in paginated_results]

As an async iterator (not recommended):

>>> try:
...    while True:
...        process(await paginated_results.__anext__())
... except StopAsyncIteration:
...    pass

Additionally, you can make use of some of the provided helper methods on this class to perform basic operations easily.

Iterating across the items with indexes (like enumerate for normal iterables):

>>> async for i, item in paginated_results.enumerate():
...    print(i, item)
(0, foo)
(1, bar)
(2, baz)

Limiting the number of results you iterate across:

>>> async for item in paginated_results.limit(3):
...    process(item)

awaiting #

awaiting(window_size: int = 10) -> LazyIterator[ValueT]

Await each item concurrently in a fixed size window.

Warning

Setting a large window size, or setting it to 0 to await everything is a dangerous thing to do if you are making API calls. Some endpoints will get ratelimited and cause a backup of waiting tasks, others may begin to spam global rate limits instead (the fetch_user endpoint seems to be notorious for doing this).

Note

This call assumes that the iterator contains awaitable values as input. MyPy cannot detect this nicely, so any cast is forced internally. If the item is not awaitable, you will receive a TypeError instead. You have been warned. You cannot escape the ways of the duck type young grasshopper.

PARAMETER	DESCRIPTION
`window_size`	The window size of how many tasks to await at once. You can set this to `0` to await everything at once, but see the below warning. TYPE: `int` DEFAULT: `10`

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	The new lazy iterator to return.

chunk #

chunk(chunk_size: int) -> LazyIterator[Sequence[ValueT]]

Return results in chunks of up to chunk_size amount of entries.

PARAMETER	DESCRIPTION
`chunk_size`	The limit for how many results should be returned in each chunk. TYPE: `int`

RETURNS	DESCRIPTION
`LazyIterator[Sequence[ValueT]]`	`hikari.iterators.LazyIterator` that emits each chunked sequence.

collect `async` #

collect(collector: Callable[[Sequence[ValueT]], Collection[ValueT]]) -> Collection[ValueT]

Collect the results into a given type and return it.

PARAMETER	DESCRIPTION
`collector`	A function that consumes a sequence of values and returns a collection. TYPE: `Callable[[Sequence[ValueT]], Collection[ValueT]]`

count `async` #

count() -> int

Count the number of results.

RETURNS	DESCRIPTION
`int`	Number of results found.

enumerate #

enumerate(*, start: int = 0) -> LazyIterator[tuple[int, ValueT]]

Enumerate the paginated results lazily.

This behaves as an asyncio-friendly version of enumerate which uses much less memory than collecting all the results first and calling enumerate across them.

PARAMETER	DESCRIPTION
`start`	Optional int to start at. If omitted, this is `0`. TYPE: `int` DEFAULT: `0`

Examples:

>>> async for i, item in paginated_results.enumerate():
...    print(i, item)
(0, foo)
(1, bar)
(2, baz)
(3, bork)
(4, qux)

>>> async for i, item in paginated_results.enumerate(start=9):
...    print(i, item)
(9, foo)
(10, bar)
(11, baz)
(12, bork)
(13, qux)

>>> async for i, item in paginated_results.enumerate(start=9).limit(3):
...    print(i, item)
(9, foo)
(10, bar)
(11, baz)

RETURNS	DESCRIPTION
`LazyIterator[Tuple[int, T]]`	A paginated results view that asynchronously yields an increasing counter in a tuple with each result, lazily.

filter #

filter(*predicates: Union[tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any) -> LazyIterator[ValueT]

Filter the items by one or more conditions.

Each condition is treated as a predicate, being called with each item that this iterator would return when it is requested.

All conditions must evaluate to True for the item to be returned. If this is not met, then the item is discarded and ignored, the next matching item will be returned instead, if there is one.

PARAMETER DESCRIPTION

*predicates

Predicates to invoke. These are functions that take a value and return True if it is of interest, or False otherwise. These may instead include 2-tuple objects consisting of a str attribute name (nested attributes are referred to using the . operator), and values to compare for equality. This allows you to specify conditions such as members.filter(("user.bot", True)).

TYPE: Union[tuple[str, Any], Callable[[ValueT], bool]] DEFAULT: ()

**attrs

Alternative to passing 2-tuples. Cannot specify nested attributes using this method.

TYPE: Any DEFAULT: {}

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	`hikari.iterators.LazyIterator` that only emits values where all conditions are matched.

flat_map #

flat_map(flattener: _FlattenerT[ValueT, AnotherValueT]) -> LazyIterator[AnotherValueT]

Perform a flat mapping operation.

This will pass each item in the iterator to the given function parameter, expecting a new typing.Iterable or typing.AsyncIterator to be returned as the result. This means you can map to a new hikari.iterators.LazyIterator, typing.AsyncIterator, typing.Iterable, async generator, or generator.

Remember that typing.Iterator implicitly provides typing.Iterable compatibility.

This is used to provide lazy conversions, and can be used to implement reactive-like pipelines if desired.

All results are combined into one large lazy iterator and yielded lazily.

PARAMETER	DESCRIPTION
`flattener`	A function that returns either an async iterator or iterator of new values. Could be an attribute name instead. TYPE: `_FlattenerT[ValueT, AnotherValueT]`

Examples:

The following example generates a distinct collection of all mentioned users in the given channel from the past 500 messages.

def iter_mentioned_users(message: hikari.Message) -> typing.Iterable[Snowflake]:
    for match in re.findall(r"<@!?(\d+)>", message.content):
        yield Snowflake(match)


mentioned_users = await (
    channel.history()
    .limit(500)
    .map(".content")
    .flat_map(iter_mentioned_users)
    .distinct()
)

RETURNS	DESCRIPTION
`LazyIterator[AnotherValueT]`	The new lazy iterator to return.

flatten #

flatten() -> LazyIterator[AnotherValueT]

Flatten an async iterator of iterables.

for_each `async` #

for_each(consumer: Callable[[ValueT], Any]) -> None

Forward each value to a given consumer immediately.

last `async` #

last() -> ValueT

Return the last element of this iterator only.

Note

This method will consume the whole iterator if run.

RETURNS	DESCRIPTION
`ValueT`	The last result.

RAISES	DESCRIPTION
`LookupError`	If no result exists.

limit #

limit(limit: int) -> LazyIterator[ValueT]

Limit the number of items you receive from this async iterator.

PARAMETER	DESCRIPTION
`limit`	The number of items to get. This must be greater than zero. TYPE: `int`

Examples:

    >>> async for item in paginated_results.limit(3):
    ...     print(item)

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	A paginated results view that asynchronously yields a maximum of the given number of items before completing.

map #

map(transformation: Union[Callable[[ValueT], AnotherValueT], str]) -> LazyIterator[AnotherValueT]

Map the values to a different value.

PARAMETER	DESCRIPTION
`transformation`	The function to use to map the attribute. This may alternatively be a string attribute name to replace the input value with. You can provide nested attributes using the `.` operator. TYPE: `Union[Callable[[ValueT], AnotherValueT], str]`

RETURNS	DESCRIPTION
`LazyIterator[AnotherValueT]`	`hikari.iterators.LazyIterator` that maps each value to another value.

next `async` #

next() -> ValueT

Return the next element of this iterator only.

RETURNS	DESCRIPTION
`ValueT`	The next result.

RAISES	DESCRIPTION
`LookupError`	If no more results exist.

reversed #

reversed() -> LazyIterator[ValueT]

Return a lazy iterator of the remainder of this iterator's values reversed.

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	The lazy iterator of this iterator's remaining values reversed.

skip #

skip(number: int) -> LazyIterator[ValueT]

Drop the given number of items, then yield anything after.

PARAMETER	DESCRIPTION
`number`	The max number of items to drop before any items are yielded. TYPE: `int`

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	A paginated results view that asynchronously yields all items AFTER the given number of items are discarded first.

skip_until #

skip_until(*predicates: Union[tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any) -> LazyIterator[ValueT]

Discard items while all conditions are False.

Items after this will be yielded as normal.

PARAMETER DESCRIPTION

*predicates

Predicates to invoke. These are functions that take a value and return True if it is of interest, or False otherwise. These may instead include 2-tuple objects consisting of a str attribute name (nested attributes are referred to using the . operator), and values to compare for equality. This allows you to specify conditions such as members.skip_until(("user.bot", True)).

TYPE: Union[tuple[str, Any], Callable[[ValueT], bool]] DEFAULT: ()

**attrs

Alternative to passing 2-tuples. Cannot specify nested attributes using this method.

TYPE: Any DEFAULT: {}

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	LazyIterator that only emits values once a condition has failed. All items before this are discarded.

skip_while #

skip_while(*predicates: Union[tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any) -> LazyIterator[ValueT]

Discard items while all conditions are True.

Items after this will be yielded as normal.

PARAMETER DESCRIPTION

*predicates

Predicates to invoke. These are functions that take a value and return True if it is of interest, or False otherwise. These may instead include 2-tuple objects consisting of a str attribute name (nested attributes are referred to using the . operator), and values to compare for equality. This allows you to specify conditions such as members.skip_while(("user.bot", True)).

TYPE: Union[tuple[str, Any], Callable[[ValueT], bool]] DEFAULT: ()

**attrs

Alternative to passing 2-tuples. Cannot specify nested attributes using this method.

TYPE: Any DEFAULT: {}

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	LazyIterator that only emits values once a condition has been met. All items before this are discarded.

sort `async` #

sort(*, key: Any = None, reverse: bool = False) -> Sequence[ValueT]

Collect all results, then sort the collection before returning it.

take_until #

take_until(*predicates: Union[tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any) -> LazyIterator[ValueT]

Return each item until any conditions pass or the end is reached.

PARAMETER DESCRIPTION

*predicates

Predicates to invoke. These are functions that take a value and return True if it is of interest, or False otherwise. These may instead include 2-tuple objects consisting of a str attribute name (nested attributes are referred to using the . operator), and values to compare for equality. This allows you to specify conditions such as members.take_until(("user.bot", True)).

TYPE: Union[tuple[str, Any], Callable[[ValueT], bool]] DEFAULT: ()

**attrs

Alternative to passing 2-tuples. Cannot specify nested attributes using this method.

TYPE: Any DEFAULT: {}

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	LazyIterator that only emits values until any conditions are matched.

take_while #

take_while(*predicates: Union[tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any) -> LazyIterator[ValueT]

Return each item until any conditions fail or the end is reached.

PARAMETER DESCRIPTION

*predicates

Predicates to invoke. These are functions that take a value and return True if it is of interest, or False otherwise. These may instead include 2-tuple objects consisting of a str attribute name (nested attributes are referred to using the . operator), and values to compare for equality. This allows you to specify conditions such as members.take_while(("user.bot", True)).

TYPE: Union[tuple[str, Any], Callable[[ValueT], bool]] DEFAULT: ()

**attrs

Alternative to passing 2-tuples. Cannot specify nested attributes using this method.

TYPE: Any DEFAULT: {}

RETURNS	DESCRIPTION
`LazyIterator[ValueT]`	LazyIterator that only emits values until any conditions are not matched.

NOOPLazyIterator #

NOOPLazyIterator(iterator: AsyncIterable[ValueT])

Bases: Generic[ValueT], LazyIterator[ValueT]

A lazy iterator that uses an underlying async iterator and does nothing.

hikari.iterators#

AnotherValueT module-attribute #

ValueT module-attribute #

All #

conditions instance-attribute #

AttrComparator #

BufferedLazyIterator #

FlatLazyIterator #

LazyIterator #

awaiting #

chunk #

collect async #

count async #

enumerate #

filter #

flat_map #

flatten #

for_each async #

last async #

limit #

map #

next async #

reversed #

skip #

skip_until #

skip_while #

sort async #

take_until #

take_while #

NOOPLazyIterator #

`hikari.iterators`#

AnotherValueT `module-attribute` #

ValueT `module-attribute` #

conditions `instance-attribute` #

collect `async` #

count `async` #

for_each `async` #

last `async` #

next `async` #

sort `async` #