hikari.iterators
#
Lazy iterators for data that requires repeated API calls to retrieve.
For consumers of this API, the only class you need to worry about is
hikari.iterators.LazyIterator
. Everything else is internal detail only exposed for people who
wish to extend this API further!
AnotherValueT
module-attribute
#
AnotherValueT = TypeVar('AnotherValueT')
Type-hint of the type of a value by a mapped lazy iterator.
ValueT
module-attribute
#
ValueT = TypeVar('ValueT')
Type-hint of the type of the value returned by a lazy iterator.
All
#
All(conditions: Collection[Callable[[ValueT], bool]])
Helper that wraps predicates and invokes them together.
Calling this object will pass the input item to each item, returning
True
only when all wrapped predicates return True when called
with the given item.
For example...
is equivalent to
This behaves like a lazy wrapper implementation of the all
builtin.
Note
Like the rest of the standard library, this is a short-circuiting
operation. This means that if a predicate returns False
, no
predicates after this are invoked, as the result is already known. In
this sense, they are invoked in-order.
Warning
You should not generally need to use this outside of extending the iterators API in this library!
Operators
PARAMETER | DESCRIPTION |
---|---|
conditions |
The predicates to wrap. |
conditions
instance-attribute
#
conditions: Collection[Callable[[ValueT], bool]] = conditions
Collection of the condition callbacks within this.
AttrComparator
#
AttrComparator(
attr_name: str,
expected_value: Any,
cast: Optional[Callable[[ValueT], Any]] = None,
)
A comparator that compares the result of a call with something else.
This uses the spel
module internally.
PARAMETER | DESCRIPTION |
---|---|
attr_name |
The attribute name. Can be prepended with a
TYPE:
|
expected_value |
The expected value.
TYPE:
|
cast |
Optional cast to perform on the input value when being called before comparing it to the expected value but after accessing the attribute. |
BufferedLazyIterator
#
Bases: Generic[ValueT]
, LazyIterator[ValueT]
, ABC
A special kind of lazy iterator that is used by internal components.
The purpose of this is to provide an interface to lazily deserialize
collections of payloads received from paginated API endpoints such as
GET /channels/{channel_id}/messages
, which will return a certain number
of messages at a time on a low level. This class provides the base interface
for handling lazily decoding each item in those responses and returning them
in the expected format when iterating across this object.
Implementations are expected to provide a _next_chunk
private method
which when awaited returns a lazy generator of each deserialized object
to later yield. This will be iterated across lazily by this implementation,
thus reducing the amount of work needed if only a few objects out of, say,
100, need to be deserialized.
This _next_chunk
should return None
once the end of all items
has been reached.
An example would look like the following:
async def some_http_call(i):
...
class SomeEndpointLazyIterator(BufferedLazyIterator[SomeObject]):
def __init__(self):
super().__init__()
self._i = 0
def _next_chunk(self) -> typing.Optional[typing.Generator[ValueT, None, None]]:
raw_items = await some_http_call(self._i)
self._i += 1
if not raw_items:
return None
generator = (SomeObject(raw_item) for raw_item in raw_items)
return generator
awaiting
#
awaiting(window_size: int = 10) -> LazyIterator[ValueT]
Await each item concurrently in a fixed size window.
Warning
Setting a large window size, or setting it to 0 to await everything
is a dangerous thing to do if you are making API calls. Some
endpoints will get ratelimited and cause a backup of waiting
tasks, others may begin to spam global rate limits instead
(the fetch_user
endpoint seems to be notorious for doing this).
Note
This call assumes that the iterator contains awaitable values as
input. MyPy cannot detect this nicely, so any cast is forced
internally.
If the item is not awaitable, you will receive a
TypeError
instead.
You have been warned. You cannot escape the ways of the duck type
young grasshopper.
PARAMETER | DESCRIPTION |
---|---|
window_size |
The window size of how many tasks to await at once. You can set this
to
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The new lazy iterator to return. |
chunk
#
chunk(chunk_size: int) -> LazyIterator[Sequence[ValueT]]
Return results in chunks of up to chunk_size
amount of entries.
PARAMETER | DESCRIPTION |
---|---|
chunk_size |
The limit for how many results should be returned in each chunk.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Sequence[ValueT]]
|
|
collect
async
#
collect(
collector: Callable[[Sequence[ValueT]], Collection[ValueT]]
) -> Collection[ValueT]
Collect the results into a given type and return it.
PARAMETER | DESCRIPTION |
---|---|
collector |
A function that consumes a sequence of values and returns a collection.
TYPE:
|
count
async
#
count() -> int
Count the number of results.
RETURNS | DESCRIPTION |
---|---|
int
|
Number of results found. |
enumerate
#
enumerate(*, start: int = 0) -> LazyIterator[Tuple[int, ValueT]]
Enumerate the paginated results lazily.
This behaves as an asyncio-friendly version of enumerate
which uses much less memory than collecting all the results first and
calling enumerate
across them.
PARAMETER | DESCRIPTION |
---|---|
start |
Optional int to start at. If omitted, this is
TYPE:
|
Examples:
>>> async for i, item in paginated_results.enumerate():
... print(i, item)
(0, foo)
(1, bar)
(2, baz)
(3, bork)
(4, qux)
>>> async for i, item in paginated_results.enumerate(start=9):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
(12, bork)
(13, qux)
>>> async for i, item in paginated_results.enumerate(start=9).limit(3):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Tuple[int, T]]
|
A paginated results view that asynchronously yields an increasing counter in a tuple with each result, lazily. |
filter
#
filter(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Filter the items by one or more conditions.
Each condition is treated as a predicate, being called with each item that this iterator would return when it is requested.
All conditions must evaluate to True
for the item to be
returned. If this is not met, then the item is discarded and ignored,
the next matching item will be returned instead, if there is one.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
|
flat_map
#
flat_map(
flattener: _FlattenerT[ValueT, AnotherValueT]
) -> LazyIterator[AnotherValueT]
Perform a flat mapping operation.
This will pass each item in the iterator to the given function
parameter, expecting a new typing.Iterable
or typing.AsyncIterator
to be returned as the result. This means you can map to a new
hikari.iterators.LazyIterator
, typing.AsyncIterator
, typing.Iterable
,
async generator, or generator.
Remember that typing.Iterator
implicitly provides typing.Iterable
compatibility.
This is used to provide lazy conversions, and can be used to implement reactive-like pipelines if desired.
All results are combined into one large lazy iterator and yielded lazily.
PARAMETER | DESCRIPTION |
---|---|
flattener |
A function that returns either an async iterator or iterator of new values. Could be an attribute name instead.
TYPE:
|
Examples:
The following example generates a distinct collection of all mentioned users in the given channel from the past 500 messages.
def iter_mentioned_users(message: hikari.Message) -> typing.Iterable[Snowflake]:
for match in re.findall(r"<@!?(\d+)>", message.content):
yield Snowflake(match)
mentioned_users = await (
channel
.history()
.limit(500)
.map(".content")
.flat_map(iter_mentioned_users)
.distinct()
)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
The new lazy iterator to return. |
for_each
async
#
Forward each value to a given consumer immediately.
last
async
#
last() -> ValueT
Return the last element of this iterator only.
Note
This method will consume the whole iterator if run.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The last result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no result exists. |
limit
#
limit(limit: int) -> LazyIterator[ValueT]
Limit the number of items you receive from this async iterator.
PARAMETER | DESCRIPTION |
---|---|
limit |
The number of items to get. This must be greater than zero.
TYPE:
|
Examples:
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields a maximum of the given number of items before completing. |
map
#
map(
transformation: Union[Callable[[ValueT], AnotherValueT], str]
) -> LazyIterator[AnotherValueT]
Map the values to a different value.
PARAMETER | DESCRIPTION |
---|---|
transformation |
The function to use to map the attribute. This may alternatively
be a string attribute name to replace the input value with. You
can provide nested attributes using the |
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
|
next
async
#
next() -> ValueT
Return the next element of this iterator only.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The next result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no more results exist. |
reversed
#
reversed() -> LazyIterator[ValueT]
Return a lazy iterator of the remainder of this iterator's values reversed.
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The lazy iterator of this iterator's remaining values reversed. |
skip
#
skip(number: int) -> LazyIterator[ValueT]
Drop the given number of items, then yield anything after.
PARAMETER | DESCRIPTION |
---|---|
number |
The max number of items to drop before any items are yielded.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields all items AFTER the given number of items are discarded first. |
skip_until
#
skip_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are False.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has failed. All items before this are discarded. |
skip_while
#
skip_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are True.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has been met. All items before this are discarded. |
sort
async
#
Collect all results, then sort the collection before returning it.
take_until
#
take_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions pass or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are matched. |
take_while
#
take_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions fail or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are not matched. |
FlatLazyIterator
#
Bases: Generic[ValueT]
, LazyIterator[ValueT]
A lazy iterator that has all items in-memory and ready.
This can be iterated across as a normal iterator, or as an async iterator.
awaiting
#
awaiting(window_size: int = 10) -> LazyIterator[ValueT]
Await each item concurrently in a fixed size window.
Warning
Setting a large window size, or setting it to 0 to await everything
is a dangerous thing to do if you are making API calls. Some
endpoints will get ratelimited and cause a backup of waiting
tasks, others may begin to spam global rate limits instead
(the fetch_user
endpoint seems to be notorious for doing this).
Note
This call assumes that the iterator contains awaitable values as
input. MyPy cannot detect this nicely, so any cast is forced
internally.
If the item is not awaitable, you will receive a
TypeError
instead.
You have been warned. You cannot escape the ways of the duck type
young grasshopper.
PARAMETER | DESCRIPTION |
---|---|
window_size |
The window size of how many tasks to await at once. You can set this
to
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The new lazy iterator to return. |
chunk
#
chunk(chunk_size: int) -> LazyIterator[Sequence[ValueT]]
Return results in chunks of up to chunk_size
amount of entries.
PARAMETER | DESCRIPTION |
---|---|
chunk_size |
The limit for how many results should be returned in each chunk.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Sequence[ValueT]]
|
|
collect
async
#
collect(
collector: Callable[[Sequence[ValueT]], Collection[ValueT]]
) -> Collection[ValueT]
Collect the results into a given type and return it.
PARAMETER | DESCRIPTION |
---|---|
collector |
A function that consumes a sequence of values and returns a collection.
TYPE:
|
count
async
#
count() -> int
Count the number of results.
RETURNS | DESCRIPTION |
---|---|
int
|
Number of results found. |
enumerate
#
enumerate(*, start: int = 0) -> LazyIterator[Tuple[int, ValueT]]
Enumerate the paginated results lazily.
This behaves as an asyncio-friendly version of enumerate
which uses much less memory than collecting all the results first and
calling enumerate
across them.
PARAMETER | DESCRIPTION |
---|---|
start |
Optional int to start at. If omitted, this is
TYPE:
|
Examples:
>>> async for i, item in paginated_results.enumerate():
... print(i, item)
(0, foo)
(1, bar)
(2, baz)
(3, bork)
(4, qux)
>>> async for i, item in paginated_results.enumerate(start=9):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
(12, bork)
(13, qux)
>>> async for i, item in paginated_results.enumerate(start=9).limit(3):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Tuple[int, T]]
|
A paginated results view that asynchronously yields an increasing counter in a tuple with each result, lazily. |
filter
#
filter(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Filter the items by one or more conditions.
Each condition is treated as a predicate, being called with each item that this iterator would return when it is requested.
All conditions must evaluate to True
for the item to be
returned. If this is not met, then the item is discarded and ignored,
the next matching item will be returned instead, if there is one.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
|
flat_map
#
flat_map(
flattener: _FlattenerT[ValueT, AnotherValueT]
) -> LazyIterator[AnotherValueT]
Perform a flat mapping operation.
This will pass each item in the iterator to the given function
parameter, expecting a new typing.Iterable
or typing.AsyncIterator
to be returned as the result. This means you can map to a new
hikari.iterators.LazyIterator
, typing.AsyncIterator
, typing.Iterable
,
async generator, or generator.
Remember that typing.Iterator
implicitly provides typing.Iterable
compatibility.
This is used to provide lazy conversions, and can be used to implement reactive-like pipelines if desired.
All results are combined into one large lazy iterator and yielded lazily.
PARAMETER | DESCRIPTION |
---|---|
flattener |
A function that returns either an async iterator or iterator of new values. Could be an attribute name instead.
TYPE:
|
Examples:
The following example generates a distinct collection of all mentioned users in the given channel from the past 500 messages.
def iter_mentioned_users(message: hikari.Message) -> typing.Iterable[Snowflake]:
for match in re.findall(r"<@!?(\d+)>", message.content):
yield Snowflake(match)
mentioned_users = await (
channel
.history()
.limit(500)
.map(".content")
.flat_map(iter_mentioned_users)
.distinct()
)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
The new lazy iterator to return. |
for_each
async
#
Forward each value to a given consumer immediately.
last
async
#
last() -> ValueT
Return the last element of this iterator only.
Note
This method will consume the whole iterator if run.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The last result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no result exists. |
limit
#
limit(limit: int) -> LazyIterator[ValueT]
Limit the number of items you receive from this async iterator.
PARAMETER | DESCRIPTION |
---|---|
limit |
The number of items to get. This must be greater than zero.
TYPE:
|
Examples:
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields a maximum of the given number of items before completing. |
map
#
map(
transformation: Union[Callable[[ValueT], AnotherValueT], str]
) -> LazyIterator[AnotherValueT]
Map the values to a different value.
PARAMETER | DESCRIPTION |
---|---|
transformation |
The function to use to map the attribute. This may alternatively
be a string attribute name to replace the input value with. You
can provide nested attributes using the |
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
|
next
async
#
next() -> ValueT
Return the next element of this iterator only.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The next result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no more results exist. |
reversed
#
reversed() -> LazyIterator[ValueT]
Return a lazy iterator of the remainder of this iterator's values reversed.
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The lazy iterator of this iterator's remaining values reversed. |
skip
#
skip(number: int) -> LazyIterator[ValueT]
Drop the given number of items, then yield anything after.
PARAMETER | DESCRIPTION |
---|---|
number |
The max number of items to drop before any items are yielded.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields all items AFTER the given number of items are discarded first. |
skip_until
#
skip_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are False.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has failed. All items before this are discarded. |
skip_while
#
skip_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are True.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has been met. All items before this are discarded. |
sort
async
#
Collect all results, then sort the collection before returning it.
take_until
#
take_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions pass or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are matched. |
take_while
#
take_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions fail or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are not matched. |
LazyIterator
#
A set of results that are fetched asynchronously from the API as needed.
This is a typing.AsyncIterable
and typing.AsyncIterator
with several
additional helpful methods provided for convenience.
Examples:
You can use this in multiple ways.
As an async iterable:
As an eagerly retrieved set of results (performs all API calls at once, which may be slow for large sets of data):
>>> results = await paginated_results
>>> # ... which is equivalent to this...
>>> results = [item async for item in paginated_results]
As an async iterator (not recommended):
>>> try:
... while True:
... process(await paginated_results.__anext__())
... except StopAsyncIteration:
... pass
Additionally, you can make use of some of the provided helper methods on this class to perform basic operations easily.
Iterating across the items with indexes (like enumerate
for normal
iterables):
>>> async for i, item in paginated_results.enumerate():
... print(i, item)
(0, foo)
(1, bar)
(2, baz)
Limiting the number of results you iterate across:
awaiting
#
awaiting(window_size: int = 10) -> LazyIterator[ValueT]
Await each item concurrently in a fixed size window.
Warning
Setting a large window size, or setting it to 0 to await everything
is a dangerous thing to do if you are making API calls. Some
endpoints will get ratelimited and cause a backup of waiting
tasks, others may begin to spam global rate limits instead
(the fetch_user
endpoint seems to be notorious for doing this).
Note
This call assumes that the iterator contains awaitable values as
input. MyPy cannot detect this nicely, so any cast is forced
internally.
If the item is not awaitable, you will receive a
TypeError
instead.
You have been warned. You cannot escape the ways of the duck type
young grasshopper.
PARAMETER | DESCRIPTION |
---|---|
window_size |
The window size of how many tasks to await at once. You can set this
to
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The new lazy iterator to return. |
chunk
#
chunk(chunk_size: int) -> LazyIterator[Sequence[ValueT]]
Return results in chunks of up to chunk_size
amount of entries.
PARAMETER | DESCRIPTION |
---|---|
chunk_size |
The limit for how many results should be returned in each chunk.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Sequence[ValueT]]
|
|
collect
async
#
collect(
collector: Callable[[Sequence[ValueT]], Collection[ValueT]]
) -> Collection[ValueT]
Collect the results into a given type and return it.
PARAMETER | DESCRIPTION |
---|---|
collector |
A function that consumes a sequence of values and returns a collection.
TYPE:
|
count
async
#
count() -> int
Count the number of results.
RETURNS | DESCRIPTION |
---|---|
int
|
Number of results found. |
enumerate
#
enumerate(*, start: int = 0) -> LazyIterator[Tuple[int, ValueT]]
Enumerate the paginated results lazily.
This behaves as an asyncio-friendly version of enumerate
which uses much less memory than collecting all the results first and
calling enumerate
across them.
PARAMETER | DESCRIPTION |
---|---|
start |
Optional int to start at. If omitted, this is
TYPE:
|
Examples:
>>> async for i, item in paginated_results.enumerate():
... print(i, item)
(0, foo)
(1, bar)
(2, baz)
(3, bork)
(4, qux)
>>> async for i, item in paginated_results.enumerate(start=9):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
(12, bork)
(13, qux)
>>> async for i, item in paginated_results.enumerate(start=9).limit(3):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Tuple[int, T]]
|
A paginated results view that asynchronously yields an increasing counter in a tuple with each result, lazily. |
filter
#
filter(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Filter the items by one or more conditions.
Each condition is treated as a predicate, being called with each item that this iterator would return when it is requested.
All conditions must evaluate to True
for the item to be
returned. If this is not met, then the item is discarded and ignored,
the next matching item will be returned instead, if there is one.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
|
flat_map
#
flat_map(
flattener: _FlattenerT[ValueT, AnotherValueT]
) -> LazyIterator[AnotherValueT]
Perform a flat mapping operation.
This will pass each item in the iterator to the given function
parameter, expecting a new typing.Iterable
or typing.AsyncIterator
to be returned as the result. This means you can map to a new
hikari.iterators.LazyIterator
, typing.AsyncIterator
, typing.Iterable
,
async generator, or generator.
Remember that typing.Iterator
implicitly provides typing.Iterable
compatibility.
This is used to provide lazy conversions, and can be used to implement reactive-like pipelines if desired.
All results are combined into one large lazy iterator and yielded lazily.
PARAMETER | DESCRIPTION |
---|---|
flattener |
A function that returns either an async iterator or iterator of new values. Could be an attribute name instead.
TYPE:
|
Examples:
The following example generates a distinct collection of all mentioned users in the given channel from the past 500 messages.
def iter_mentioned_users(message: hikari.Message) -> typing.Iterable[Snowflake]:
for match in re.findall(r"<@!?(\d+)>", message.content):
yield Snowflake(match)
mentioned_users = await (
channel
.history()
.limit(500)
.map(".content")
.flat_map(iter_mentioned_users)
.distinct()
)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
The new lazy iterator to return. |
for_each
async
#
Forward each value to a given consumer immediately.
last
async
#
last() -> ValueT
Return the last element of this iterator only.
Note
This method will consume the whole iterator if run.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The last result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no result exists. |
limit
#
limit(limit: int) -> LazyIterator[ValueT]
Limit the number of items you receive from this async iterator.
PARAMETER | DESCRIPTION |
---|---|
limit |
The number of items to get. This must be greater than zero.
TYPE:
|
Examples:
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields a maximum of the given number of items before completing. |
map
#
map(
transformation: Union[Callable[[ValueT], AnotherValueT], str]
) -> LazyIterator[AnotherValueT]
Map the values to a different value.
PARAMETER | DESCRIPTION |
---|---|
transformation |
The function to use to map the attribute. This may alternatively
be a string attribute name to replace the input value with. You
can provide nested attributes using the |
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
|
next
async
#
next() -> ValueT
Return the next element of this iterator only.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The next result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no more results exist. |
reversed
#
reversed() -> LazyIterator[ValueT]
Return a lazy iterator of the remainder of this iterator's values reversed.
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The lazy iterator of this iterator's remaining values reversed. |
skip
#
skip(number: int) -> LazyIterator[ValueT]
Drop the given number of items, then yield anything after.
PARAMETER | DESCRIPTION |
---|---|
number |
The max number of items to drop before any items are yielded.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields all items AFTER the given number of items are discarded first. |
skip_until
#
skip_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are False.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has failed. All items before this are discarded. |
skip_while
#
skip_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are True.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has been met. All items before this are discarded. |
sort
async
#
Collect all results, then sort the collection before returning it.
take_until
#
take_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions pass or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are matched. |
take_while
#
take_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions fail or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are not matched. |
NOOPLazyIterator
#
NOOPLazyIterator(iterator: AsyncIterable[ValueT])
Bases: Generic[ValueT]
, LazyIterator[ValueT]
A lazy iterator that uses an underlying async iterator and does nothing.
awaiting
#
awaiting(window_size: int = 10) -> LazyIterator[ValueT]
Await each item concurrently in a fixed size window.
Warning
Setting a large window size, or setting it to 0 to await everything
is a dangerous thing to do if you are making API calls. Some
endpoints will get ratelimited and cause a backup of waiting
tasks, others may begin to spam global rate limits instead
(the fetch_user
endpoint seems to be notorious for doing this).
Note
This call assumes that the iterator contains awaitable values as
input. MyPy cannot detect this nicely, so any cast is forced
internally.
If the item is not awaitable, you will receive a
TypeError
instead.
You have been warned. You cannot escape the ways of the duck type
young grasshopper.
PARAMETER | DESCRIPTION |
---|---|
window_size |
The window size of how many tasks to await at once. You can set this
to
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The new lazy iterator to return. |
chunk
#
chunk(chunk_size: int) -> LazyIterator[Sequence[ValueT]]
Return results in chunks of up to chunk_size
amount of entries.
PARAMETER | DESCRIPTION |
---|---|
chunk_size |
The limit for how many results should be returned in each chunk.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Sequence[ValueT]]
|
|
collect
async
#
collect(
collector: Callable[[Sequence[ValueT]], Collection[ValueT]]
) -> Collection[ValueT]
Collect the results into a given type and return it.
PARAMETER | DESCRIPTION |
---|---|
collector |
A function that consumes a sequence of values and returns a collection.
TYPE:
|
count
async
#
count() -> int
Count the number of results.
RETURNS | DESCRIPTION |
---|---|
int
|
Number of results found. |
enumerate
#
enumerate(*, start: int = 0) -> LazyIterator[Tuple[int, ValueT]]
Enumerate the paginated results lazily.
This behaves as an asyncio-friendly version of enumerate
which uses much less memory than collecting all the results first and
calling enumerate
across them.
PARAMETER | DESCRIPTION |
---|---|
start |
Optional int to start at. If omitted, this is
TYPE:
|
Examples:
>>> async for i, item in paginated_results.enumerate():
... print(i, item)
(0, foo)
(1, bar)
(2, baz)
(3, bork)
(4, qux)
>>> async for i, item in paginated_results.enumerate(start=9):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
(12, bork)
(13, qux)
>>> async for i, item in paginated_results.enumerate(start=9).limit(3):
... print(i, item)
(9, foo)
(10, bar)
(11, baz)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[Tuple[int, T]]
|
A paginated results view that asynchronously yields an increasing counter in a tuple with each result, lazily. |
filter
#
filter(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Filter the items by one or more conditions.
Each condition is treated as a predicate, being called with each item that this iterator would return when it is requested.
All conditions must evaluate to True
for the item to be
returned. If this is not met, then the item is discarded and ignored,
the next matching item will be returned instead, if there is one.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
|
flat_map
#
flat_map(
flattener: _FlattenerT[ValueT, AnotherValueT]
) -> LazyIterator[AnotherValueT]
Perform a flat mapping operation.
This will pass each item in the iterator to the given function
parameter, expecting a new typing.Iterable
or typing.AsyncIterator
to be returned as the result. This means you can map to a new
hikari.iterators.LazyIterator
, typing.AsyncIterator
, typing.Iterable
,
async generator, or generator.
Remember that typing.Iterator
implicitly provides typing.Iterable
compatibility.
This is used to provide lazy conversions, and can be used to implement reactive-like pipelines if desired.
All results are combined into one large lazy iterator and yielded lazily.
PARAMETER | DESCRIPTION |
---|---|
flattener |
A function that returns either an async iterator or iterator of new values. Could be an attribute name instead.
TYPE:
|
Examples:
The following example generates a distinct collection of all mentioned users in the given channel from the past 500 messages.
def iter_mentioned_users(message: hikari.Message) -> typing.Iterable[Snowflake]:
for match in re.findall(r"<@!?(\d+)>", message.content):
yield Snowflake(match)
mentioned_users = await (
channel
.history()
.limit(500)
.map(".content")
.flat_map(iter_mentioned_users)
.distinct()
)
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
The new lazy iterator to return. |
for_each
async
#
Forward each value to a given consumer immediately.
last
async
#
last() -> ValueT
Return the last element of this iterator only.
Note
This method will consume the whole iterator if run.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The last result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no result exists. |
limit
#
limit(limit: int) -> LazyIterator[ValueT]
Limit the number of items you receive from this async iterator.
PARAMETER | DESCRIPTION |
---|---|
limit |
The number of items to get. This must be greater than zero.
TYPE:
|
Examples:
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields a maximum of the given number of items before completing. |
map
#
map(
transformation: Union[Callable[[ValueT], AnotherValueT], str]
) -> LazyIterator[AnotherValueT]
Map the values to a different value.
PARAMETER | DESCRIPTION |
---|---|
transformation |
The function to use to map the attribute. This may alternatively
be a string attribute name to replace the input value with. You
can provide nested attributes using the |
RETURNS | DESCRIPTION |
---|---|
LazyIterator[AnotherValueT]
|
|
next
async
#
next() -> ValueT
Return the next element of this iterator only.
RETURNS | DESCRIPTION |
---|---|
ValueT
|
The next result. |
RAISES | DESCRIPTION |
---|---|
LookupError
|
If no more results exist. |
reversed
#
reversed() -> LazyIterator[ValueT]
Return a lazy iterator of the remainder of this iterator's values reversed.
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
The lazy iterator of this iterator's remaining values reversed. |
skip
#
skip(number: int) -> LazyIterator[ValueT]
Drop the given number of items, then yield anything after.
PARAMETER | DESCRIPTION |
---|---|
number |
The max number of items to drop before any items are yielded.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
A paginated results view that asynchronously yields all items AFTER the given number of items are discarded first. |
skip_until
#
skip_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are False.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has failed. All items before this are discarded. |
skip_while
#
skip_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Discard items while all conditions are True.
Items after this will be yielded as normal.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values once a condition has been met. All items before this are discarded. |
sort
async
#
Collect all results, then sort the collection before returning it.
take_until
#
take_until(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions pass or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are matched. |
take_while
#
take_while(
*predicates: Union[Tuple[str, Any], Callable[[ValueT], bool]], **attrs: Any
) -> LazyIterator[ValueT]
Return each item until any conditions fail or the end is reached.
PARAMETER | DESCRIPTION |
---|---|
*predicates |
Predicates to invoke. These are functions that take a value and
return
TYPE:
|
**attrs |
Alternative to passing 2-tuples. Cannot specify nested attributes using this method.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
LazyIterator[ValueT]
|
LazyIterator that only emits values until any conditions are not matched. |