Compare commits

..
285 Commits
Author SHA1 Message Date
Faris Basha TM 088f92a30c Fix dead links to moved source files and the Collections guide (#469)
The channel and debounce sources moved into Channels/ and Debounce/,
AsyncChunksOfCountAndSignalSequence.swift is now
AsyncChunksOfCountOrSignalSequence.swift, and the Collections guide
lives in the DocC catalog, so these links return 404.
2026-11-18 06:44:13 -08:00
Rick Newton-Rogers 74a48bddc6 Revert "Bump swiftlang/github-workflows to 0.0.18 (#467)" (#468)
This reverts commit ba5872bfcb.
2026-11-16 19:27:54 -05:00
Rick Newton-Rogers ba5872bfcb Bump swiftlang/github-workflows to 0.0.18 (#467)
The new workflows use macOS Golden Gate rather than macOS Tahoe by default.
2026-11-16 19:22:41 -05:00
Hokila Jan cbde9aed74 Preserve debounce upstream failure without outstanding demand (#451)
* Preserve debounce upstream failure without demand

* Add debounce upstream failure regression tests
2026-10-05 16:11:40 +02:00
Tessaro d9c26817af Fix AsyncAlgorithms 1.3 availability macro typo (#453)
The `#else` branch declared `AsyncAlgorithms_v1_3` with the macro name
`AsyncAlgorithms 1.2`, so that branch defined `AsyncAlgorithms 1.2` twice
and never defined `AsyncAlgorithms 1.3`.

Issue: https://github.com/apple/swift-async-algorithms/issues/452
2026-10-02 14:35:24 +02:00
Aniket 822cfb8331 Fix MultiProducerSingleConsumerAsyncChannel doc examples to use takeChannel() (#460)
The stored channel property of ChannelAndStream is intentionally
internal, so the documented consume channelAndSource.channel snippet
fails to compile for clients. The supported API is takeChannel(); use
it in both doc examples.

Signed-off-by: Aniket <aniketkomailkro@gmail.com>
2026-10-02 14:04:26 +02:00
Francesco Paolo Severino 13713a4ffd Fix compilation errors in AsyncStreaming (#459)
* Fix compilation errors

* Enable traits in CI
2026-09-29 13:33:55 +02:00
Tim Condon 789dcf1f3d fix for Swift Collections 1.7.0 (#456) 2026-09-23 19:56:14 +02:00
Tianhao Liu 11645b8feb [AsyncStreaming] Propagate cancellation to late readers (#449)
* [AsyncStreaming] Propagate cancellation to late readers

Distinguish cancellation from normal source completion when a reader continuation is installed after the channel has already finished.

* [AsyncStreaming] Distinguish terminal read states

Track cancellation separately from an already-consumed terminal result so normal source completion does not throw CancellationError. Precondition on repeated reads after cancellation, a final element, or a source failure, with coverage for each terminal path.

* [AsyncStreaming] Handle pending reader cancellation

Deliver cancellation exactly once when it races with a resumed reader, preserve fused final-element completion, and align terminal-read tests and documentation with the reader contract.

* [AsyncStreaming] Test cancellation through public API

Remove the internal-only state-machine test so the regression suite exercises the supported AsyncStreaming interface without @testable import.
2026-09-03 10:53:07 +02:00
da-hye0 a5195afbd5 Fix typos in Evolution and DocC docs (#446) 2026-08-09 14:23:18 +02:00
Zayan KhanandClaude Fable 5 46b3cc3a4e Fix filename typo and template placeholder in Evolution docs (#445)
Rename 0016-mutli-producer-single-consumer-channel.md to the
correctly spelled 0016-multi-...; the document's own proposal link
already uses the correct spelling and currently 404s. Fill in the
NNNN template placeholder in 0017-map-error.md's proposal link.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-23 14:17:02 +02:00
Franz Busch 3da39bbc4e [AsyncStreaming] Redesign AsyncReader.collect to take a target container (#436)
Replace `collect(upTo:body:)` with three target-based overloads: `collect(into:)`, `collect(exactlyInto:)`, and `collect(into:maximumSize:)`. Add `AsyncReaderInsufficientElementsError` for the exact-match shortfall case.
2026-06-25 10:14:54 +02:00
kSibalic c8a9349238 [AsyncStreaming] Exclude the proposal file from the target (#434)
- NNNN-async-streaming.md is intentionally kept alongside the AsyncStreaming
implementation
- Exclude it from the target in Package.swift so SwiftPM no
longer reports it as an unhandled file (#433)
2026-06-23 12:41:18 +02:00
Franz Busch e81438119c [AsyncStreaming] Skip empty buffers in AsyncReader.forEachBuffer (#435)
When a reader signals end-of-stream with an empty buffer, body was still invoked with that empty buffer. Skip the body call when the buffer is empty.
2026-06-23 10:48:22 +02:00
David SmithandFranz Busch 8ee3d2be19 "[AsyncStreaming] Introduce a MultiProducerSingleConsumerChannel and a duplex" + a build fix (#428)
* [AsyncStreaming] Introduce a `MultiProducerSingleConsumerChannel` and a duplex

* Add tests

* Fix UnstableAsyncStreaming test build

---------

Co-authored-by: Franz Busch <f.busch@apple.com>
2026-06-05 17:15:19 -07:00
Franz Busch 8e16632775 [AsyncStreaming] Add bidirectional adapters for the async writers (#424)
* [AsyncStreaming] Add support for final elements

## Motivation

Real transports deliver structured data alongside end-of-stream and need to fuse the last write with the close. HTTP trailers and gRPC status are the obvious cases. Neither can be expressed by an empty-buffer terminator, and both lose H2/H3/QUIC's DATA+END_STREAM coalescing without a fused call.

## Modification

Adds `FinalElement: ~Copyable = Void` as a primary associated type on all four protocols. `AsyncReader` delivers it via a `consuming FinalElement?` closure parameter; `CallerAsyncReader` returns it. Both writers gain a consuming `finish` carrying the last chunk and the payload in one call.

`forEachBuffer`, `collect`, and the `pipe` variants thread the payload through. `collect` becomes consuming and gains a `Void`-final overload returning just the result.

Adds an "Alternatives considered" entry covering what would break without
this: HTTP body, gRPC, and fused close on H2/H3/QUIC.

## Result

The four protocols can back HTTP body, gRPC streaming, and similar shapes without giving up the fused close. Default `Void` keeps simple conformers unchanged; `Never` marks infinite streams. Custom and `~Copyable` payloads work end to end.

* [AsyncStreaming] Add bidirectional adapters for the async writers

## Motivation

It is quite common to have one type of writer and wanting to adapt to the other type of writer.

## Modifications

This PR adds two adapters into either direction between the writers to make conversion seamless.

## Result

It is easy to go from one type of writer to the other type.
2026-06-02 08:16:42 +02:00
Franz Busch f4177dcc17 [AsyncStreaming] Add support for final elements (#422)
## Motivation

Real transports deliver structured data alongside end-of-stream and need to fuse the last write with the close. HTTP trailers and gRPC status are the obvious cases. Neither can be expressed by an empty-buffer terminator, and both lose H2/H3/QUIC's DATA+END_STREAM coalescing without a fused call.

## Modification

Adds `FinalElement: ~Copyable = Void` as a primary associated type on all four protocols. `AsyncReader` delivers it via a `consuming FinalElement?` closure parameter; `CallerAsyncReader` returns it. Both writers gain a consuming `finish` carrying the last chunk and the payload in one call.

`forEachBuffer`, `collect`, and the `pipe` variants thread the payload through. `collect` becomes consuming and gains a `Void`-final overload returning just the result.

Adds an "Alternatives considered" entry covering what would break without
this: HTTP body, gRPC, and fused close on H2/H3/QUIC.

## Result

The four protocols can back HTTP body, gRPC streaming, and similar shapes without giving up the fused close. Default `Void` keeps simple conformers unchanged; `Never` marks infinite streams. Custom and `~Copyable` payloads work end to end.
2026-06-02 08:16:01 +02:00
Aditya Singh 268087d1de Correct reductions docs: exclusive variants do not emit the initial value (#423)
The doc comments for the exclusive reductions overloads stated that the
returned sequence is "the initial value followed by the reduced elements".
That does not match the behavior: the iterator pulls a base element before
emitting anything, so the first emitted element is the result of combining
the initial value with the first base element, and the initial value is
never emitted. For an empty base sequence the result is empty rather than a
single-element sequence containing the initial value.

This is the behavior the existing tests already pin down, and changing it
would be source-breaking, so this corrects the documentation to match.
Updates all four exclusive overloads (throwing and non-throwing, value and
into:) and adds empty-base tests covering the now-documented edge case.

The inclusive reductions and the DocC Reductions guide already describe the
actual behavior and are left unchanged.
2026-05-27 15:29:38 +02:00
Franz Busch 3bd2de010e Add new pipe convenience methods for the async readers (#420)
## Motivation

Often developers want to pipe the contents of an async reader into a writer. This is a common scenario in asynchronous streaming APIs. Readers are modeling sources of data where writers are modeling sinks. As an example imagine reading from a network socket and writing the contents into a file. Importantly, the entire operation needs to uphold backpressure.

## Modifications

This PR adds 2 new `pipe` methods to each reader one for each writer type. Depending on the buffer ownership those methods will the pipe operation is copy free, requires a copy or even requires an intermediate allocation for copying. The method parameter labels are trying to express what the operation does.

## Result

It is very easy for developers now to move elements from readers into writers.
2026-05-20 09:49:59 +02:00
Guoye Zhang b2beeb25b4 Change all AsyncStreaming availability from 26.2 to SwiftStdlib 5.0 (#421)
* Change all AsyncStreaming availability from 26.2 to SwiftStdlib 5.0

* Spelling
2026-05-19 10:31:31 +02:00
Franz Busch d0b4a06d0f Rework AsyncStreaming protocols to use generic RangeReplaceableContainer buffers (#417)
Replace `InputSpan`/`OutputSpan` with generic `Buffer` associated types constrained to `RangeReplaceableContainer` across all four streaming protocols. This sidesteps the limitation that `OutputSpan` (and other `~Escapable` types) cannot be used in async contexts today, while still allowing conforming types to choose a buffer representation optimized for their use case (e.g. `UniqueArray` for heap-backed storage, or a future stack-allocated container for embedded).

Switch callee-owned protocols from `consuming` to `inout` closure semantics, rename `forEachChunk` to `forEachBuffer`, and move `forEach`/`collect` to future directions in the proposal.

Adds test coverage for `AsyncWriter`, `CallerAsyncReader`, and `EitherError`. Updates all documentation to reflect the new API surface.
2026-05-14 23:42:25 +02:00
Franz Busch 6a92272e13 Fix a few new region isolation errors in nightly-main (#419)
* Rework `AsyncStreaming` protocols to use generic `RangeReplaceableContainer` buffers

Replace `InputSpan`/`OutputSpan` with generic `Buffer` associated types constrained to `RangeReplaceableContainer` across all four streaming protocols. This sidesteps the limitation that `OutputSpan` (and other `~Escapable` types) cannot be used in async contexts today, while still allowing conforming types to choose a buffer representation optimized for their use case (e.g. `UniqueArray` for heap-backed storage, or a future stack-allocated container for embedded).

Switch callee-owned protocols from `consuming` to `inout` closure semantics, rename `forEachChunk` to `forEachBuffer`, and move `forEach`/`collect` to future directions in the proposal.

Adds test coverage for `AsyncWriter`, `CallerAsyncReader`, and `EitherError`. Updates all documentation to reflect the new API surface.

* Fix a few new region isolation errors in `nightly-main`

Region isolation got smarter and started diagnosing more potential data races. This PR addresses the new errors. Importantly I had to introduce one `@unchecked Sendable` conformance since we can't yet use `UniqueDeque`
2026-05-14 23:22:27 +02:00
Franz Busch 6783e0f2d2 Adds a new AsyncReader.forEach method (#416)
It is often needed to iterate over all elements of a reader. This PR provides two convenience `forEach` methods to make iteration easier.
2026-05-13 13:20:41 +02:00
Franz Busch d14a83b31f Adds a new AsyncReader.collect method (#415)
Often developers want to collect elements from a reader up to a certain limit. This PR provides a few convenience methods for this.
2026-05-12 23:07:29 +02:00
Franz Busch 0a5f92d159 Expose the AsyncStreaming target as a product (#414) 2026-05-12 22:20:26 +02:00
Franz Busch be2467e0c0 Add new experimental _AsyncStreaming module (#400)
This PR adds an experimental module for asynchronous streaming interfaces as a potential evolution to async sequence. These APIs are considered highly unstable and are mostly for experimentation purposes; hence, they are part of an underscored module and all APIs are hidden behind a trait. I expect significant changes to those APIs to offer more functionality such as gather scatter and more.
2026-05-11 14:58:21 +02:00
Philippe Hausler 9d349bcc32 Cleanup pass for flatMapLatest and housekeeping tasks around proposals (#403)
* rename the map error proposal to have a specific number

* rename the flatMapLatest proposal to have a specific numbering

* Update availability markers for flatmapLatest

* Cleanup formatting pass for FlatMapLatest

* Remove stray extra single quote

* Add a fallback for older versions of swift for the next method on flatMapLatest
2026-03-04 14:27:48 -08:00
5043186441 feat: Add AsyncSequence.flatMapLatest operator (#382)
* feat: Add initial naive implementation of flatMapLatest

This introduces a simplified version of the flatMapLatest operator for AsyncSequence, along with a basic unit test. Note that this implementation uses unstructured concurrency and may have race conditions regarding task cancellation.

* refactor: Implement robust flatMapLatest with generation tracking

- Replaces naive implementation with a thread-safe approach using ManagedCriticalState.
- Introduces generation tracking to prevent race conditions where cancelled inner sequences could yield stale values.
- Adds test_interleaving_race_condition to verify correctness under concurrent load.
- Ensures Swift 6 Sendable compliance.

* refactor: Implement high-performance State Machine for flatMapLatest

- Replaces AsyncThrowingStream implementation with a custom AsyncSequence, Storage, and StateMachine.
- Implements explicit state management using Lock for thread safety.
- Handles concurrency between outer and inner sequences robustly.
- Ensures correct cancellation propagation and error handling.
- Verified with test_interleaving_race_condition.

* refactor: Split flatMapLatest into separate files

- Moves AsyncFlatMapLatestSequence.swift to Sources/AsyncAlgorithms/FlatMapLatest/
- Extracts FlatMapLatestStateMachine and FlatMapLatestStorage into their own files.
- Aligns project structure with other complex operators like CombineLatest and Debounce.

* test: Add comprehensive tests for flatMapLatest

- Added test_outer_throwing to verify outer sequence error propagation
- Added test_inner_throwing to verify inner sequence error propagation
- Added test_cancellation to verify proper cancellation handling
- Added test_empty_outer to verify empty outer sequence handling
- Added test_empty_inner to verify empty inner sequence handling
- Fixed test_simple_sequence to be more robust against timing issues

* fix: Make test_outer_throwing stable

- Changed from synchronous map with throwIf to AsyncThrowingStream
- Added delay to ensure proper error propagation timing
- Fixes intermittent test failures

* refactor: Clean up verbose comments in FlatMapLatestStateMachine

- Removed LLM-style thinking comments
- Kept only essential, professional explanations
- No functional changes, all tests pass

* docs: Add comprehensive guide for flatMapLatest

- Added FlatMapLatest.md documentation guide
- Authored by Peter Friese
- Includes introduction, code samples, and use cases
- Covers search-as-you-type, location-based data, and dynamic config examples
- Compares with similar operators in ReactiveX and Combine

* docs: Add evolution proposal for flatMapLatest

- Created SAA-00nn proposal for flatMapLatest operator
- Includes motivation, detailed design, and examples
- Covers implementation strategy with state machine
- Compares with ReactiveX switchMap and Combine switchToLatest

* mapError - transforming failures (#366)

* Implement AsyncMapErrorSequence

* Update the proposal, implementation and tests with formatting, additional details and modifications for being able to push it to a review

* Foratting and preambles

* Update Copyright date for error sequence

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update copyright year for mapError tests

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update availablity and modify the mapError algorithm to be opaque

* Correct pre 6.0 build avail parameters

* Make the formatting of the algorithm accesor a bit more consistent

* Update the 5.8 package builder to define 1.2

* re-run formatter

---------

Co-authored-by: Clive <clive819@gmail.com>
Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update the flatMapLatest proposal and implementation to a pitchable state

* Fix race condition in flatMapLatest and add regression test

* chore: Remove test_output.log and add it to .gitignore

---------

Co-authored-by: Philippe Hausler <phausler@apple.com>
Co-authored-by: Clive <clive819@gmail.com>
Co-authored-by: Clive <clive819@users.noreply.github.com>
2026-03-04 13:58:48 -08:00
Franz Busch 2971dd5d9f Fix a data race error with the internal Optional.takeSending (#399)
We added a little helper to move a non-Sendable type into an optional and take it out as sending so that it is disconnected. This was actually a data race hole which got fixed by the compiler so we need to use a proper disconnected abstractions.
2026-02-11 11:02:07 -08:00
Oleksii M. 103f5e5bea Fix swift tool version (#374) 2026-01-26 09:34:26 -08:00
Kenta Kubo b8006a9de0 Add support for FreeBSD (#375)
* Add partial support for FreeBSD

This change will allow `swift build` to succeed. However, SwiftPM support is still incomplete and does not yet allow adding `.freebsd` to `.when(platforms:)`, so `swift test` cannot be executed.

* Make `swift test` work on FreeBSD
2026-01-26 09:33:54 -08:00
Alessio NossaandPhilippe Hausler 2de50036f6 Fix unbuffered share deadlock (#387)
* Introduce test_share_with_no_buffering

* Update test_share_with_no_buffering implementation

* Check limit bigger than 0 in AsyncShareSequence iterate

* Revert "Check limit bigger than 0 in AsyncShareSequence iterate"

This reverts commit 3c9e1ab0ff.

* Check limit bigger than 0 in iterate in AsyncShareSequence

* Hand correct the lint failures in TestShare.swift

* An additional hand-edit lint fix for TestShare.swift

---------

Co-authored-by: Philippe Hausler <phausler@apple.com>
2026-01-26 09:29:41 -08:00
scottmarchant ba540240d3 docs: Update comments in Locking.swift to better clarify threading scenario for WASILibc. (#393)
### Motivation:

The previous comments could mislead readers into thinking WASILibc doesn't have pthreads or locks, which isn't correct. The updated comment adds needed clarification to help readers know there multiple variations and configurations of Swift for WebAssembly, some of which provide pthreads and locks.

### Modifications:

- Updated code comments in Locking.swift

### Result:

- Better explanation of WASILibc capabilities.
2026-01-13 22:12:46 +00:00
65ed06871b mapError - transforming failures (#366)
* Implement AsyncMapErrorSequence

* Update the proposal, implementation and tests with formatting, additional details and modifications for being able to push it to a review

* Foratting and preambles

* Update Copyright date for error sequence

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update copyright year for mapError tests

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update availablity and modify the mapError algorithm to be opaque

* Correct pre 6.0 build avail parameters

* Make the formatting of the algorithm accesor a bit more consistent

* Update the 5.8 package builder to define 1.2

* re-run formatter

---------

Co-authored-by: Clive <clive819@gmail.com>
Co-authored-by: Clive <clive819@users.noreply.github.com>
2026-01-12 12:21:06 -08:00
scottmarchant d98a48b8c2 feat: Add support for compiling to wasm when pthreads aren't available (#389) 2026-01-12 10:37:45 -08:00
Franz Busch 6c050d5ef8 Make MPSCAsyncChannel source methods nonisolated(nonsending) (#380)
`nonisolated(nonsending)` is a 6.2 language feature that allows the inheritance of the callers isolation. The new `MPSCAsyncChannel.Source` send methods should adopt this to avoid unnecessary isolation hops.
2025-12-01 11:18:00 -08:00
Joseph Heck 2773d41253 This removes the build constraint at Swift Package Index from Swift 5.8 so that (#377)
symbols that are only available in later versions (for example, MultiProducerSingleConsumerAsyncChannel)
are reflected in the hosted documentation.
2025-11-19 22:22:29 +01:00
Franz BuschandJoe Heck 4686d2edd3 Add MultiProducerSingleConsumerChannel (#305)
* Add `AsyncBackpressuredStream` proposal and implementation

# Motivation

The pitch to add external backpressure support to the standard libraries `AsyncStream` got returned for revision since there are larger open questions around `AsyncSequence`. However, having external backpressure in a source asynchronous sequence is becoming more and more important.

# Modification

This PR adds a modified proposal and implementation that brings the Swift Evolution proposal over to Swift Async Algorithms.

* Update proposal and implementation

* Update proposal

* Add example project

* Formatting

* Fix Swift 6.0 build

* Future direction for ~Copyable elements

* Apply formatting

* Fix CI

* Move to 6.1 and update proposal

* Guard tests

* Minor edits to the proposal

* Fix revision order

* FIxup setOnTerminationCallback

* Address review feedback

* Rename to `MultiProducerSingleConsumerAsyncChannel`

* Allow one termination callback per source.

* Fix all sendable warnings

* Remove unbounded strategy, rename copy -> makeAdditionalSource, rename asyncSequence() -> elements(), and move enqueue and cancel to the `CallbackHandle` previously called `CallbackToken`

* Remove Example and fix docs and fix format

* fixes grammar and some word order issues

* Remove unnecessary consume

---------

Co-authored-by: Joe Heck <j_heck@apple.com>
2025-11-18 15:56:41 -08:00
Melissa Kilby 24bd5acc81 chore: restrict GitHub workflow permissions - future-proof (#373)
Signed-off-by: Melissa Kilby <mkilby@apple.com>
2025-10-27 15:09:46 -07:00
Changmin Lee 26111a6fb7 Fix broken documentation links in Interspersed guide (#371) 2025-10-06 09:28:20 -07:00
Janosch Hildebrand daf3c0d723 Fix typo in doc comment for share (#370) 2025-10-01 08:43:45 -07:00
Philippe HauslerandJamie 70c36ce9a1 Share algorithm (#357)
This is an implementation of share; an algorithm for allowing AsyncSequence types to be iterated by multiple consumers. In addition this cleans up the project to be a starting point for the 1.1 release - it fixes the continuous integration support and configurations but has the cost of only supporting 3 active releases (6.0, 6.1, and 6.2). Previous releases are only going to have legacy support via older released versions.

* Fix the remaining todo on hard cancellation vs soft cancellation

* Update Sources/AsyncAlgorithms/AsyncShareSequence.swift

Co-authored-by: Jamie <2119834+jamieQ@users.noreply.github.com>

* Update the implementation of share to handle sendability requirements, address some edge cases and update to the latest version of the discussion around buffering behaviors as well as adding some documentation and commentary

* Add some first drafts at unit tests for verifying share behaviors

* Cleanup some of the documentation and address feedback for implementation details

* Fix merge damage by restoring the changes from @jamieQ

* Fix up some availability issues w.r.t. sendable metatypes

* Add a preamble to SendableMetatypes.swift

* Add a fallback for next (for pre 6.2 swift's)

* Slight adjustments for older compiler builds and expose the sendable metatypes since those are part of documentation

* Update formatting and gate availability for older compilers

* Speculative fix for pre 6.1 compiler crashes by removing the sending keyword

* Revert "Speculative fix for pre 6.1 compiler crashes by removing the sending keyword"

This reverts commit 875fa81bc6.

* Speculative fix for pre 6.1 compiler crashes by removing the sending keyword, and correct sendable metatype shim availability

* Remove block comment for formatting check

* Take a more conservative approach and roll back to only supporting share on 6.2

* Only test share on 6.2 or newer

* roll back to older sendable metatype constraints

* yet another stab at 5.10 visionOS guards

* Formatting pass

* Disable 5.10 builds and leave those to older releases

* Ignore swift-version files to avoid local swiftly toolchain selection from being required by others

* Renumber the proposals and copy share into a guide

* Add a link from the primary doc interface to share

---------

Co-authored-by: Jamie <2119834+jamieQ@users.noreply.github.com>
2025-09-10 10:00:31 -07:00
Philippe Hausler ca38935fc0 Clear up CI issues and testing failures (#365)
* Conditionalize the testing targets for non-windows targets to avoid the TaskDriver requirements
* Some driveby formatting fixes and remove the freebsd target since that is not supported in swiftpm yet
* Rework validation import conditions to dependency conditionals
* Exclude 5.9 workflows and enable macOS testing scheme for CI
* Additional ifdefing out non windows targets for sequence diagram testing and more formatting fixes
2025-09-08 16:23:41 -07:00
Philippe Hausler 18e2398481 Update the project to address concurrency failures for 6.2 build modes. (#362)
* Correct build failures while updating to 6.2 swift build modes

* Add a package definition for 5.8 builds

* Add some brief explanation for the alteration of the Element in the buffer to an UnsafeTransfer

* Fix broken merge for wasi threads
2025-09-08 09:38:39 -07:00
Max Desiatov c537393339 Add enable_wasm_sdk_build: true to pull_request.yml (#363)
`swift-async-algorithms` should be built for Wasm as one of the officially supported platforms on CI to prevent possible future regressions.
2025-09-02 16:10:59 +01:00
Philippe Hausler 6dcb3acddd Simplify the version definitions and add a new definition for 1.1 (#359) 2025-08-11 14:51:05 -07:00
Franz Busch 3997ce3255 Remove the DocC plugin as a dependency (#354)
When DocC was introduced the DocC plugin was useful for building documentation locally. Nowadays both Xcode and VSCode have built-in support to generate this without the need for the plugin.

This PR removes the direct dependency on the plugin.
2025-07-02 08:09:32 -07:00
George Barnett 042e1c4d9d Move platform requirements to availability annotations (#348)
* Move platform requirements to availability annotations

Adding or raising the deployment platforms in the package manifest is a
SemVer major breaking change as consumers must also add or raise their
deployment platforms. This is a known limitation of SwiftPM.

Unforunately this means that it's very difficult for non-leaf packages
to adopt packages which declare their platforms in the manifest. Doing
so puts the brakes on adoption and ecosystem growth. For 'core' packages
like this one availability constraints should be expressed on
declarations rather than in the manifest.

This patch adds equivalent availability annotations to declarations
across the package and removes platforms from the package manifest.

* Address naming feedback
2025-04-22 08:07:37 +01:00
Mishal Shah 57b0814edf [CI] Add support for GitHub Actions (#344) 2025-03-29 09:49:13 +01:00
Franz Busch 9fcf11df15 Fix doc paste mistake 2025-03-28 21:59:41 +01:00
Franz Busch 08bb1bb014 Fix yaml check and remove docker compose 2025-03-28 21:59:41 +01:00
Franz Busch 5878737018 Language check 2025-03-28 21:59:41 +01:00
Franz Busch 552856a85c License headers 2025-03-28 21:59:41 +01:00
Franz Busch ba30f2051b Documentation 2025-03-28 21:59:41 +01:00
Franz Busch d5b49aab17 Apply formatting 2025-03-28 21:59:41 +01:00
Franz Busch ba33a225e9 Format rules 2025-03-28 21:59:41 +01:00
orobio 4c3ea81f81 Fix memory leak in Lock (#331) 2024-10-09 10:47:15 +02:00
Mason Kim 5c8bd186f4 Fix a few typos (#329)
* [DOCS] fix typo: precicely → precisely

* [DOCS] fix typo: ever → every

* [DOCS] fix typo: AsyncBufferSequence, relative
2024-09-02 14:15:02 +02:00
finagolfin 2503842d68 Use Bionic module from new Android overlay in Swift 6 instead (#326)
The new module and overlay were merged into Swift 6 in swiftlang/swift#74758.
2024-08-07 11:18:51 +01:00
Adam Fowler e83857ca55 Add Musl import, error if unrecognised platform (#325)
* Add Musl import, error if unrecognised platform

* Add #else #error for all platform checks
2024-08-06 12:26:55 +02:00
Yuta Saito 6ae9a051f7 Add support for SWIFTCI_USE_LOCAL_DEPS convention (#311)
To use this package in utils/build-script pipeline
2024-04-10 14:41:45 -07:00
Franz Busch 46b4464735 Fix some strict concurrency warnings (#310)
# Motivation

There were a few new strict concurrency warnings that this PR fixes.
2024-04-04 20:46:16 +01:00
Eric Horacek d162617838 Depend on specific products from swift-collections (#307)
* Only depend on OrderedCollections from swift-collections

Before this change, once consumers update to SwiftCollections 1.1.0 they'll pull in all of the new collection dependencies (HashTree, BitCollections, etc.). In our case, this increased our binary size by ~1MB. To fix this, we switch to only depending on what we need.

* Add DequeModule dependency
2024-02-28 12:47:12 +00:00
leogdion 14729ea7f9 Update README.md (#306)
Missing Version Number
2024-01-24 11:28:26 +00:00
Franz Busch da4e36f865 Fix potential deadlocks when resuming a continuation while holding a lock (#303)
* Fix potential deadlocks in zip

* Fix debounce

* Fix combineLatest

* Fix Channel

* Fix buffer
2023-11-17 16:20:14 +00:00
Artem Gavrilik 5bbdcc1d37 Fix typo in BoundedBufferStateMachine (#301) 2023-10-02 17:42:24 +01:00
Franz Busch cb417003f9 Make throttle underscored (#296)
# Motivation
During the discussion in https://github.com/apple/swift-async-algorithms/issues/248 it became clear that the semantics of `throttle` are not 100% figured out yet.

# Modification
This PR is making `throttle` an underscored API for the upcoming 1.0.0 release. This gives us more time to investigate what exact semantics we want to have.
2023-09-27 08:35:58 -07:00
Franz Busch 8cfdf03b51 Fix 5.7/5.8 build errors (#298)
# Motivation

Currently, this repo fails to build on Swift 5.7 and 5.8 since we were using clock APIs that were only available on 5.9.
2023-09-27 08:26:01 -07:00
Franz Busch 6360ca0344 Remove @_implementationOnly usage (#294)
# Motivation
We added `@_implementationOnly` a little while ago to hide the dependency on the `DequeModule`. However, with recent compilers this produces a warning since `@_implementationOnly` is only intended to be used in resilient libraries.

# Modification
This PR removes the usage of `@_implementationOnly`
2023-09-27 08:19:00 -07:00
Franz Busch 0a3866daec Require 5.8 for SPI 2023-09-23 18:16:32 +02:00
Franz Busch 260e198221 Add support for the Swift package index (#297)
# Motivation
We had some scripts around to generate docs for this package but all of our other packages are now providing documentation on the Swift package index.

# Modification
This removes the old script to generate documentation and adds an `.spi.yml` file. Furthermore, I removed the conditional dependency on the docc plugin which we are unconditionally depending on in most of our other packages like NIO.
2023-09-21 08:20:02 -07:00
Franz Busch f56556930f Remove majority of @unchecked Sendable usages (#295)
# Motivation
`@unchecked Sendable` is a great way to make a type `Sendable` while it is not really `Sendable` this is rarely useful and we should rather use conditional `@unchecked Sendable` annotations such as the one on `ManagedCriticalState`

# Modification
This PR removes all `@unchecked Sendable` in the main algorithm target except the one on `Merge` since we are doing manual locking there.

# Result
No more `@unchecked Sendable` usage.
2023-09-21 08:19:00 -07:00
Philippe Hausler 220f86f792 Ensure the last element of reduction in the throttle is emitted and use appropriate delay (#292) 2023-09-21 08:18:20 -07:00
Philippe Hausler c889832c64 Add a proposal for AsyncChannel (#216) 2023-09-21 11:09:43 +02:00
Franz Busch 6dfbfd5e49 Remove background correction in AsyncTimerSequence (#289)
# Motivation
Currently, the `AsyncTimerSequence` is trying to correct for when an application becomes suspended and the timer might fire multiple times once the application gets foregrounded again.

However, this is already handled by the `Clock` types themselves. The `SuspendingClock` is correcting for suspension of the app whereas the `ContinuousClock` is not.

Additionally, this was not only hit by background an application but by just calling `Task.sleep` in the for-await loop that is consuming the sequence.

# Modification
This removes the part of the code in `AsyncTimerSequence` which corrected for suspension of the application.
2023-09-21 11:07:08 +02:00
Franz Busch 281e27c5d0 Fix chunks(countOf: 1) (#293) 2023-09-19 17:23:38 +02:00
Philippe Hausler 4a1fb99f00 Rework availability for executor to avoid warnings (#288) 2023-08-22 16:09:28 -07:00
Franz Busch b0ec4694d2 Remove the validation packages from public products (#287)
# Motivation
The current validation testing library has a bunch of warnings and we haven't put it through a proper API review yet. Since we are preparing for a 1.0.0, I think it is best if we remove the products for the validation targets for now and take some time to make sure the interfaces are taken through an API review.

# Modification
Remove products for the validation targets.
2023-08-18 11:19:58 -07:00
Philippe Hausler e639e5c889 Ensure tests work in deployments that host as swift 5.7 (#285) 2023-08-15 15:40:57 -07:00
Franz Busch 7ac31f979f Fix more Sendable warnings and flaky test (#281)
# Motivation
We still had some `Sendable` warnings left under strict Concurrency checking.

# Modification
This PR fixes a bunch of `Sendable` warnings but we still have some left in the validation tests. Additionally, I fixed a flaky test.
2023-08-01 16:09:34 +01:00
Franz Busch f5d5fb6483 Mark all iterators as non-Sendable (#280) 2023-07-27 22:16:37 +01:00
Freya Alminde df46b4c235 Fix typo in Guides/Chunked.md (#277) 2023-07-24 16:42:27 +01:00
Fabian Fett 07a0c1ee08 Remove copy and paste within MergeStorage (#275) 2023-07-11 14:18:10 +01:00
0xpablo adb12bfccc Add WebAssembly support (#273) 2023-06-23 15:54:35 -07:00
Franz Busch 2ace7010e9 Add docker files for CI (#270) 2023-06-23 14:21:48 -07:00
Philippe Hausler a42b72ad84 Audit pass on Sendable conformances and requirements (#272)
* Audit pass on Sendable conformances and requirements

* A slightly cleaner mark for Sendability of OrderedSet

* Remove flags for strict mode
2023-06-23 14:21:34 -07:00
Philippe Hausler 156bb4bf49 Reduce the overall warnings from tests (#274)
* Migrate from semaphore style testing waits to async test fulfillment

* Reduce warnings for missing hashable and comparable conformances per availability

* Reduce warnings for executor (still one known warning)
2023-06-23 12:22:23 -07:00
Franz Busch 936a68d012 [AsyncInterspersedSequence] Integrate review feedback (#267)
* Integrate review feedback

This integrates all of the feedback from the review thread. Here is a quick summary:

- Change the trailing separator behaviour. We are no longer returning a separator before we are forwarding the error
- Add a synchronous and asynchronous closure based `interspersed` method.
- Support interspersing every n elements

* Add AsyncThrowingInterspersedSequence

* Update examples
2023-06-23 15:47:25 +02:00
Philippe Hausler 92747901e0 Audit pass on inline and initialization (#271) 2023-06-23 15:40:09 +02:00
Kazumasa Shimomura ac9309b2ac Rename downStream to downstream (#263) 2023-06-22 12:54:24 -07:00
Franz Busch b3394663fd Add Interspersed proposal (#259)
* Clean up old evolution proposals

# Motivation

Our old proposals had a couple of stale links and were not all aligned. This cleans them up.

* Add Interspersed proposal

# Motivation
Add a new evolution proposal for the interspersed algorithm.
2023-05-10 15:18:41 +01:00
Franz Busch 0c00af2372 Clean up old evolution proposals (#258)
# Motivation

Our old proposals had a couple of stale links and were not all aligned. This cleans them up.
2023-05-10 13:10:31 +01:00
elmetal 9d83fff5b8 [Channel] Fix Source links in documentation (#264) 2023-05-10 13:05:56 +01:00
Max Desiatov e6b21d2a6d Package.swift: make package name consistent (#255)
Rest of SwiftPM packages provided by Apple follow a dash-case naming scheme with `swift-` prefix.
2023-03-09 10:40:27 +00:00
Franz Busch 7e45005473 Switch from group.waitForAll() to group.next() (#254)
# Motivation
Swift 5.8 is including a change to how `group.waitForAll()` is working. It now properly waits for all tasks to finish even if one of the tasks throws. We have used `group.waitForAll()` in multiple places and need to change this code accordingly.

# Modification
Switch code from `group.waitForAll()` to `group.next()`.

# Result
This fixes a few stuck tests that we have seen when running against development snapshots.
2023-02-28 08:35:03 +00:00
Thibault Wittemberg ffa6ee9f0b evolution: add buffer (#252) 2023-02-15 22:32:12 +00:00
Thibault Wittemberg 9cfed92b02 buffer: implement with custom storages (#239) 2023-02-10 11:27:06 -08:00
Franz Busch ca2ac60495 Tolerate cancelling before we create our task (#250)
# Motivation
We switched the point in time when we create tasks to the first call to `next()`. This had the side effect that we can now be cancelled before we even created our task. Which by itself is fine but we didn't handle it in the state machines.

# Modification
Adapt the state machines of `debounce`, `zip`, `merge` and `combineLatest` to handle cancellation before `next()`.

# Result
We shouldn't run into these preconditions anymore.
2023-02-06 17:05:07 +00:00
ga eun kim bf0ec24bde fix iterator variable name on zip (#251) 2023-02-01 15:20:40 +00:00
Thibault Wittemberg 0ebc8058fc channel: re-implement with a state machine (#235) 2023-01-12 09:40:37 -08:00
Philippe Hausler a07a3f41b5 Rename AsyncLazySequence to AsyncSyncSequence per review feedback (#246)
* Rename AsyncLazySequence to AsyncSyncSequence per review feedback

* Rename and number the async proposal
2023-01-10 15:36:42 -08:00
Philippe Hausler e7b9b6039d Extract the result to test the preconditions seperate from the function calls for locking (#244) 2023-01-03 15:24:40 -08:00
Dan AppelandDan Appel 3f12423c47 Add precondition guards to pthread_mutex calls (#243)
While in the current configuration lock/unlock shouldn't ever fail,
it seems polite to guard against basic misuse in debug builds.

Co-authored-by: Dan Appel <dan_appel@apple.com>
2023-01-03 15:15:34 -08:00
Philippe Hausler aed5422380 AsyncLazySequence proposal (#189) 2023-01-03 09:31:58 -08:00
Philippe Hausler b6dff00602 Transform the reductions and remove duplicates guides into a proposal (#233) 2023-01-03 09:31:36 -08:00
Philippe Hausler 4ba079747d Transform the chunk and timer guides into a proposal (#231) 2023-01-03 09:31:15 -08:00
Philippe Hausler 7586d2060f Convert the guides for debounce and throttle into a proposal (#194)
* Convert the guides for debounce and throttle into a proposal

* Adjust Sendability of the iterator requirements
2023-01-03 09:30:31 -08:00
Hayashi Tatsuya fb15375758 Fix Source links in documentations (#234) 2022-12-16 10:09:34 -08:00
Max Desiatov 5656338b5d Remove print from implementation of some algorithms (#240) 2022-11-28 08:03:04 -05:00
Philippe Hausler 9abd029409 Ensure all imports of other modules are implementation only (#232) 2022-11-16 10:54:45 -08:00
Thibault Wittemberg dd97b564ae tests: harmonize debounce availability (#223) 2022-11-16 10:19:45 -08:00
Philippe Hausler 0174f58455 Cleanup Sendable conformances and @preconcurrency imports to build as strict concurrency clean (#230) 2022-11-16 10:15:26 -08:00
Franz Busch ae26222e86 Rework combineLatest to remove the Sendable constraint on the base iterator (#228)
# Motivation
This follows the recent rework of the `merge` and `zip` algorithms where we dropped the `Sendable` constraint on the `BaseX.AsyncIterator`. This PR now removes the same constraint on `combineLatest`.

# Modification
This PR overhauls the implementation of `combineLatest` with the goal of changing these things:
- Drop the `Sendable` constraint of the base `AsyncIterator`s
- Only create a single `Task`

# Result
`combineLatest` now doesn't require the base `AsyncIterator` to be `Sendable`
2022-11-16 08:49:50 -08:00
Franz Busch ed0b086089 Remove Sendable from AsyncBufferedByteIterator (#220)
# Motivation
I raise on the pitch of `AsyncBufferedByteIterator` on the forums that I think the iterator must not be `Sendable`. The reasoning for this is twofold. First, an iterator is the connection of a consumer to the `AsyncSequence`; therefore, iterators should not be shared since it breaks that assumption. Secondly, the implementation of the `AsyncBufferedByteIterator` can be more straight forward since we don't have to check for unique ownership of the storage.

# Modification
Remove the `Sendable` conformances from  `AsyncBufferedByteIterator`.
2022-10-20 11:49:54 -07:00
Franz Busch 8f5e8cee42 Merge the zip2 and zip3 implementations (#221)
# Motivation
We recently merged https://github.com/apple/swift-async-algorithms/pull/201 which removed the `Sendable` constraint of the `AsyncIterator` from `zip`. There were some left over comments that we wanted to fix in a follow up. Furthermore, we also can merge the implementations of `zip2` and `zip3` relatively easily. The merging also aids us in understanding what we need from the variadic generics proposals and allows us to give feedback since `zip` is non-trivial to implement with variadic generics.
Lastly, the merged state machine will be a good base for the overhaul of `combineLatest`.

# Modification
This PR merges the state machines from `zip2` and `zip3` into a single one. Furthermore, it addresses some of the open feedback from the last PR.

# Result
We now have a single state machine which is a good foundation for our changes to `combineLatest`.
2022-10-20 11:49:13 -07:00
Franz Busch 6c6ed33514 Remove Sendability from all Iterators (#219)
# Motivation
We have been discussing this a lot in the other PRs, but in general said that an `AsyncIterator` is the connection between the consuming task and the `AsyncSequence`. If an `AsyncIterator` would be `Sendable` then we don't have that relationship anymore. Therefore, no iterator of an `AsyncIterator` should be `Sendable`.

# Modification
Remove all `Sendable` conformances of the iterators even the conditional ones. We can always add them back later.
2022-10-20 09:34:24 -07:00
Philippe Hausler 2cfb16691a Task.select is no longer used and can potentially impact performance negatively (#217) 2022-10-20 09:34:05 -07:00
Philippe Hausler cc0621eb1b Renumber bytes's proposal 2022-10-18 13:14:44 -07:00
Philippe Hausler f32328ac6f Convert the bytes iterator guide to a proposal (#187) 2022-10-18 13:13:45 -07:00
Philippe Hausler 3890885e80 Renumber chain's proposal 2022-10-18 13:06:34 -07:00
Philippe Hausler cf70e78632 Convert the chain guide to a proposal (#188) 2022-10-14 13:30:17 -07:00
Franz Busch be681c12e3 Remove Sendable constraint for AsyncChain2Sequence (#213)
# Motivation
We want to remove any `Sendable` constraints on iterators since they ought to not be passed between `Task`s.

# Modification
Explicitly mark the `AsyncChain2Sequence` to be not `Sendable`.

# Result
Nobody can make this `Sendable`.
2022-10-14 08:41:25 -07:00
Franz Busch c21ee2d706 Remove Sendable constraint for AsyncAdjacentPairsSequence (#212)
# Motivation
We want to remove any `Sendable` constraints on iterators since they ought to not be passed between `Task`s.

# Modification
Explicitly mark the `AsyncAdjacentPairsSequence` to be not `Sendable`.

# Result
Nobody can make this `Sendable`.
2022-10-14 08:39:53 -07:00
Philippe Hausler aedcb785a4 Simplify chunks algorithm by re-using merge and composition (#209) 2022-10-12 09:35:07 -07:00
Philippe Hausler 9d427ce159 Relax Sendable conditional conformance for AsyncJoinedBySeparatorSequence (#210) 2022-10-12 09:25:11 -07:00
Doug Russell 081ca79161 Deprecation Housekeeping (#206) (#207)
`withTaskCancellationHandler<T>(_:, operation:)` -> `withTaskCancellationHandler<T>(operation:onCancel:)`
2022-10-11 19:54:17 +01:00
Thibault Wittemberg 76b27aebef zip: remove Sendable constraint + limit task creation (#201) 2022-10-10 08:58:22 -07:00
Doug Russell 1bd073999c Deprecation Housekeeping (#206)
`withTaskCancellationHandler<T>(_:, operation:)` -> `withTaskCancellationHandler<T>(operation:onCancel:)`
2022-10-10 08:35:35 -07:00
Franz Busch 314b10c634 Remove AsyncIterator: Sendable requirement from merge (#185)
* Remove `AsyncIterator: Sendable` requirement from merge

# Motivation
Currently a lot of the operator implementations in here that consume other `AsyncSequence`s require the `AsyncIterator` to be `Sendable`. This is mostly due to the fact that we are calling `makeAsyncIterator` on the upstream `AsyncSequence` and then pass that iterator around to various newly spawned `Task`s. This has two downsides:
1. It only allows users to use operators like `merge` if their `AsyncSequence.AsyncIterator` is `Sendable`
2. In merge we are creating new `Task`s for every new demand. Creating `Task`s is not cheap.

My main goal of this PR was to remove the `Sendable` constraint from `merge`.

# Modification
This PR overhauls the complete inner workings of the `AsyncMerge2Sequence`. It does a couple of things:
1. The main change is that instead of creating new `Task`s for every demand, we are creating one `Task` when the `AsyncIterator` is created. This task has as child task for every upstream sequence.
2. When calling `next` we are signalling the child tasks to demand from the upstream
3. A new state machine that is synchronizing the various concurrent operations that can happen
4. Handling cancellation since we are creating a bunch of continuations.

# Result
In the end, this PR swaps the implementation of `AsyncMerge2Sequence` and drops the `Sendable` constraint and passes all tests. Furthermore, on my local performance testing I saw up 50% speed increase in throughput.

# Open points
1. I need to make this sequence re-throwing but before going down that rabbit whole I wanna get buy-in on the implementation.
2. We should discuss and document if `merge` and other operators are hot or cold, i.e. if they only request if they got downstream demand
3. I need to switch `AsyncMerge3Sequence` over to the same iplementation

* Split logic into multiple files, adapt merge3 and incorporate PR feedback

* Add more tests and fix upstream throw behaviour

* Remove internal class from the sequence

* Setup task after first next

* Remove unnecessary tests
2022-10-10 08:34:50 -07:00
Jeff Lewis 4feeb83d13 make initializers '@usableFromInline' instead of '@inlinable' to support building for library evolution (#205) 2022-10-05 09:39:14 -07:00
Philippe Hausler af9ce91db4 Simplify resumptions to avoid needing a resumption type and mark continuations as resumed directly in critical regions (#203) 2022-10-04 14:28:14 -07:00
Franz Busch f05e450f0b Remove AsyncIterator: Sendable requirement from debounce (#198)
* Remove `AsyncIterator: Sendable` requirement from debounce

# Motivation
The current implementation of `AsyncDebounceSequence` requires the base `AsyncIterator` to be `Sendable`. This is causing two problems:

1. It only allows users to use `debounce` if their `AsyncSequence.AsyncIterator` is `Sendable`
2. In `debounce` we are creating a lot of new `Task`s and reating `Task`s is not cheap.

My main goal of this PR was to remove the `Sendable` constraint from `debounce`.

# Modification
This PR overhauls the implementation of `debounce` and aligns it with the implementation of the open `merge` PR https://github.com/apple/swift-async-algorithms/pull/185 . The most important changes are this:
- I removed the `Sendable` requirement from the base sequences `AsyncIterator`.
- Instead of creating new Tasks for the sleep and for the upstream consumption. I am now creating one Task and manipulate it by signalling continuations
- I am not cancelling the sleep. Instead I am recalculating the time left to sleep when a sleep finishes.

# Result
In the end, this PR swaps the implementation of `AsyncDebounceSequence` and drops the `Sendable` constraint and passes all tests. Furthermore, on my local performance testing I saw up 150% speed increase in throughput.

* Fix https://github.com/apple/swift-async-algorithms/issues/174

* Code review

* Remove lock methods

* Cleanup some unused code

* Setup task after first call to next
2022-09-30 10:12:11 -07:00
Thibault Wittemberg 2b5fbf3c91 [Channel] improve send cancellation (#184)
* asyncChannel: introduce ChannelToken to model Pending and Awaiting

* asyncChannel: harmonize send and next cancellation

* asyncChannel: add .finished as an emission state

* asyncThrowingChannel: harmonize send and next cancellation

* channel: update documentation for cancellation
2022-09-14 15:24:29 -07:00
Thibault Wittemberg 68c8dc285b asyncChainSequence: harmonize unit tests nomenclature (#191) 2022-08-30 10:48:20 -07:00
Thibault Wittemberg cba18c3f88 asyncLazySequence: harmonize unit tests nomenclature (#190) 2022-08-30 10:47:50 -07:00
Philippe Hausler c0fdfdb8aa Add an introduction and formatting for the combineLatest algorithm into a proposal. (#177)
* Add an introduction and formatting for the `combineLatest` algorithm into a proposal

* Update the proposal # to 0006
2022-08-04 09:44:07 -07:00
Philippe Hausler e3a18a0353 Modify the adjacent pairs guide into a proposal ready for review (#178)
* Modify the adjacent pairs guide into a proposal ready for review

* Update the proposal # to 0005
2022-08-04 09:43:14 -07:00
Franz BuschandPhilippe Hausler b92ba058d8 Proposal for joined and joined(separator:) (#180)
* Proposal for `joined` and `joined(separator:)`

* Small fixes

* Update proposal # to 0004

Co-authored-by: Philippe Hausler <phausler@apple.com>
2022-08-04 09:42:14 -07:00
Thibault Wittemberg f44d02eb26 Improve adjacent pairs unit tests (#179)
* adjacentPairs: improve unit tests

* adjacentPairs: conform indentation to project standards
2022-07-18 09:28:55 -07:00
Philippe Hausler 57072e617a Correct proposal link for compacted 2022-07-13 10:19:35 -07:00
Philippe Hausler 45aa5b950c Rename compacted proposal to 0003 2022-07-13 10:15:10 -07:00
Philippe Hausler 6ff1f1b411 Compacted proposal (#162) 2022-07-13 09:02:40 -07:00
shiz 2a211e18be Fix typo in Validation.md (#172)
* Add escaping characters before |

* Add more escaping characters before |
2022-06-27 14:23:48 -07:00
Thibault Wittemberg 9baff2ad3d [AsyncThrowingChannel] make the fail terminal event non async (#164)
* asyncThrowingChannel: make fail set a terminal state

* asyncChannel: align naming on asyncThrowingChannel

* asyncChannel: remove a test base on Task.sleep

* guides: update Channel with non async fail(_:)
2022-06-27 09:04:37 -07:00
Thibault Wittemberg 6df7fcdd10 merge: improve unit tests (#161) 2022-06-27 09:04:15 -07:00
Ole Begemann 8a355f015f Fix a few typos (#173) 2022-06-27 09:03:09 -07:00
shiz 816cf644af Add missing back quote (#171) 2022-06-27 09:00:33 -07:00
Philippe Hausler cca423ff03 Remove the disable of availability and mark methods as available to clock/instant/duration (#165)
* Remove the disable of availability and mark methods as available to clock/instant/duration

* Rework availability to be more permissive for tests

* Use hand coded values instead of defines (only available via Darwin versions of Dispatch)

* Adjust additional test availability

* Correct debounce test availability guard

* Update build instructions to indicate Xcode requirements

* Ensure the clock property of the validation diagram is only available for targets that can support it
2022-06-16 13:06:20 -07:00
Thibault Wittemberg 9a3e94d440 compacted: rename unit tests (#166) 2022-06-15 23:28:45 -07:00
Philippe Hausler 07b8d74cc7 [SAA-0002] Merge (#156) 2022-06-13 11:04:03 -07:00
Philippe Hausler 434591a571 Update availability to be in sync with Clock/Instant/Duration (#159) 2022-06-13 10:18:45 -07:00
Max Desiatov 6e9a5757bc Fix typo in BufferedBytes.md (#158)
`BufferedAsyncByteIterator` -> `AsyncBufferedByteIterator`
2022-05-30 19:29:53 -07:00
Thibault Wittemberg a973b06d06 Fix async channel finishes all (#152)
* asyncChannel: enforce termination for all when finished

* asyncThrowingChannel: enforce termination for all when finished

* asyncChannel: improve unit tests readability

* asyncChannel: adapt the Guide to the new finish behaviour

* asyncChannel: adapt the Guide to the new finish behaviour
2022-05-04 13:07:33 -07:00
Kevin Tan 6b59cfb8d0 Minor documentation improvements (#153) 2022-05-03 21:17:53 -07:00
Philippe Hausler d185b2068b [SAA-0001] Zip (#149) 2022-04-27 16:08:26 -07:00
Thibault Wittemberg bbbe08d331 zip: improve unit tests readability (#151) 2022-04-25 17:50:09 -07:00
Thibault Wittemberg 6d5d6319f4 asyncThrowingChannel: fixes potential crashes when no more awaiting (#150)
This commit fixes crashes where the state is "awaiting" with no more awaiting clients.
The state is now set to .idle to reflect the reality.
2022-04-21 11:10:14 -07:00
Thibault Wittemberg 047ab6abef asyncChannel: fixes potential crashes when no more awaiting (#143)
This commit fixes crashes where the state is "awaiting" with no more awaiting clients.
The state is now set to .idle to reflect the reality.
2022-04-15 10:05:40 -07:00
Tiago Lopes d2164b9ef6 Fix redundant where clauses in AsyncBufferSequence extension (#141) 2022-04-12 07:23:05 -07:00
Tiago Lopes b3d4ffb94c Fix broken links to guides in README.md (#146) 2022-04-11 16:53:54 -07:00
Tiago Lopes 5b78eaf423 Fix some typos in Chain.md (#144) 2022-04-11 15:06:39 -07:00
Kyle a3576e669b Add docc support and publish documentation to GitHub pages (#137)
* Move Guides into docc base location

* Add Github gh-page support

* Conditionally add docc-plugin dependency

* Temp skip AsyncSequenceValidation target
2022-04-11 14:25:33 -07:00
Kristoffer Johansson 94a7ecec28 Moves XCTestCase.validate into its own module (#135)
* Move XCTestCase.validate into it’s own module

* Remove unnecessary import statement
2022-04-08 14:15:55 -07:00
Philippe Hausler 6a8aec2a1f #131 Cancel sends as if they were terminal events on task cancellation (#132) 2022-04-08 10:44:30 -07:00
Philippe Hausler 9d03d90a13 #120 Ensure concurrent access to AsyncBufferedByteIterator does not crash from dangling pointers (#121) 2022-04-08 10:44:06 -07:00
k-kohey c174e4603e fix merge example (#140) 2022-04-08 08:47:25 -07:00
Yuka Ezura b6755e6313 Fix typo (#138) 2022-04-07 12:18:56 -07:00
Philippe Hausler 18749897a8 #113 Define the first element of throttled sequences by not exceeding the rate (and fix a expectation misalignment of events) (#133)
* #113 Define the first element of throttled sequences by not exceeding the rate (and fix a expectation misalignment of events)

* Update guide for throttle referencing the first element and duration measurement
2022-04-01 11:40:44 -07:00
Guillaume Lessard 6971c8fc67 Update README.md (#134)
Insert a forgotten word.
2022-04-01 09:24:42 -07:00
Tiago Lopes f1581d0973 Fix a grammatical sentence in Collections.md (#127) 2022-03-30 10:20:47 -07:00
Tiago Lopes 442d20e8c8 Fix TestRangeReplaceableCollection.swift file name (#128) 2022-03-30 10:20:19 -07:00
Laszlo Teveli b8a8d5d2ae Rename AdjacentPairsSequence.swift (#126) 2022-03-29 12:15:04 -07:00
Laszlo Teveli b136d48e7f Adjacent Pairs (#111) 2022-03-29 10:01:55 -07:00
Reza Shirazian 4b5a3ef050 measureSequenceThroughput-duration-clean-up (#125)
duration and start property in measureSequenceThroughput is vestigial from early development. This PR cleans that up.
2022-03-29 09:10:51 -07:00
Tiago Lopes 9ce65b089c Fix the README link to the Swift Forums (#123) 2022-03-28 17:54:26 -07:00
Reza Shirazian fac138c517 #45 - Two sided performance test for merge (#114)
This PR extends `measureSequenceThroughput` to take two outputs so functions like merge/ zip etc can be tested by both sides.
2022-03-28 17:09:22 -07:00
Jordan Kay a8bba84598 Fix spelling and typos (#122)
* Fix spelling

* Fix typos
2022-03-28 17:02:51 -07:00
Vikram Kriplaney 739815f65d Fix zip example (#119) 2022-03-28 11:08:46 -07:00
Reza Shirazian 0dfb244b1b Fixing a few typos (#115)
This PR fixes a few typos across various comments in the code.
2022-03-27 08:19:52 -07:00
Philippe Hausler f9369af30c Add instructions for building with swift 5.6 toolchains (#106) 2022-03-25 14:49:20 -07:00
jamieQ 1656fd8839 Fix typo in AsyncChannel documentation (#108)
fix typo
2022-03-25 14:49:10 -07:00
Philippe Hausler db847ef410 Remove shims for use with the swift development toolchains (#105) 2022-03-24 16:03:21 -07:00
Philippe Hausler 21f20e9030 Only emit diagnostics for unexpected diagram mismatches (#103) 2022-03-24 15:14:40 -07:00
Philippe Hausler df97fdaced Update the code of conduct to avoid it going out of sync (#102) 2022-03-24 12:28:19 -07:00
Philippe Hausler 98d4229317 Add shims and adjust availability to allow for swift 5.6 toolchains to build until source stability is achieved (#100)
* Add shims and adjust availability to allow for swift 5.6 toolchains to build until source stability is achieved

* Restore availability again
2022-03-24 12:09:07 -07:00
Philippe Hausler f9d0911c0c Remove the docc plugin dependency from main (to make a docs branch) (#99) 2022-03-23 13:18:15 -07:00
Chris AdamsonandChris Adamson 33494899af Light edit of recent doc comments. (#98)
Co-authored-by: Chris Adamson <cadamson@apple.com>
2022-03-22 10:05:21 -04:00
Philippe Hausler 718d647945 Add a smattering of brief inline docs for most functions/types (#97) 2022-03-18 15:15:55 -07:00
kperryua d4bdb34283 Add a no-separator variant of joined() to match the stdlib (#87)
* Add a no-separator variant of joined() to match the stdlib

* @inlinable and @frozen
2022-03-18 14:04:00 -07:00
Philippe Hausler 11d7cfe1c9 First pass at some documentation for chunked/chunks (#92)
* First pass at some documentation for chunked/chunks

* Manually line break docs
2022-03-18 14:03:46 -07:00
Philippe Hausler b11a32fe4e First pass at some documentation for combineLatest (#94)
* First pass at some documentation for combineLatest

* Add additional docs for extension methods and AsyncCombineLatest2Sequence structure
2022-03-18 13:32:15 -07:00
Philippe Hausler f560d1c61f First pass at some documentation for debounce (#95)
* First pass at some documentation for debounce

* Add additional docs for extension methods and iterator
2022-03-18 13:31:40 -07:00
Philippe Hausler 7f3df92863 Add support for rendering docc (#89) 2022-03-18 12:32:46 -07:00
Philippe HauslerandChris Adamson bcd0ee1e50 [WIP] First pass at some documentation for AsyncChannel (#91)
* First pass at some documentation for AsyncChannel

* Apply suggestions from code review

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Manually line break docs

Co-authored-by: Chris Adamson <invalidname@gmail.com>
2022-03-18 11:39:45 -07:00
Philippe HauslerandChris Adamson 37ad908591 First pass at some documentation for AsyncBufferSequence (#90)
* First pass at some documentation for AsyncBufferSequence

* Apply suggestions from code review

Co-authored-by: Chris Adamson <invalidname@gmail.com>

Co-authored-by: Chris Adamson <invalidname@gmail.com>
2022-03-18 11:39:32 -07:00
Philippe HauslerandChris Adamson b8b04919e4 slight adjustments to make docc rendering of AsyncBufferedByteIterator documentation nicer (#93)
* slight adjustments to make docc rendering of AsyncBufferedByteIterator documentation nicer

* Apply suggestions from code review

Co-authored-by: Chris Adamson <invalidname@gmail.com>

Co-authored-by: Chris Adamson <invalidname@gmail.com>
2022-03-18 10:22:00 -07:00
4d4022e91b Add a whole bunch more to the guides (#64)
* Rebase guides WIP to avoid source changes

* Add prose for debounce and throttle

* Add an alterantive for debounce

* Starts to prose for reductions

* Add effect listings for all types

* Add some examples for reductions

* Update Guides/Channel.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Channel.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Channel.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Channel.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Collections.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Throttle.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Throttle.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Guides/Throttle.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Update Sources/AsyncAlgorithms/AsyncInclusiveReductionsSequence.swift

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Adjust events to elements for channels

* Add a bit more detail to the README and a start to the guide for Task.select

* Chunked updates

* Add a getting started section to the README

* Add some more prose for Channel

* Update Guides/RemoveDuplicates.md

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Apply suggestions from code review

Co-authored-by: Chris Adamson <invalidname@gmail.com>

* Add some brief prose for Compacted

* reductions could be named scan

* spelling and correctness fixes

* Apply @kperryua's suggestion for changing the intro of Compacted

* Add some brief prose for Timer

* Apply suggestions from code review for README

Co-authored-by: Tony Parker <anthony.parker@apple.com>

* Add some brief prose for RemoveDuplicates

* Fleshing out of `Joined`

Co-authored-by: Chris Adamson <invalidname@gmail.com>
Co-authored-by: Kevin Perry <kperry@apple.com>
Co-authored-by: Tony Parker <anthony.parker@apple.com>
2022-03-17 18:40:14 -07:00
Philippe Hausler e87e9ddd02 Reduce warnings for preconcurrency and non mutated vars (#88) 2022-03-17 16:13:57 -07:00
Philippe Hausler 3447a342ac Get nearly everything up and rolling for linux (#85)
* Get nearly everything up and rolling for linux

* Alter validation diagrams for an internal function to expect failures
2022-03-17 15:14:57 -07:00
Chris AdamsonandChris Adamson 5ac8bb4d07 Light edit on guides and sequence source docs (rdar://90176127) (#86)
* Light-touch edit on existing guides.

* Light edit on the sequences' source docs.

Co-authored-by: Chris Adamson <cadamson@apple.com>
2022-03-17 15:14:34 -07:00
Philippe Hausler 571bfd1a60 Add an implementation of inclusive reductions (#81) 2022-03-15 13:08:15 -07:00
Philippe Hausler 822ca9365c Restore availability and mark everything as 9999 to match stdlib availability for 5.7 for now (#82) 2022-03-15 09:21:51 -07:00
Kevin Perry ed1d2a04cf Fixed simple mistake from pr/buffer 2022-03-14 14:06:13 -07:00
Philippe Hausler 7545f0b7d2 Remove debugging prints from validation tests 2022-03-14 13:53:21 -07:00
Philippe Hausler 9d46190d67 Implement source location information for pairwise block builders (#58)
* Implement source location information for pairwise block builders

* Enable better source locations for latest validation diagrams

* Use the new naming for partial builders

* Ensure pairwise builder expressions preserve source locations
2022-03-14 13:51:00 -07:00
Philippe HauslerandKevin Perry 6ba2c05cfa A full knobs-and-dials buffering strategy transformation (#61)
* A full knobs-and-dials buffering strategy transformation

* Refine the AsyncBuffer type to require actor and add some additonal tests for custom buffer types

* Migrate the internal state to an actor

* Remove default unbounded policy from buffers

* Remove some dead code tests and fix space removal to avoid superfluous checks

* Simplification of nominal state machine, which helps eliminate actor interleaving issues. Fixed error/termination handling. Validation diagram tests, using “delay next” operators

* Most robust behavior when multiple tasks are concurrently accessing the iterator

* Removed Envelope, added base iterator Sendable requirement, and added conditional Sendable conformances

* Minor future-proofing

Co-authored-by: Kevin Perry <kperry@apple.com>
2022-03-14 13:41:44 -07:00
Philippe Hausler f0f348a680 Unify naming of chunks a bit more (#74) 2022-03-14 11:47:07 -07:00
Philippe Hausler de73225ecc Resolve a race condition that caused repeated testing of channel cancellation to sometimes hang (#80)
* Resolve a race condition that caused repeated testing of channel cancellation to sometimes hang

* Better name for removal for cancellation in inner Async(Throwing)Channel
2022-03-14 11:37:02 -07:00
Philippe Hausler 0fda7f3e0b Add a label for consistency with AsyncThrowingChannel for the element type of AsyncChannel (#76) 2022-03-11 17:59:15 -08:00
Philippe Hausler 09803ec7f9 Rename tests for Task.select and remove the testable import to enable release mode testing (#79) 2022-03-11 17:59:03 -08:00
Philippe Hausler 8ae8027341 Print diagnostics such that the comparisons of expected versus actual use the diagram as would be written (#77) 2022-03-11 16:24:24 -08:00
Philippe Hausler 307118031e Remove clock stubs and use the _Concurrency provided Clock APIs (#78)
* Remove the clock stubs

* Add default implementations using clocks
2022-03-11 16:24:08 -08:00
Philippe Hausler 4b11faad62 Clean up some more warnings via an additional Sendability pass audit (#72) 2022-03-11 12:56:43 -08:00
kperryua 624eea0387 Merge pull request #73 from apple/pr/delay-next
Add delayNext “,” operator to validation diagram
2022-03-09 11:17:40 -08:00
Kevin Perry fee904ba10 Add delayNext “,” operator to validation diagram 2022-03-08 16:54:53 -08:00
Philippe Hausler a52b55c8ac Reduce warnings from preconcurrency issues and unused values (#71) 2022-03-07 12:55:17 -08:00
kperryua 2722655bdb Merge pull request #69 from apple/pr/chunked-name-fixes
Tweaked signatures of chunks/chunked methods
2022-03-03 17:05:45 -08:00
Kevin Perry 625ac79d5f Tweaked signatures of chunks/chunked methods 2022-03-03 12:08:57 -08:00
Philippe Hausler 325923a5e6 Add a static method for constructing timers (#67) 2022-03-01 15:23:16 -08:00
Philippe Hausler 61bc878106 Add a performance tuned asynchronous iterator for creeating asynchronous byte sequences (#39)
* Add a performance tuned asynchronous iterator for creeating asynchronous byte sequences

* Some spelling fixes and language clarity improvements

* Add some more detail on the docs for the `AsyncBufferedByteIterator` initializer

* Move the storage to a usable from inline type
2022-03-01 13:25:55 -08:00
Philippe Hausler 9377b4de58 Validate AsyncSequence specification SE-0323 (#60) 2022-03-01 12:37:45 -08:00
kperryua d90945abc8 Merge pull request #66 from apple/pr/chunked-collection-type-fix
Fixed element type of chunks(ofCount:collectedInto:)
2022-03-01 12:05:53 -08:00
Kevin Perry 7b62584d99 Fixed element type of chunks(ofCount:collectedInto:) 2022-03-01 12:03:32 -08:00
Philippe Hausler 2683ddc9c3 Rename to AsyncChannel (#63) 2022-03-01 09:11:37 -08:00
kperryua b8242c4a6b Add sequences that chunk values into collections based on a variety of criteria (#55)
* Add sequences that chunk values into collections based on a variety of criteria

* Factor out Merge2 logic so its termination behavior can be customized

* Rewrite AsyncChunksOfCountAndTriggerSequence in terms of the newly configurable merge state machine

* Name changes

* Fixed tests

* Enable chunked(byTime:) functions

* Broken up into one file per sequence
2022-02-23 17:17:20 -08:00
Philippe Hausler eb0ddf54cc Rename validation tests to match latest testing API (#59) 2022-02-23 15:53:21 -08:00
Philippe Hausler 4c35512ba6 implement a sequence for debouncing incoming events to an expected quiescence duration (#43)
* implement a sequence fir debouncing incoming events to an expected quiessence duration

* Ensure debouced sequences are sendable and update testing to the new validate method

* Add hangover tests and cancel sleep per iteration of production
2022-02-23 15:04:45 -08:00
Philippe Hauslerandkperryua 7969001299 Add a detailed proposal-like guide for AsyncSequence validation (#56)
* Add a detailed proposal like guide for MarbleDiagram

* Slight spacing change

Co-authored-by: kperryua <kperry@apple.com>

* Fix some pluralization agreement

Co-authored-by: kperryua <kperry@apple.com>

* Clarify the testing of merge and add a future direction for unordered groups

* rename the diagram system to a more general 'validate' based name

* Spelling fixes

* Grammar fixes for credits

Co-authored-by: kperryua <kperry@apple.com>
2022-02-22 11:03:27 -08:00
Philippe Hausler a6ad9b82e9 Conditional conformance for Sendable for throttle types (#52) 2022-02-18 16:44:26 -08:00
Philippe Hausler 5bfab6f07a Add a timer based async sequence (#54)
* Support cancellation for marble diagrams

* Add a timer based async sequence

* Test skewing events by sleeps inside the map

* A less confusing skew cancellation tick
2022-02-18 16:44:07 -08:00
Philippe Hausler 17bca92f90 fix some typos in the code samples for combineLatest (#53) 2022-02-18 15:24:21 -08:00
Philippe Hausler aebc7af342 instant math for throttle 2022-02-18 00:07:55 -08:00
Philippe Hausler 46b3bb088b implement a sequence for throttling incoming rates to a minimum spaced duration (#41)
* implement a sequence for throttling incoming rates to a minimum spaced duration

* Add delayed emission tests for throttle
2022-02-18 00:06:06 -08:00
Philippe Hausler d96606e7f7 Clean up the marble diagram implementation and factor it into a per Task based queue system (#44)
* Split the marble diagram code out into their own files for easier refactoring

* Update latest landing InstantProtocol definition stub

* Migrate to a per Task based queue system to avoid blocking valid future jobs

* Factor inputs to be specialized waits for side tracking

* Add grouping support

* Add tests for grouping and parse failures

* Fixup misstep in package definition for clock stub imports for the marable diagrams

* Add descriptions for partial iterations for debugging aides

* Simplify Task.select implementation to handle continuations more encapsulatedly

* Add descriptions for WorkQueue items for debugging and use stable work queue drains
2022-02-18 00:05:36 -08:00
Philippe Hausler 303e6a9b15 Add marble diagrams for merge and correct partial state transitions (#42) 2022-02-18 00:04:44 -08:00
Philippe Hausler f20f7fbe85 Rework marble diagrams to account for task execution (#40)
* Rework marble diagrams to account for task execution

* Add marble diagram tests to validate failure paths

* Test for space no-op in marable diagram DSL

* Factor out parsing into events and generalize
2022-02-14 11:50:05 -08:00
Philippe Hausler 47aa4ccba6 A continuous form of reduce and a throwing variant of that reduction (#35) 2022-02-08 10:16:17 -08:00
f91de2e1f3 Add a guide for zip(_:...) (#38)
* Add a guide for zip(_:...)

* Update Guides/Zip.md

Co-authored-by: Tony Parker <anthony.parker@apple.com>

* Typo corrections for reference to Combine's Publisher.zip

* Better wording for rethrowing behavior of `AsyncZipSequence`

* Update Guides/Zip.md

Co-authored-by: kperryua <kperry@apple.com>

Co-authored-by: Tony Parker <anthony.parker@apple.com>
Co-authored-by: kperryua <kperry@apple.com>
2022-02-07 16:04:03 -08:00
Philippe Hausler 2de7a14d32 Add a guide for interspersed(with:) (#36) 2022-02-07 16:02:36 -08:00
Philippe Hauslerandkperryua c2c73ecffe Add a guide for merge(_:...) (#37)
* Add a guide for merge(_:...)

* Typo corrections for reference to Combine's Publisher.merge

* Update Guides/Merge.md

Co-authored-by: kperryua <kperry@apple.com>

* Update Guides/Merge.md

Co-authored-by: kperryua <kperry@apple.com>

Co-authored-by: kperryua <kperry@apple.com>
2022-02-07 15:59:32 -08:00
Philippe HauslerandTony Parker 5c34eadcc0 Add a guide for combineLatest(_:...) (#33)
* Add a guide for combineLatest(_:...)

* Update Guides/CombineLatest.md

Co-authored-by: Tony Parker <anthony.parker@apple.com>

* Update README.md link to CombineLatest.md

* Add a sample output table to indicate timing and combine behavior

Co-authored-by: Tony Parker <anthony.parker@apple.com>
2022-02-01 15:25:44 -08:00
Philippe Hausler aed374f1f6 Add some brief descriptions of the work so far done and some prose about the package as a whole and how to use it (#32)
* Add some brief descriptions of the work so far done and some prose about the package as a whole and how to use it

* Remove blog post reference for now
2022-02-01 14:01:24 -08:00
Philippe Hausler d905d2a465 Async on both send and iteration subject like behavior (#34)
* Async on both send and iteration subject like behavior

* rename the initial state to idle for subjects
2022-02-01 09:21:06 -08:00
Philippe Hausler 0e37dffc8c Correct generation rename misstep 2022-01-31 15:40:40 -08:00
Philippe Hausler 845d11a3bd Marble diagram based testing (#30)
* First draft at marble diagram based tests

* Correct indefinite marble runs, support grouping, and properly handle any number of inputs (by the result builder construction)

* enhance isolation of diagrams with fileprivate
2022-01-31 15:28:17 -08:00
Philippe Hausler c147131251 Add a testing support item for manual clocks (#27)
* Add a testing support item for manual clocks

* rename `id` to `generation` and add an advance by steps
2022-01-31 15:25:42 -08:00
Philippe Hausler a4d279df70 Add a development guide for chain(_:...) (#31) 2022-01-27 16:22:44 -08:00
Philippe HauslerandTony Parker 1dbed37b56 Set up a directory for proposals for evolving swift-async-algorithms (#25)
* Set up a directory for proposals for evolving swift-async-algorithms

* Remove some langauge in the proposal template that is more focused upon Swift as a language rather than library evolution

* Update Evoluition/0000-swift-async-algorithms-template.md

Co-authored-by: Tony Parker <anthony.parker@apple.com>

* Correct typo of directory name for evolution

* Update bug listing to the github issues link

* Remove ABI consideration section and just leave API resilience

Co-authored-by: Tony Parker <anthony.parker@apple.com>
2022-01-26 13:33:41 -08:00
Philippe Hausler 91aa4a85d4 refine the sample time for throughput measurements so that it does not take so long for the tests (#28) 2022-01-25 14:57:04 -08:00
Philippe Hausler 7e3b1c1164 Add a 2 and 3 arity version of combineLatest (#20) 2022-01-25 13:14:38 -08:00
kperryua 91968fa3e9 AsyncSequence throughput performance tests (#19)
* AsyncSequence throughput performance tests

* Reduce source sequence overhead

* Normalize whitespace

* Added joined() throughput test

* File renaming
2022-01-25 12:07:23 -08:00
Philippe Hausler 18fcfe8e29 [NFC] Fixup attribution dates in comments (#22) 2022-01-25 09:52:32 -08:00
Philippe Hausler 27e6645329 Add a validator testing support mechanism (#24)
* Add a validator testing support mechanism

* Move the remaining support items into the test support directory

* Add rudimentary tests to validate the validator
2022-01-24 17:40:58 -08:00
Philippe Hausler ad2d2fca7e Add a stub implementation of the clock/instant/duration (#26) 2022-01-24 17:40:20 -08:00
kperryua 4c93fa2c87 Merge pull request #17 from apple/pr/joined
Add a sequence for joining multiple async sequences with separators
2022-01-24 17:24:23 -08:00
Philippe Hausler 21520e710c Rework Task.first to a better name/behavior of Task.select (#21)
* Rework Task.first to a better name/behavior of Task.select

* Downgrade UnsafeResumption to internal
2022-01-24 13:38:51 -08:00
Philippe Hausler b015c5e49e Clean up some asserts by making helper overloads (#18) 2022-01-24 13:36:22 -08:00
Philippe Hausler 76785b34e4 [NFC] Move testing support files into their own directory (#23) 2022-01-24 13:31:21 -08:00
Kevin Perry 8c93e6ba6c Add a sequence for joining multiple async sequences with separators 2022-01-21 13:14:42 -08:00
Philippe Hausler 34682caadf AsyncChainSequences should be Sendable if their bases are Sendable (#15) 2022-01-21 12:32:36 -08:00
Philippe Hausler 3883a4033b Remove Codable, Hashable, and Equatable conditional conformance (#16) 2022-01-21 12:32:21 -08:00
kperryua b515cd8458 Merge pull request #14 from apple/pr/intersperse
Add a sequence that intersperses separator values
2022-01-21 12:10:06 -08:00
kperryua f614b51e55 Merge pull request #13 from apple/pr/remove-duplicates
Add a sequence for removing adjacent duplicates
2022-01-21 12:09:59 -08:00
Kevin Perry 431393dc68 Fixed test indentation 2022-01-21 12:09:07 -08:00
Kevin Perry 93b868b59a Fixed comments 2022-01-21 12:09:07 -08:00
Kevin Perry 450f2d6827 Test cleanup 2022-01-21 12:09:07 -08:00
Kevin Perry 8295e01ac8 Add a sequence that intersperses separator values 2022-01-21 12:09:02 -08:00
Philippe Hausler 205414d19d Add a 2 and 3 arity version of merge with tuple backed storage (#8)
* Add a 2 and 3 arity version of merge with tuple backed storage

* Xcode really likes Metal, one could say the weighting for importing for that is heavy...
2022-01-21 11:52:37 -08:00
Philippe Hausler 0316ac34f3 Add a sequence for compacting nils (#6)
* Add a sequence for compacting nils

* Remove Codable, Hashable, and Equatable conditional conformance

* Add test for upstream throwing for compacted
2022-01-21 11:52:27 -08:00
Philippe Hausler d1a8f225eb Add documentation for AsyncLazySequence (#4) 2022-01-21 11:52:15 -08:00
Philippe Hausler 73b29b9cbf Add a 2 and 3 arity version of zip (#10) 2022-01-21 11:37:25 -08:00
Philippe Hausler 402f59efb4 Add a draft guide for AsyncLazySequence (#12) 2022-01-19 15:00:59 -08:00
Kevin Perry e71fe66e29 Test cleanup 2022-01-19 14:49:40 -08:00
Kevin Perry 8419f51fa6 Sendable conformance 2022-01-19 14:46:08 -08:00
Kevin Perry e7dc87d00c Add a sequence for removing adjacent duplicates 2022-01-19 14:46:08 -08:00
Philippe Hausler 8a1078810c Add a 2 and 3 arity version of chain (#9)
* Add a 2 and 3 arity version of chain

* Tests for cancellation of chain sequences

* Correct typo of missing [] in chain 3 test for cancellation

* Make chain 2/3 inlinable and frozen
2022-01-18 11:50:25 -08:00
Philippe Hausler b2aad0be7d Ignore the .swiftpm directory to avoid potential commits of project data (#11) 2022-01-18 11:18:11 -08:00
Philippe Hausler beb509d829 Fix test build failure with @testable import 2022-01-14 08:26:57 -08:00
Philippe Hausler 1c11d4a7c8 Add an algorithm to obtain the first result of a list of tasks (#5)
* Add an algorithm to obtain the first result of a list of tasks

* A much simpler and less risky implementation of Task.first that supports cancellation propigation

* Slight whitespace fix for Package.swift

* Demote Task.first to internal - revisit for better naming later as public
2022-01-14 08:25:41 -08:00
Philippe HauslerandNate Cook 1c08de7680 Add async rethrowing initializers for RangeReplaceableCollection, SetAlgebra, and Dictionary for use with AsyncSequence types (#3)
* Add async rethrowing initializers for RangeReplaceableCollection, SetAlgebra, and Dictionary for use with AsyncSequence types

* Add some additional tests for throwing coverage and documentation on collection initializers

* Spelling fix of unique

Co-authored-by: Nate Cook <natecook@apple.com>

* Spelling fix for unique (throwing version)

Co-authored-by: Nate Cook <natecook@apple.com>

Co-authored-by: Nate Cook <natecook@apple.com>
2022-01-14 08:23:14 -08:00
Philippe Hausler 2b4f602320 Add a primitive storage for managing critical state and platform abstracted locking (#7) 2022-01-13 16:18:50 -08:00
Philippe Hausler 361c7926d1 Add a package layout and initial implementation and tests for AsyncLazySequence (#2)
* Add a package layout and initial implementation and tests for AsyncLazySequence; a type for converting non asynchronous sequences into asynchronous ones

* Add a first iteration pass expectation to ensure that tasks spawn for testing cancellation of AsyncLazySequence

* Simplify the iterator for AsyncLazySequence
2022-01-11 10:26:32 -08:00
Tony Parker 7f073235cb Merge pull request #1 from parkera/license
Add Swift code of conduct, contribution guidelines, license
2022-01-10 14:31:26 -08:00
Tony Parker 4c0a02a63c Add Swift code of conduct, contribution guidelines, license 2022-01-10 14:30:58 -08:00
Ted Kremenek d1155eca5c Initial commit 2022-01-10 14:02:09 -08:00
208 changed files with 38437 additions and 0 deletions

No files matched your search

+8
View File
@@ -0,0 +1,8 @@
root = true
[*]
indent_style = space
indent_size = 2
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
+26
View File
@@ -0,0 +1,26 @@
name: Pull request
permissions:
contents: read
on:
pull_request:
types: [opened, reopened, synchronize]
jobs:
tests:
name: Test
uses: swiftlang/github-workflows/.github/workflows/swift_package_test.yml@main
with:
linux_exclude_swift_versions: "[{\"swift_version\": \"5.9\"}, {\"swift_version\": \"5.10\"}]]"
windows_exclude_swift_versions: "[{\"swift_version\": \"5.9\"}]"
enable_wasm_sdk_build: true
swift_flags: "--enable-all-traits"
swift_nightly_flags: "--enable-all-traits"
soundness:
name: Soundness
uses: swiftlang/github-workflows/.github/workflows/soundness.yml@main
with:
license_header_check_project_name: "Swift Async Algorithms"
format_check_container_image: "swiftlang/swift:nightly-main-noble" # Needed due to https://github.com/swiftlang/swift-format/issues/1081
+13
View File
@@ -0,0 +1,13 @@
.DS_Store
/.build
/Packages
/*.xcodeproj
xcuserdata/
DerivedData/
.swiftpm/config/registries.json
.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata
.netrc
.swiftpm
Package.resolved
.swift-version
test_output.log
+10
View File
@@ -0,0 +1,10 @@
@@===----------------------------------------------------------------------===@@
@@
@@ This source file is part of the Swift Async Algorithms open source project
@@
@@ Copyright (c) YEARS Apple Inc. and the Swift project authors
@@ Licensed under Apache License v2.0 with Runtime Library Exception
@@
@@ See https://swift.org/LICENSE.txt for license information
@@
@@===----------------------------------------------------------------------===@@
+40
View File
@@ -0,0 +1,40 @@
.gitignore
**/.gitignore
.licenseignore
.gitattributes
.mailfilter
.mailmap
.spi.yml
.swift-format
.editorconfig
.github/*
*.md
*.txt
*.yml
*.yaml
*.json
Package.swift
**/Package.swift
Package@-*.swift
Package@swift-*.swift
**/Package@-*.swift
Package.resolved
**/Package.resolved
Makefile
*.modulemap
**/*.modulemap
**/*.docc/*
*.xcprivacy
**/*.xcprivacy
*.symlink
**/*.symlink
Dockerfile
**/Dockerfile
Snippets/*
dev/git.commit.template
*.crt
**/*.crt
*.pem
**/*.pem
*.der
**/*.der
+4
View File
@@ -0,0 +1,4 @@
version: 1
builder:
configs:
- documentation_targets: [AsyncAlgorithms]
+58
View File
@@ -0,0 +1,58 @@
{
"fileScopedDeclarationPrivacy" : {
"accessLevel" : "private"
},
"indentation" : {
"spaces" : 2
},
"indentConditionalCompilationBlocks" : false,
"indentSwitchCaseLabels" : false,
"lineBreakAroundMultilineExpressionChainComponents" : false,
"lineBreakBeforeControlFlowKeywords" : false,
"lineBreakBeforeEachArgument" : true,
"lineBreakBeforeEachGenericRequirement" : true,
"lineLength" : 120,
"maximumBlankLines" : 1,
"prioritizeKeepingFunctionOutputTogether" : true,
"respectsExistingLineBreaks" : true,
"rules" : {
"AllPublicDeclarationsHaveDocumentation" : false,
"AlwaysUseLowerCamelCase" : false,
"AmbiguousTrailingClosureOverload" : true,
"BeginDocumentationCommentWithOneLineSummary" : false,
"DoNotUseSemicolons" : true,
"DontRepeatTypeInStaticProperties" : true,
"FileScopedDeclarationPrivacy" : true,
"FullyIndirectEnum" : true,
"GroupNumericLiterals" : true,
"IdentifiersMustBeASCII" : true,
"NeverForceUnwrap" : false,
"NeverUseForceTry" : false,
"NeverUseImplicitlyUnwrappedOptionals" : false,
"NoAccessLevelOnExtensionDeclaration" : true,
"NoAssignmentInExpressions" : true,
"NoBlockComments" : true,
"NoCasesWithOnlyFallthrough" : true,
"NoEmptyTrailingClosureParentheses" : true,
"NoLabelsInCasePatterns" : false,
"NoLeadingUnderscores" : false,
"NoParensAroundConditions" : true,
"NoVoidReturnOnFunctionSignature" : true,
"OneCasePerLine" : true,
"OneVariableDeclarationPerLine" : true,
"OnlyOneTrailingClosureArgument" : true,
"OrderedImports" : false,
"ReturnVoidInsteadOfEmptyTuple" : true,
"UseEarlyExits" : true,
"UseLetInEveryBoundCaseVariable" : false,
"UseShorthandTypeNames" : true,
"UseSingleLinePropertyGetter" : false,
"UseSynthesizedInitializer" : false,
"UseTripleSlashForDocumentationComments" : true,
"UseWhereClausesInForLoops" : false,
"ValidateDocumentationComments" : false
},
"spacesAroundRangeFormationOperators" : false,
"tabWidth" : 4,
"version" : 1
}
+7
View File
@@ -0,0 +1,7 @@
extends: default
rules:
line-length: false
document-start: false
truthy:
check-keys: false # Otherwise we get a false positive on GitHub action's `on` key
+5
View File
@@ -0,0 +1,5 @@
# Code of Conduct
The code of conduct for this project can be found at https://swift.org/code-of-conduct.
<!-- Copyright (c) 2022 Apple Inc and the Swift Project authors. All Rights Reserved. -->
+10
View File
@@ -0,0 +1,10 @@
By submitting a pull request, you represent that you have the right to license
your contribution to Apple and the community, and agree by submitting the patch
that your contributions are licensed under the [Swift
license](https://swift.org/LICENSE.txt).
---
Before submitting the pull request, please make sure you have tested your
changes and that they follow the Swift project [guidelines for contributing
code](https://swift.org/contributing/#contributing-code).
@@ -0,0 +1,67 @@
# Feature name
* Proposal: [NNNN](NNNN-filename.md)
* Authors: [Author 1](https://github.com/swiftdev), [Author 2](https://github.com/swiftdev)
* Review Manager: TBD
* Status: **Awaiting implementation**
*During the review process, add the following fields as needed:*
* Implementation: [apple/swift-async-algorithms#NNNNN](https://github.com/apple/swift-async-algorithms/pull/NNNNN)
* Decision Notes: [Rationale](https://forums.swift.org/), [Additional Commentary](https://forums.swift.org/)
* Bugs: [NNNN](https://github.com/apple/swift-async-algorithms/issues)
## Introduction
A short description of what the feature is. Try to keep it to a
single-paragraph "elevator pitch" so the reader understands what
problem this proposal is addressing.
Swift forums thread: [Discussion thread topic for that proposal](https://forums.swift.org/)
## Motivation
Describe the problems that this proposal seeks to address. If the
problem is that some common pattern is currently hard to express, show
how one can currently get a similar effect and describe its
drawbacks. If it's completely new functionality that cannot be
emulated, motivate why this new functionality would help Swift
developers create better Swift code.
## Proposed solution
Describe your solution to the problem. Provide examples and describe
how they work. Show how your solution is better than current
workarounds: is it cleaner, safer, or more efficient?
## Detailed design
Describe the design of the solution in detail. If it involves new
syntax in the language, show the additions and changes to the Swift
grammar. If it's a new API, show the full API and its documentation
comments detailing what it does. The detail in this section should be
sufficient for someone who is *not* one of the authors to be able to
reasonably implement the feature.
## Effect on API resilience
API resilience describes the changes one can make to a public API
without breaking its ABI. Does this proposal introduce features that
would become part of a public API? If so, what kinds of changes can be
made without breaking ABI? Can this feature be added/removed without
breaking ABI? For more information about the resilience model, see the
[library evolution
document](https://github.com/apple/swift/blob/main/docs/LibraryEvolution.rst)
in the Swift repository.
## Alternatives considered
Describe alternative approaches to addressing the same problem, and
why you chose this approach instead.
## Acknowledgments
If significant changes or improvements suggested by members of the
community were incorporated into the proposal as it developed, take a
moment here to thank them for their contributions. Swift evolution is a
collaborative process, and everyone's input should receive recognition!
+108
View File
@@ -0,0 +1,108 @@
# Zip
* Proposal: [SAA-0001](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0001-zip.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Zip/AsyncZip2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Zip/AsyncZip3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestZip.swift)]
* Decision Notes:
* Bugs:
## Introduction
The swift standard library has a function that allows for the combining of two sequences into one sequence of tuples of the elements of the base sequences. This concept can be achieved for `AsyncSequence` with the iteration being asynchronous but also each side being concurrently iterated while still rethrowing potential failures. This proposal covers that parity between `AsyncSequence` and `Sequence`. It is often times useful to describe asynchronous sequences of events as paired occurrences. The fundamental algorithm for this is zip.
## Detailed Design
Zip combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
```swift
let appleFeed = URL(string: "http://www.example.com/ticker?symbol=AAPL")!.lines
let nasdaqFeed = URL(string: "http://www.example.com/ticker?symbol=^IXIC")!.lines
for try await (apple, nasdaq) in zip(appleFeed, nasdaqFeed) {
print("APPL: \(apple) NASDAQ: \(nasdaq)")
}
```
Given some sample inputs the following zipped events can be expected.
| Timestamp | appleFeed | nasdaqFeed | combined output |
| ----------- | --------- | ---------- | ----------------------------- |
| 11:40 AM | 173.91 | | |
| 12:25 AM | | 14236.78 | AAPL: 173.91 NASDAQ: 14236.78 |
| 12:40 AM | | 14218.34 | |
| 1:15 PM | 173.00 | | AAPL: 173.00 NASDAQ: 14218.34 |
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func zip<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncZip2Sequence<Base1, Base2>
public func zip<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncZip3Sequence<Base1, Base2, Base3>
public struct AsyncZip2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable, Base2.Element: Sendable {
public typealias Element = (Base1.Element, Base2.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
public struct AsyncZip3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable, Base3: Sendable
Base1.Element: Sendable, Base2.Element: Sendable, Base3.Element: Sendable {
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The `zip(_:...)` function takes two or more asynchronous sequences as arguments with the resulting `AsyncZipSequence` which is an asynchronous sequence.
Each iteration of an `AsyncZipSequence` will await for all base iterators to produce a value. This iteration will be done concurrently to produce a singular tuple result. If any of the base iterations terminates by returning `nil` from its iteration, the `AsyncZipSequence` iteration is immediately considered unsatisfiable and returns `nil` and all iterations of other bases will be cancelled. If any iteration of the bases throws an error, then the other iterations concurrently running are cancelled and the produced error is rethrown, terminating the iteration.
`AsyncZipSequence` requires that the iterations are done concurrently. This means that the base sequences, their elements, and iterators must all be `Sendable`. That makes `AsyncZipSequence` inherently `Sendable`.
The source of throwing of `AsyncZipSequence` is determined by its bases. That means that if any base can throw an error then the iteration of the `AsyncZipSequence` can throw. If no bases can throw, then the `AsyncZipSequence` does not throw.
### Naming
The `zip(_:...)` function takes its name from the Swift standard library function of the same name. The `AsyncZipSequence` family of types take their name from the same family from the standard library for the type returned by `zip(_:_:)`. The one difference is that this asynchronous version allows for the affordance of recognizing the eventual variadic generic need of expanding a zip of more than just two sources.
It is common in some libraries to have a `ZipMap` or some other combination of `zip` and `map`. This is a common usage pattern, but leaving a singular type for composition feels considerably more approachable.
### Comparison with other libraries
**Swift** The swift standard library has an [API definition of zip](https://developer.apple.com/documentation/swift/1541125-zip) as a top level function for combining two sequences.
**ReactiveX** ReactiveX has an [API definition of Zip](https://reactivex.io/documentation/operators/zip.html) as a top level function for combining Observables.
**Combine** Combine has an [API definition of zip](https://developer.apple.com/documentation/combine/publisher/zip(_:)/) as an operator style method for combining Publishers.
## Effect on API resilience
### `@frozen` and `@inlinable`
These types utilize rethrowing mechanisms that are awaiting an implementation in the compiler for supporting implementation based rethrows. So none of them are marked as frozen or marked as inlinable. This feature (discussed as `rethrows(unsafe)` or `rethrows(SourceOfRethrowyness)` has not yet been reviewed or implemented. The current implementation takes liberties with an internal protocol to accomplish this task. Future revisions will remove that protocol trick to replace it with proper rethrows semantics at the actual call site. The types are expected to be stable boundaries to prevent that workaround for the compilers yet to be supported rethrowing (or TaskGroup rethrowing) mechanisms. As soon as that feature is resolved; a more detailed investigation on performance impact of inlining and frozen should be done before 1.0.
## Alternatives considered
It was considered to have zip be shaped as an extension method on `AsyncSequence` however that infers a "primary-ness" of one `AsyncSequence` over another. Since the standard library spells this as a global function (which infers no preference to one side or another) it was decided that having symmetry between the asynchronous version and the synchronous version inferred the right connotations.
There are other methods with similar behavior that could be controlled by options passed in. This concept has merit but was initially disregarded since that would complicate the interface. Design-wise this is still an open question if having a "zip-behavior-options" parameter to encompass combining the latest values or zipping based upon a preference to a "primary" side or not is meaningful.
It is common to have a zip+map to create structures instead of tuples, however that was disregarded since that concept could easily be expressed by composing zip and map.
+101
View File
@@ -0,0 +1,101 @@
# Merge
* Proposal: [SAA-0002](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0002-merge.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Merge/AsyncMerge2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Merge/AsyncMerge3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestMerge.swift)]
* Decision Notes:
* Bugs:
## Introduction
In the category of combinations of asynchronous sequences there are a few different potential behaviors. This category all take two or more `AsyncSequence` types and produce one `AsyncSequence`. One fundamental behavior is taking all values produced by the inputs and resuming the iteration of the singular downstream `AsyncSequence` with those values. This shape is called merge.
## Detailed Design
Merge takes two or more asynchronous sequences sharing the same element type and combines them into one singular asynchronous sequence of those elements.
```swift
let appleFeed = URL(string: "http://www.example.com/ticker?symbol=AAPL")!.lines.map { "AAPL: " + $0 }
let nasdaqFeed = URL(string:"http://www.example.com/ticker?symbol=^IXIC")!.lines.map { "^IXIC: " + $0 }
for try await ticker in merge(appleFeed, nasdaqFeed) {
print(ticker)
}
```
Given some sample inputs the following merged events can be expected.
| Timestamp | appleFeed | nasdaqFeed | merged output |
| ----------- | --------- | ---------- | --------------- |
| 11:40 AM | 173.91 | | AAPL: 173.91 |
| 12:25 AM | | 14236.78 | ^IXIC: 14236.78 |
| 12:40 AM | | 14218.34 | ^IXIC: 14218.34 |
| 1:15 PM | 173.00 | | AAPL: 173.00 |
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func merge<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncMerge2Sequence<Base1, Base2>
public func merge<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncMerge3Sequence<Base1, Base2, Base3>
public struct AsyncMerge2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Sendable
where
Base1.Element == Base2.Element,
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable, Base2.Element: Sendable {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
public struct AsyncMerge3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where
Base1.Element == Base2.Element, Base1.Element == Base3.Element,
Base1: Sendable, Base2: Sendable, Base3: Sendable
Base1.Element: Sendable, Base2.Element: Sendable, Base3.Element: Sendable {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The `merge(_:...)` function takes two or more asynchronous sequences as arguments and produces an `AsyncMergeSequence` which is an asynchronous sequence.
Since the bases comprising the `AsyncMergeSequence` must be iterated concurrently to produce the latest value, those sequences must be able to be sent to child tasks. This means that a prerequisite of the bases must be that the base asynchronous sequences, their iterators, and the elements they produce must be `Sendable`.
When iterating a `AsyncMergeSequence`, the sequence terminates when all of the base asynchronous sequences terminate, since this means there is no potential for any further elements to be produced.
The throwing behavior of `AsyncMergeSequence` is that if any of the bases throw, then the composed asynchronous sequence throws on its iteration. If at any point an error is thrown by any base, the other iterations are cancelled and the thrown error is immediately thrown to the consuming iteration.
### Naming
Since the inherent behavior of `merge(_:...)` merges values from multiple streams into a singular asynchronous sequence, the naming is intended to be quite literal. There are precedent terms of art in other frameworks and libraries (listed in the comparison section). Other naming takes the form of "withLatestFrom". This was disregarded since the "with" prefix is often most associated with the passing of a closure and some sort of contextual concept; `withUnsafePointer` or `withUnsafeContinuation` are prime examples.
### Comparison with other libraries
**ReactiveX** ReactiveX has an [API definition of Merge](https://reactivex.io/documentation/operators/merge.html) as a top level function for merging Observables.
**Combine** Combine has an [API definition of merge(with:)](https://developer.apple.com/documentation/combine/publisher/merge(with:)-7qt71/) as an operator style method for merging Publishers.
## Effect on API resilience
### `@frozen` and `@inlinable`
These types utilize rethrowing mechanisms that are awaiting an implementation in the compiler for supporting implementation based rethrows. So none of them are marked as frozen or marked as inlinable. This feature (discussed as `rethrows(unsafe)` or `rethrows(SourceOfRethrowyness)` has not yet been reviewed or implemented. The current implementation takes liberties with an internal protocol to accomplish this task. Future revisions will remove that protocol trick to replace it with proper rethrows semantics at the actual call site. The types are expected to be stable boundaries to prevent that workaround for the compilers yet to be supported rethrowing (or TaskGroup rethrowing) mechanisms. As soon as that feature is resolved; a more detailed investigation on performance impact of inlining and frozen should be done before 1.0.
## Alternatives considered
It was considered to have merge be shaped as an extension method on `AsyncSequence` however that infers a "primary-ness" of one `AsyncSequence` over another. Since the behavior of this as a global function (which infers no preference to one side or another) it was decided that having symmetry between the asynchronous version and the synchronous version inferred the right connotations.
+61
View File
@@ -0,0 +1,61 @@
# Compacted
* Proposal: [SAA-0003](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0003-compacted.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncCompactedSequence.swift)
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestCompacted.swift)
## Proposed Solution
Similar to the Swift Algorithms package we propose that a new method be added to `AsyncSequence` to fit this need.
```swift
extension AsyncSequence {
public func compacted<Unwrapped>() -> AsyncCompactedSequence<Self, Unwrapped>
where Element == Unwrapped?
}
```
This is equivalent to writing `.compactMap { $0 }` from a behavioral standpoint but is easier to reason about and is more efficient since it does not need to execute or store a closure.
## Detailed Design
The `AsyncCompactedSequence` type from an effects standpoint works just like `AsyncCompactMapSequence`. When the base asynchronous sequence throws, the iteration of `AsyncCompactedSequence` can throw. Likewise if the base does not throw then the iteration of `AsyncCompactedSequence` does not throw. This type is conditionally `Sendable` when the base, base element, and base iterator are `Sendable.
```swift
public struct AsyncCompactedSequence<Base: AsyncSequence, Element>: AsyncSequence
where Base.Element == Element? {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator())
}
}
extension AsyncCompactedSequence: Sendable
where
Base: Sendable, Base.Element: Sendable,
Base.AsyncIterator: Sendable { }
extension AsyncCompactedSequence.Iterator: Sendable
where
Base: Sendable, Base.Element: Sendable,
Base.AsyncIterator: Sendable { }
```
## Effect on API resilience
Compacted has a trivial implementation and is marked as `@frozen` and `@inlinable`. This removes the ability of this type and functions to be ABI resilient boundaries at the benefit of being highly optimizable.
## Alternatives considered
None; shy of potentially eliding this since the functionality is so trivial. However the utility of this function aides in ease of use and approachability along with parity with the Swift Algorithms package.
## Acknowledgments
This transformation function is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Compacted.md)
+67
View File
@@ -0,0 +1,67 @@
# Joined
* Proposal: [SAA-0004](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0004-joined.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Review Manager: [Franz Busch](https://github.com/FranzBusch)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncJoinedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestJoin.swift)]
* Decision Notes:
* Bugs:
## Introduction
The `joined()` and `joined(separator:)` algorithms on `AsyncSequence`s provide APIs to concatenate an `AsyncSequence` of `AsyncSequence`s.
```swift
extension AsyncSequence where Element: AsyncSequence {
public func joined() -> AsyncJoinedSequence<Self>
}
extension AsyncSequence where Element: AsyncSequence {
public func joined<Separator: AsyncSequence>(separator: Separator) -> AsyncJoinedBySeparatorSequence<Self, Separator>
}
```
## Detailed Design
These algorithms iterate over the elements of each `AsyncSequence` one bye one, i.e. only after the iteration of one `AsyncSequence` has finished the next one will be started.
```swift
let appleFeed = URL("http://www.example.com/ticker?symbol=AAPL").lines
let nasdaqFeed = URL("http://www.example.com/ticker?symbol=^IXIC").lines
for try await line in [appleFeed, nasdaqFeed].async.joined() {
print("\(line)")
}
```
Given some sample inputs the following combined events can be expected.
| Timestamp | appleFeed | nasdaqFeed | output |
| ----------- | --------- | ---------- | ----------------------------- |
| 11:40 AM | 173.91 | | 173.91 |
| 12:25 AM | | 14236.78 | |
| 12:40 AM | | 14218.34 | |
| 1:15 PM | 173.00 | | 173.00 |
| 1:15 PM | | | 14236.78 |
| 1:15 PM | | | 14218.34 |
The `joined()` and `joined(separator:)` methods are available on `AsyncSequence`s with elements that are `AsyncSequence`s themselves and produce either an `AsyncJoinedSequence` or an `AsyncJoinedBySeparatorSequence`.
As soon as an inner `AsyncSequence` returns `nil` the algorithm continues with iterating the next inner `AsyncSequence`.
The throwing behaviour of `AsyncJoinedSequence` and `AsyncJoinedBySeparatorSequence` is that if any of the inner `AsyncSequence`s throws, then the composed sequence throws on its iteration.
### Naming
The naming follows to current method naming of the standard library's [`joined`](https://developer.apple.com/documentation/swift/array/joined(separator:)-7uber) method.
Prior art in the reactive community often names this method `concat`; however, we think that an alignment with the current method on `Sequence` is better.
### Comparison with other libraries
**ReactiveX** ReactiveX has an [API definition of Concat](https://reactivex.io/documentation/operators/concat.html) as a top level function for concatenating Observables.
**Combine** Combine has an [API definition of append](https://developer.apple.com/documentation/combine/publisher/append(_:)-5yh02) which offers similar functionality but limited to concatenating two individual `Publisher`s.
+55
View File
@@ -0,0 +1,55 @@
# AdjacentPairs
* Proposal: [SAA-0005](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0005-adjacent-pairs.md)
* Author(s): [László Teveli](https://github.com/tevelee)
* Review Manager: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAdjacentPairsSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestAdjacentPairs.swift)]
* Decision Notes:
* Bugs:
## Introduction
The `adjacentPairs()` API serve the purpose of collecting adjacent values. This operation is available for any `AsyncSequence` by calling the `adjacentPairs()` method.
```swift
extension AsyncSequence {
public func adjacentPairs() -> AsyncAdjacentPairsSequence<Self>
}
```
## Detailed Design
The `adjacentPairs()` algorithm produces elements of tuple (size of 2), containing a pair of the original `Element` type.
The interface for this algorithm is available on all `AsyncSequence` types. The returned `AsyncAdjacentPairsSequence` conditionally conforms to `Sendable`.
Its iterator keeps track of the previous element returned in the `next()` function and updates it in every turn.
```swift
for await (first, second) in (1...5).async.adjacentPairs() {
print("First: \(first), Second: \(second)")
}
// First: 1, Second: 2
// First: 2, Second: 3
// First: 3, Second: 4
// First: 4, Second: 5
```
It composes well with the [Dictionary.init(_:uniquingKeysWith:)](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md) API that deals with `AsyncSequence` of tuples.
```swift
Dictionary(uniqueKeysWithValues: url.lines.adjacentPairs())
```
## Alternatives Considered
This functionality is often written as a `zip` of a sequence together with itself, dropping its first element (`zip(source, source.dropFirst())`).
It's such a dominant use-case, the [swift-algorithms](https://github.com/apple/swift-algorithms) package also [introduced](https://github.com/apple/swift-algorithms/pull/119) it to its collection of algorithms.
## Credits/Inspiration
The synchronous counterpart in [swift-algorithms](https://github.com/apple/swift-algorithms/blob/main/Guides/AdjacentPairs.md).
+93
View File
@@ -0,0 +1,93 @@
# Combine Latest
* Proposal: [SAA-0006](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0006-combineLatest.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/CombineLatest/AsyncCombineLatest2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/CombineLatest/AsyncCombineLatest3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestCombineLatest.swift)]
* Decision Notes:
* Bugs:
## Introduction
Similar to the `zip` algorithm there is a need to combine the latest values from multiple input asynchronous sequences. Since `AsyncSequence` augments the concept of sequence with the characteristic of time it means that the composition of elements may not just be pairwise emissions but instead be temporal composition. This means that it is useful to emit a new tuple _when_ a value is produced. The `combineLatest` algorithm provides precisely that.
## Detailed Design
This algorithm combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
```swift
let appleFeed = URL("http://www.example.com/ticker?symbol=AAPL").lines
let nasdaqFeed = URL("http://www.example.com/ticker?symbol=^IXIC").lines
for try await (apple, nasdaq) in combineLatest(appleFeed, nasdaqFeed) {
print("AAPL: \(apple) NASDAQ: \(nasdaq)")
}
```
Given some sample inputs the following combined events can be expected.
| Timestamp | appleFeed | nasdaqFeed | combined output |
| ----------- | --------- | ---------- | ----------------------------- |
| 11:40 AM | 173.91 | | |
| 12:25 AM | | 14236.78 | AAPL: 173.91 NASDAQ: 14236.78 |
| 12:40 AM | | 14218.34 | AAPL: 173.91 NASDAQ: 14218.34 |
| 1:15 PM | 173.00 | | AAPL: 173.00 NASDAQ: 14218.34 |
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func combineLatest<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncCombineLatest2Sequence<Base1, Base2>
public func combineLatest<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncCombineLatest3Sequence<Base1, Base2, Base3>
public struct AsyncCombineLatest2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable, Base2.Element: Sendable {
public typealias Element = (Base1.Element, Base2.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
public struct AsyncCombineLatest3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable, Base3: Sendable
Base1.Element: Sendable, Base2.Element: Sendable, Base3.Element: Sendable {
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The `combineLatest(_:...)` function takes two or more asynchronous sequences as arguments and produces an `AsyncCombineLatestSequence` which is an asynchronous sequence.
Since the bases comprising the `AsyncCombineLatestSequence` must be iterated concurrently to produce the latest value, those sequences must be able to be sent to child tasks. This means that a prerequisite of the bases must be that the base asynchronous sequences, their iterators, and the elements they produce must all be `Sendable`.
If any of the bases terminate before the first element is produced, then the `AsyncCombineLatestSequence` iteration can never be satisfied. So, if a base's iterator returns `nil` at the first iteration, then the `AsyncCombineLatestSequence` iterator immediately returns `nil` to signify a terminal state. In this particular case, any outstanding iteration of other bases will be cancelled. After the first element is produced ,this behavior is different since the latest values can still be satisfied by at least one base. This means that beyond the construction of the first tuple comprised of the returned elements of the bases, the terminal state of the `AsyncCombineLatestSequence` iteration will only be reached when all of the base iterations reach a terminal state.
The throwing behavior of `AsyncCombineLatestSequence` is that if any of the bases throw, then the composed asynchronous sequence throws on its iteration. If at any point (within the first iteration or afterwards), an error is thrown by any base, the other iterations are cancelled and the thrown error is immediately thrown to the consuming iteration.
### Naming
Since the inherent behavior of `combineLatest(_:...)` combines the latest values from multiple streams into a tuple the naming is intended to be quite literal. There are precedent terms of art in other frameworks and libraries (listed in the comparison section). Other naming takes the form of "withLatestFrom". This was disregarded since the "with" prefix is often most associated with the passing of a closure and some sort of contextual concept; `withUnsafePointer` or `withUnsafeContinuation` are prime examples.
### Comparison with other libraries
Combine latest often appears in libraries developed for processing events over time since the event ordering of a concept of "latest" only occurs when asynchrony is involved.
**ReactiveX** ReactiveX has an [API definition of CombineLatest](https://reactivex.io/documentation/operators/combinelatest.html) as a top level function for combining Observables.
**Combine** Combine has an [API definition of combineLatest](https://developer.apple.com/documentation/combine/publisher/combinelatest(_:)/) has an operator style method for combining Publishers.
+80
View File
@@ -0,0 +1,80 @@
# Chain
* Proposal: [SAA-0007](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0007-chain.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChain2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChain3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChain.swift)]
## Introduction
Combining asynchronous sequences can occur in multiple ways. One such way that is common for non asynchronous sequences is iterating a prefix sequence and then iterating a suffix sequence. The asynchronous version is just as useful and common. This algorithm has been dubbed `chain` in the swift algorithms package.
The chain algorithm brings together two or more asynchronous sequences together sequentially where the elements from the resulting asynchronous sequence are comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs.
This operation is available for all `AsyncSequence` types who share the same `Element` type.
```swift
let preamble = [
"// Some header to add as a preamble",
"//",
""
].async
let lines = chain(preamble, URL(fileURLWithPath: "/tmp/Sample.swift").lines)
for try await line in lines {
print(line)
}
```
The above example shows how two `AsyncSequence` types can be chained together. In this case it prepends a preamble to the `lines` content of the file.
## Detailed Design
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func chain<Base1: AsyncSequence, Base2: AsyncSequence>(_ s1: Base1, _ s2: Base2) -> AsyncChain2Sequence<Base1, Base2> where Base1.Element == Base2.Element
public func chain<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ s1: Base1, _ s2: Base2, _ s3: Base3) -> AsyncChain3Sequence<Base1, Base2, Base3>
public struct AsyncChain2Sequence<Base1: AsyncSequence, Base2: AsyncSequence> where Base1.Element == Base2.Element {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChain2Sequence: Sendable where Base1: Sendable, Base2: Sendable { }
extension AsyncChain2Sequence.Iterator: Sendable where Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable { }
public struct AsyncChain3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence> where Base1.Element == Base2.Element, Base1.Element == Base3.Element {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChain3Sequence: Sendable where Base1: Sendable, Base2: Sendable, Base3: Sendable { }
extension AsyncChain3Sequence.Iterator: Sendable where Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable, Base3.AsyncIterator: Sendable { }
```
The `chain(_:...)` function takes two or more sequences as arguments.
The resulting `AsyncChainSequence` type is an asynchronous sequence, with conditional conformance to `Sendable` when the arguments also conform to it.
When any of the asynchronous sequences being chained together come to their end of iteration, the `AsyncChainSequence` iteration proceeds to the next asynchronous sequence. When the last asynchronous sequence reaches the end of iteration, the `AsyncChainSequence` then ends its iteration.
At any point in time, if one of the comprising asynchronous sequences throws an error during iteration, the resulting `AsyncChainSequence` iteration will throw that error and end iteration. The throwing behavior of `AsyncChainSequence` is that it will throw when any of its comprising bases throw, and will not throw when all of its comprising bases do not throw.
### Naming
This function's and type's name match the term of art used in other languages and libraries.
This combinator function is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Chain.md).
+59
View File
@@ -0,0 +1,59 @@
# AsyncBufferedByteIterator
* Proposal: [SAA-0008](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0008-bytes.md)
* Authors: [David Smith](https://github.com/Catfish-Man), [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncBufferedByteIterator.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestBufferedByteIterator.swift)]
## Introduction
Sources of bytes are a common point of asynchrony; reading from files, reading from the network, or other such tasks. Having an easy to use, uniform, and performant utility to make this approachable is key to unlocking highly scalable byte handling. This has proven useful for `FileHandle`, `URL`, and a number of others in Foundation.
This type provides infrastructure for creating `AsyncSequence` types with an `Element` of `UInt8` backed by file descriptors or similar read sources.
```swift
struct AsyncBytes: AsyncSequence {
public typealias Element = UInt8
var handle: ReadableThing
internal init(_ readable: ReadableThing) {
handle = readable
}
public func makeAsyncIterator() -> AsyncBufferedByteIterator {
return AsyncBufferedByteIterator(capacity: 16384) { buffer in
// This runs once every 16384 invocations of next()
return try await handle.read(into: buffer)
}
}
}
```
## Detailed Design
```swift
public struct AsyncBufferedByteIterator: AsyncIteratorProtocol {
public typealias Element = UInt8
public init(
capacity: Int,
readFunction: @Sendable @escaping (UnsafeMutableRawBufferPointer) async throws -> Int
)
public mutating func next() async throws -> UInt8?
}
```
For each invocation of `next`, the iterator will check if a buffer has been filled. If the buffer is filled with some amount of bytes, a fast path is taken to directly return a byte out of that buffer. If the buffer is not filled, the read function is invoked to acquire the next filled buffer, at which point it takes a byte out of that buffer.
If the read function returns `0`, indicating it didn't read any more bytes, the iterator is decided to be finished and no additional invocations to the read function are made.
If the read function throws, the error will be thrown by the iteration. Subsequent invocations to the iterator will then return `nil` without invoking the read function.
If the task is cancelled during the iteration, the iteration will check the cancellation only in passes where the read function is invoked, and will throw a `CancellationError`.
### Naming
This type was named precisely for what it does: it is an asynchronous iterator that buffers bytes.
+48
View File
@@ -0,0 +1,48 @@
# AsyncSyncSequence
* Proposal: [SAA-0009](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0009-async.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Implemented**
* Implementation:
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncSyncSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestLazy.swift)
## Introduction
`AsyncSyncSequence` converts a non-asynchronous sequence into an asynchronous one.
This operation is available for all `Sequence` types.
```swift
let numbers = [1, 2, 3, 4].async
let characters = "abcde".async
```
This transformation can be useful to test operations specifically available on `AsyncSequence` but also is useful
to combine with other `AsyncSequence` types to provide well known sources of data.
The `.async` property returns an `AsyncSyncSequence` that is generic upon the base `Sequence` it was constructed from.
```swift
extension Sequence {
public var async: AsyncSyncSequence<Self> { get }
}
public struct AsyncSyncSequence<Base: Sequence>: AsyncSequence {
...
}
extension AsyncSyncSequence: Sendable where Base: Sendable { }
extension AsyncSyncSequence.Iterator: Sendable where Base.Iterator: Sendable { }
```
### Naming
This property's and type's name match the naming approaches in the Swift standard library. The property is named with a
succinct name in inspiration from `.lazy`, and the type is named in reference to the lazy behavior of the constructed
`AsyncSequence`.
## Effect on API resilience
`AsyncSyncSequence` has a trivial implementation and is marked as `@frozen` and `@inlinable`. This removes the ability of this type and functions to be ABI resilient boundaries at the benefit of being highly optimizable.
+114
View File
@@ -0,0 +1,114 @@
# Buffer
* Proposal: [SAA-0010](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0010-buffer.md)
* Author(s): [Thibault Wittemberg](https://github.com/twittemb)
* Status: **Accepted**
* Implementation: [
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Buffer/AsyncBufferSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestBuffer.swift)
]
* Decision Notes:
* Bugs:
## Introduction
Buffering is a technique that balances supply and demand by temporarily storing elements to even out fluctuations in production and consumption rates. `AsyncStream` facilitates this process by allowing you to control the size of the buffer and the strategy for handling elements that exceed that size. However, this approach may not be suitable for all situations, and it doesn't provide a way to adapt other `AsyncSequence` types to incorporate buffering.
This proposal presents a new type that addresses these more advanced requirements and offers a comprehensive solution for buffering in asynchronous sequences.
## Motivation
As an `AsyncSequence` operates as a pull system, the production of elements is directly tied to the demand expressed by the consumer. The slower the consumer is in requesting elements, the slower the production of these elements will be. This can negatively impact the software that produces the elements, as demonstrated in the following example.
Consider an `AsyncSequence` that reads and returns a line from a file every time a new element is requested. To ensure exclusive access to the file, a lock is maintained while reading. Ideally, the lock should be held for as short a duration as possible to allow other processes to access the file. However, if the consumer is slow in processing received lines or has a fluctuating pace, the lock will be maintained for an extended period, reducing the performance of the system.
To mitigate this issue, a buffer operator can be employed to streamline the consumption of the `AsyncSequence`. This operator allows an internal iteration of the `AsyncSequence` independently from the consumer. Each element would then be made available for consumption by using a queuing mechanism.
By applying the buffer operator to the previous example, the file can be read as efficiently as possible, allowing the lock to be released in a timely manner. This results in improved performance, as the consumer can consume elements at its own pace without negatively affecting the system. The buffer operator ensures that the production of elements is not limited by the pace of the consumer, allowing both the producer and consumer to operate at optimal levels.
## Proposed Solution
We propose to extend `AsyncSequence` with a `buffer()` operator. This operator will return an `AsyncBufferSequence` that wraps the source `AsyncSequence` and handle the buffering mechanism.
This operator will accept an `AsyncBufferSequencePolicy`. The policy will dictate the behaviour in case of a buffer overflow.
As of now we propose 4 different behaviours:
```swift
public struct AsyncBufferSequencePolicy: Sendable {
public static func bounded(_ limit: Int)
public static var unbounded
public static func bufferingLatest(_ limit: Int)
public static func bufferingOldest(_ limit: Int)
}
```
And the public API of `AsyncBufferSequence` will be:
```swift
extension AsyncSequence where Self: Sendable {
public func buffer(
policy: AsyncBufferSequencePolicy
) -> AsyncBufferSequence<Self> {
AsyncBufferSequence<Self>(base: self, policy: policy)
}
}
public struct AsyncBufferSequence<Base: AsyncSequence & Sendable>: AsyncSequence {
public typealias Element = Base.Element
public func makeAsyncIterator() -> Iterator
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
}
extension AsyncBufferSequence: Sendable where Base: Sendable { }
```
## Notes on Sendable
Since all buffering means that the base asynchronous sequence must be iterated independently of the consumption (to resolve the production versus consumption issue) the base `AsyncSequence` needs to be able to be sent across task boundaries (the iterator does not need this requirement).
## Detailed Design
The choice of the buffering policy is made through an enumeration whose values are related to a storage type. To date, two types of storage are implemented:
- `BoundedBufferStorage` backing the `AsyncBufferSequencePolicy.bounded(_:)` policy,
- `UnboundedBufferStorage` backing the `AsyncBufferSequencePolicy.unbounded`, `AsyncBufferSequencePolicy.bufferingNewest(_:)` and `AsyncBufferSequencePolicy.bufferingLatest(_:)` policies.
Both storage types rely on a Mealy state machine stored in a `ManagedCriticalState`. They drive the mutations of the internal buffer, while ensuring that concurrent access to the state are safe.
### BoundedBufferStorage
`BoundedBufferStorage` is instantiated with the upstream `AsyncSequence` and a buffer maximum size. Upon the first call to the `next()` method, a task is spawned to support the iteration over this `AsyncSequence`. The iteration retrieves elements and adds them to an internal buffer as long as the buffer limit has not been reached. Meanwhile, the downstream `AsyncSequence` can access and consume these elements from the buffer. If the rate of consumption is slower than the rate of production, the buffer will eventually become full. In this case, the iteration is temporarily suspended until additional space becomes available.
### UnboundedBufferStorage
`UnboundedBufferStorage` is instantiated with the upstream `AsyncSequence` and a buffering policy. Upon the first call to the `next()` method, a task is spawned to support the iteration over this `AsyncSequence`.
From there the behaviour will depend on the buffering policy.
#### Unbounded
If the policy is `unbounded`, the iteration retrieves elements and adds them to an internal buffer until the upstream `AsyncSequence` finishes or fails. Meanwhile, the downstream `AsyncSequence` can access and consume these elements from the buffer.
#### BufferingLatest
If the policy is `bufferingLatest(_:)`, the iteration retrieves elements and adds them to an internal buffer. Meanwhile, the downstream `AsyncSequence` can access and consume these elements from the buffer. If the rate of consumption is slower than the rate of production, the buffer will eventually become full. In this case the oldest buffered elements will be removed from the buffer and the latest ones will be added.
#### BufferingOldest
If the policy is `bufferingOldest(_:)`, the iteration retrieves elements and adds them to an internal buffer. Meanwhile, the downstream `AsyncSequence` can access and consume these elements from the buffer. If the rate of consumption is slower than the rate of production, the buffer will eventually become full. In this case the latest element will be discarded and never added to the buffer.
### Terminal events and cancellation
Terminal events from the upstream `AsyncSequence`, such as normal completion or failure, are delayed, ensuring that the consumer will receive all the elements before receiving the termination.
If the consuming `Task` is cancelled, so will be the `Task` supporting the iteration of the upstream `AsyncSequence`.
## Alternatives Considered
The buffering mechanism was originally thought to rely on an open implementation of an `AsyncBuffer` protocol, which would be constrained to an `Actor` conformance. It was meant for developers to be able to provide their own implementation of a buffering algorithm. This buffer was designed to control the behavior of elements when pushed and popped, in an isolated manner.
A default implementation was provided to offers a queuing strategy for buffering elements, with options for unbounded, oldest-first, or newest-first buffering.
This implementation was eventually discarded due to the potentially higher cost of calling isolated functions on an `Actor`, compared to using a low-level locking mechanism like the one employed in other operators through the use of the `ManagedCriticalState`.
+319
View File
@@ -0,0 +1,319 @@
# Feature name
* Proposal: [SAA-0011](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0011-interspersed.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Review Manager: [Franz Busch](https://github.com/FranzBusch)
* Status: **Implemented**
* Implementation:
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Interspersed/AsyncInterspersedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestLazy.swift)
## Motivation
A common transformation that is applied to async sequences is to intersperse the elements with
a separator element.
## Proposed solution
We propose to add a new method on `AsyncSequence` that allows to intersperse
a separator between every n emitted element. This proposed API looks like this
```swift
public extension AsyncSequence {
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: The value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
func interspersed(every: Int = 1, with separator: Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
func interspersed(every: Int = 1, with separator: @Sendable @escaping () -> Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
func interspersed(every: Int = 1, with separator: @Sendable @escaping () async -> Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: @Sendable @escaping () throws -> Element) -> AsyncThrowingInterspersedSequence<Self> {
AsyncThrowingInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: @Sendable @escaping () async throws -> Element) -> AsyncThrowingInterspersedSequence<Self> {
AsyncThrowingInterspersedSequence(self, every: every, separator: separator)
}
}
```
## Detailed design
The bulk of the implementation of the new `interspersed` method is inside the new
`AsyncInterspersedSequence` struct. It constructs an iterator to the base async sequence
inside its own iterator. The `AsyncInterspersedSequence.Iterator.next()` is forwarding the demand
to the base iterator.
Below is the implementation of the `AsyncInterspersedSequence`.
```swift
/// An asynchronous sequence that presents the elements of a base asynchronous sequence of
/// elements with a separator between each of those elements.
public struct AsyncInterspersedSequence<Base: AsyncSequence> {
@usableFromInline
internal enum Separator {
case element(Element)
case syncClosure(@Sendable () -> Element)
case asyncClosure(@Sendable () async -> Element)
}
@usableFromInline
internal let base: Base
@usableFromInline
internal let separator: Separator
@usableFromInline
internal let every: Int
@usableFromInline
internal init(_ base: Base, every: Int, separator: Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .element(separator)
self.every = every
}
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () -> Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .syncClosure(separator)
self.every = every
}
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () async -> Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .asyncClosure(separator)
self.every = every
}
}
extension AsyncInterspersedSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInterspersedSequence` asynchronous sequence.
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal enum State {
case start(Element?)
case element(Int)
case separator
case finished
}
@usableFromInline
internal var iterator: Base.AsyncIterator
@usableFromInline
internal let separator: Separator
@usableFromInline
internal let every: Int
@usableFromInline
internal var state = State.start(nil)
@usableFromInline
internal init(_ iterator: Base.AsyncIterator, every: Int, separator: Separator) {
self.iterator = iterator
self.separator = separator
self.every = every
}
public mutating func next() async rethrows -> Base.Element? {
// After the start, the state flips between element and separator. Before
// returning a separator, a check is made for the next element as a
// separator is only returned between two elements. The next element is
// stored to allow it to be returned in the next iteration. However, if
// the checking the next element throws, the separator is emitted before
// rethrowing that error.
switch state {
case var .start(element):
do {
if element == nil {
element = try await self.iterator.next()
}
if let element = element {
if every == 1 {
state = .separator
} else {
state = .element(1)
}
return element
} else {
state = .finished
return nil
}
} catch {
state = .finished
throw error
}
case .separator:
do {
if let element = try await iterator.next() {
state = .start(element)
switch separator {
case let .element(element):
return element
case let .syncClosure(closure):
return closure()
case let .asyncClosure(closure):
return await closure()
}
} else {
state = .finished
return nil
}
} catch {
state = .finished
throw error
}
case let .element(count):
do {
if let element = try await iterator.next() {
let newCount = count + 1
if every == newCount {
state = .separator
} else {
state = .element(newCount)
}
return element
} else {
state = .finished
return nil
}
} catch {
state = .finished
throw error
}
case .finished:
return nil
}
}
}
@inlinable
public func makeAsyncIterator() -> AsyncInterspersedSequence<Base>.Iterator {
Iterator(base.makeAsyncIterator(), every: every, separator: separator)
}
}
```
+86
View File
@@ -0,0 +1,86 @@
# Channel
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Channels/AsyncChannel.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Channels/AsyncThrowingChannel.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChannel.swift)
]
## Introduction
`AsyncStream` introduced a mechanism to send buffered elements from a context that doesn't use Swift concurrency into one that does. That design only addressed a portion of the potential use cases; the missing portion was the back pressure excerpted across two concurrency domains.
## Proposed Solution
To achieve a system that supports back pressure and allows for the communication of more than one value from one task to another we are introducing a new type, the _channel_. The channel will be a reference-type asynchronous sequence with an asynchronous sending capability that awaits the consumption of iteration. Each value sent by the channel will await the consumption of that value by iteration. That awaiting behavior will allow for the affordance of back pressure applied from the consumption site to be transmitted to the production site. This means that the rate of production cannot exceed the rate of consumption, and that the rate of consumption cannot exceed the rate of production. Sending a terminal event to the channel will instantly resume all pending operations for every producers and consumers.
## Detailed Design
Similar to the `AsyncStream` and `AsyncThrowingStream` types, the type for sending elements via back pressure will come in two versions. These two versions will account for the throwing nature or non-throwing nature of the elements being produced.
Each type will have functions to send elements and to send terminal events.
```swift
public final class AsyncChannel<Element: Sendable>: AsyncSequence, Sendable {
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async -> Element?
}
public init(element elementType: Element.Type = Element.self)
public func send(_ element: Element) async
public func finish()
public func makeAsyncIterator() -> Iterator
}
public final class AsyncThrowingChannel<Element: Sendable, Failure: Error>: AsyncSequence, Sendable {
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async throws -> Element?
}
public init(element elementType: Element.Type = Element.self, failure failureType: Failure.Type = Failure.self)
public func send(_ element: Element) async
public func fail(_ error: Error) where Failure == Error
public func finish()
public func makeAsyncIterator() -> Iterator
}
```
Channels are intended to be used as communication types between tasks. Particularly when one task produces values and another task consumes said values. On the one hand, the back pressure applied by `send(_:)` via the suspension/resume ensures that the production of values does not exceed the consumption of values from iteration. This method suspends after enqueuing the event and is resumed when the next call to `next()` on the `Iterator` is made. On the other hand, the call to `finish()` or `fail(_:)` immediately resumes all the pending operations for every producers and consumers. Thus, every suspended `send(_:)` operations instantly resume, so as every suspended `next()` operations by producing a nil value, or by throwing an error, indicating the termination of the iterations. Further calls to `send(_:)` will immediately resume. The calls to `send(:)` and `next()` will immediately resume when their supporting task is cancelled, other operations from other tasks will remain active.
```swift
let channel = AsyncChannel<String>()
Task {
while let resultOfLongCalculation = doLongCalculations() {
await channel.send(resultOfLongCalculation)
}
channel.finish()
}
for await calculationResult in channel {
print(calculationResult)
}
```
The example above uses a task to perform intense calculations; each of which are sent to the other task via the `send(_:)` method. That call to `send(_:)` returns when the next iteration of the channel is invoked.
## Alternatives Considered
The use of the name "subject" was considered, due to its heritage as a name for a sync-to-async adapter type.
It was considered to make `AsyncChannel` and `AsyncThrowingChannel` actors, however due to the cancellation internals it would imply that these types would need to create new tasks to handle cancel events. The advantages of an actor in this particular case did not outweigh the impact of adjusting the implementations to be actors.
## Future Directions
`AsyncChannel` and `AsyncThrowingChannel` are just the prominent members of the channel-like behavior algorithms. It is reasonable to have as its own distinct type a buffering channel that provides more [tuned back pressure per a given buffer of elements](https://forums.swift.org/t/asyncchannel-should-we-allow-to-buffer/60876). These other members of the same category of algorithm should be considered on their own as distinct proposals.
## Credits/Inspiration
`AsyncChannel` and `AsyncThrowingChannel` was heavily inspired from `Subject` but with the key difference that it uses Swift concurrency to apply back pressure.
https://developer.apple.com/documentation/combine/subject/
+356
View File
@@ -0,0 +1,356 @@
# Chunked & Timer
* Proposal: [SAA-0013](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0013-chunk.md)
* Author(s): [Kevin Perry](https://github.com/kperryua), [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [
[By group](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedByGroupSequence.swift),
[On projection](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift),
[Count and signal](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountOrSignalSequence.swift)
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChunk.swift)
]
[
[Timer](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncTimerSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestTimer.swift)
]
* Decision Notes:
* Bugs:
## Introduction
Grouping of values from an asynchronous sequence is often useful for tasks that involve writing those values efficiently or useful to handle specific structured data inputs.
The groupings may be controlled by different ways but one most notable is to control them by regular intervals. Producing elements at regular intervals can be useful for composing with other algorithms. These can range from invoking code at specific times to using those regular intervals as a delimiter of events. There are other cases this exists in APIs however those do not currently interact with Swift concurrency. These existing APIs are ones like `Timer` or `DispatchTimer` but are bound to internal clocks that are not extensible.
## Proposed Solution
Chunking operations can be broken down into a few distinct categories: grouping according to a binary predicate used to determine whether consecutive elements belong to the same group, projecting an element's property to determine the element's chunk membership, by discrete count, by another signal asynchronous sequence which indicates when the chunk should be delimited, or by a combination of count and signal.
To satisfy the specific grouping by inteervals we propose to add a new type; `AsyncTimerSequence` which utilizes the new `Clock`, `Instant` and `Duration` types. This allows the interaction of the timer to custom implementations of types adopting `Clock`.
This asynchronous sequence will produce elements of the clock's `Instant` type after the interval has elapsed. That instant will be the `now` at the time that the sleep has resumed. For each invocation to `next()` the `AsyncTimerSequence.Iterator` will calculate the next deadline to resume and pass that and the tolerance to the clock. If at any point in time the task executing that iteration is cancelled the iteration will return `nil` from the call to `next()`.
## Detailed Design
### Grouping
Group chunks are determined by passing two consecutive elements to a closure which tests whether they are in the same group. When the `AsyncChunkedByGroupSequence` iterator receives the first element from the base sequence, it will immediately be added to a group. When it receives the second item, it tests whether the previous item and the current item belong to the same group. If they are not in the same group, then the iterator emits the first item's group and a new group is created containing the second item. Items declared to be in the same group accumulate until a new group is declared, or the iterator finds the end of the base sequence. When the base sequence terminates, the final group is emitted. If the base sequence throws an error, `AsyncChunkedByGroupSequence` will rethrow that error immediately and discard any current group.
```swift
extension AsyncSequence {
public func chunked<Collected: RangeReplaceableCollection>(
into: Collected.Type,
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, Collected>
where Collected.Element == Element
public func chunked(
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, [Element]>
}
```
Consider an example where an asynchronous sequence emits the following values: `10, 20, 30, 10, 40, 40, 10, 20`. Given the chunked operation to be defined as follows:
```swift
let chunks = numbers.chunked { $0 <= $1 }
for await numberChunk in chunks {
print(numberChunk)
}
```
That snippet will produce the following values:
```swift
[10, 20, 30]
[10, 40, 40]
[10, 20]
```
While `Array` is the default type for chunks, thanks to the overload that takes a `RangeReplaceableCollection` type, the same sample can be chunked into instances of `ContiguousArray`, or any other `RangeReplaceableCollection` instead.
```swift
let chunks = numbers.chunked(into: ContiguousArray.self) { $0 <= $1 }
for await numberChunk in chunks {
print(numberChunk)
}
```
That variant is the funnel method for the main implementation, which passes `[Element].self` in as the parameter.
### Projection
In some scenarios, chunks are determined not by comparing different elements, but by the element itself. This may be the case when the element has some sort of discriminator that can determine the chunk it belongs to. When two consecutive elements have different projections, the current chunk is emitted and a new chunk is created for the new element.
When the `AsyncChunkedOnProjectionSequence`'s iterator receives `nil` from the base sequence, it emits the final chunk. When the base sequence throws an error, the iterator discards the current chunk and rethrows that error.
Similarly to the `chunked(by:)` method this algorithm has an optional specification for the `RangeReplaceableCollection` which is used as the type of each chunk.
```swift
extension AsyncSequence {
public func chunked<Subject : Equatable, Collected: RangeReplaceableCollection>(
into: Collected.Type,
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, Collected>
public func chunked<Subject : Equatable>(
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, [Element]>
}
```
The following example shows how a sequence of names can be chunked together by their first characters.
```swift
let names = URL(fileURLWithPath: "/tmp/names.txt").lines
let groupedNames = names.chunked(on: \.first!)
for try await (firstLetter, names) in groupedNames {
print(firstLetter)
for name in names {
print(" ", name)
}
}
```
A special property of this kind of projection chunking is that when an asynchronous sequence's elements are known to be ordered, the output of the chunking asynchronous sequence is suitable for initializing dictionaries using the `AsyncSequence` initializer for `Dictionary`. This is because the projection can be easily designed to match the sorting characteristics and thereby guarantee that the output matches the pattern of an array of pairs of unique "keys" with the chunks as the "values".
In the example above, if the names are known to be ordered then you can take advantage of the uniqueness of each "first character" projection to initialize a `Dictionary` like so:
```swift
let names = URL(fileURLWithPath: "/tmp/names.txt").lines
let nameDirectory = try await Dictionary(uniqueKeysWithValues: names.chunked(on: \.first!))
```
### Count or Signal
Sometimes chunks are determined not by the elements themselves, but by external factors. This final category enables limiting chunks to a specific size and/or delimiting them by another asynchronous sequence which is referred to as a "signal". This particular chunking family is useful for scenarios where the elements are more efficiently processed as chunks than individual elements, regardless of their values.
This family is broken down into two sub-families of methods: ones that employ a signal plus an optional count (which return an `AsyncChunksOfCountOrSignalSequence`), and the ones that only deal with counts (which return an `AsyncChunksOfCountSequence`). Both sub-families have `Collected` as their element type, or `Array` if unspecified. These sub-families have rethrowing behaviors; if the base `AsyncSequence` can throw then the chunks sequence can also throw. Likewise if the base `AsyncSequence` cannot throw then the chunks sequence also cannot throw.
##### Count only
```swift
extension AsyncSequence {
public func chunks<Collected: RangeReplaceableCollection>(
ofCount count: Int,
into: Collected.Type
) -> AsyncChunksOfCountSequence<Self, Collected>
where Collected.Element == Element
public func chunks(
ofCount count: Int
) -> AsyncChunksOfCountSequence<Self, [Element]>
}
```
If a chunk size limit is specified via an `ofCount` parameter, the sequence will produce chunks of type `Collected` with at most the specified number of elements. When a chunk reaches the given size, the asynchronous sequence will emit it immediately.
For example, an asynchronous sequence of `UInt8` bytes can be chunked into at most 1024-byte `Data` instances like so:
```swift
let packets = bytes.chunks(ofCount: 1024, into: Data.self)
for try await packet in packets {
write(packet)
}
```
##### Signal only
```swift
extension AsyncSequence {
public func chunked<Signal, Collected: RangeReplaceableCollection>(
by signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal>
where Collected.Element == Element
public func chunked<Signal>(
by signal: Signal
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal>
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(
by timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>>
where Collected.Element == Element
public func chunked<C: Clock>(
by timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>>
}
```
If a signal asynchronous sequence is specified, the chunking asynchronous sequence emits chunks whenever the signal emits. The signals element values are ignored. If the chunking asynchronous sequence hasn't accumulated any elements since its previous emission, then no value is emitted in response to the signal.
Since time is a frequent method of signaling desired delineations of chunks, there is a pre-specialized set of overloads that take `AsyncTimerSequence`. These allow shorthand initialization by using `AsyncTimerSequence`'s static member initializers.
As an example, an asynchronous sequence of log messages can be chunked into arrays of logs in four second segments like so:
```swift
let fourSecondsOfLogs = logs.chunked(by: .repeating(every: .seconds(4)))
for await chunk in fourSecondsOfLogs {
send(chunk)
}
```
##### Count or Signal
```swift
extension AsyncSequence {
public func chunks<Signal, Collected: RangeReplaceableCollection>(
ofCount count: Int,
or signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal>
where Collected.Element == Element
public func chunks<Signal>(
ofCount count: Int,
or signal: Signal
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal>
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(
by timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>>
where Collected.Element == Element
public func chunked<C: Clock>(
by timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>>
}
```
If both count and signal are specified, the chunking asynchronous sequence emits chunks whenever *either* the chunk reaches the specified size *or* the signal asynchronous sequence emits. When a signal causes a chunk to be emitted, the accumulated element count is reset back to zero. When an `AsyncTimerSequence` is used as a signal, the timer is started from the moment `next()` is called for the first time on `AsyncChunksOfCountOrSignalSequence`'s iterator, and it emits on a regular cadence from that moment. Note that the scheduling of the timer's emission is unaffected by any chunks emitted based on count.
Like the example above, this code emits up to 1024-byte `Data` instances, but a chunk will also be emitted every second.
```swift
let packets = bytes.chunks(ofCount: 1024 or: .repeating(every: .seconds(1)), into: Data.self)
for try await packet in packets {
write(packet)
}
```
In any configuration of any of the chunking families, when the base asynchronous sequence terminates, one of two things will happen: 1) a partial chunk will be emitted, or 2) no chunk will be emitted (i.e. the iterator received no elements since the emission of the previous chunk). No elements from the base asynchronous sequence are ever discarded, except in the case of a thrown error.
## Interfaces
### Grouping
```swift
public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Collected?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChunkedByGroupSequence: Sendable
where Base: Sendable, Base.Element: Sendable { }
extension AsyncChunkedByGroupSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable, Base.Element: Sendable { }
```
### Projection
```swift
public struct AsyncChunkedOnProjectionSequence<Base: AsyncSequence, Subject: Equatable, Collected: RangeReplaceableCollection>: AsyncSequence where Collected.Element == Base.Element {
public typealias Element = (Subject, Collected)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> (Subject, Collected)?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChunkedOnProjectionSequence: Sendable
where Base: Sendable, Base.Element: Sendable { }
extension AsyncChunkedOnProjectionSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable, Base.Element: Sendable, Subject: Sendable { }
```
### Count
```swift
public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Collected?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChunksOfCountSequence : Sendable where Base : Sendable, Base.Element : Sendable { }
extension AsyncChunksOfCountSequence.Iterator : Sendable where Base.AsyncIterator : Sendable, Base.Element : Sendable { }
```
### Count or Signal
```swift
public struct AsyncChunksOfCountOrSignalSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection, Signal: AsyncSequence>: AsyncSequence, Sendable
where
Collected.Element == Base.Element,
Base: Sendable, Signal: Sendable,
Base.AsyncIterator: Sendable, Signal.AsyncIterator: Sendable,
Base.Element: Sendable, Signal.Element: Sendable {
public typealias Element = Collected
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async rethrows -> Collected?
}
public func makeAsyncIterator() -> Iterator
}
```
### Timer
```swift
public struct AsyncTimerSequence<C: Clock>: AsyncSequence {
public typealias Element = C.Instant
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async -> C.Instant?
}
public init(
interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
)
public func makeAsyncIterator() -> Iterator
}
extension AsyncTimerSequence where C == SuspendingClock {
public static func repeating(every interval: Duration, tolerance: Duration? = nil) -> AsyncTimerSequence<SuspendingClock>
}
extension AsyncTimerSequence: Sendable { }
```
## Alternatives Considered
It was considered to make the chunked element to be an `AsyncSequence` instead of allowing collection into a `RangeReplaceableCollection` however it was determined that the throwing behavior of that would be complex to understand. If that hurdle could be overcome then that might be a future direction/consideration that would be worth exploring.
Variants of `chunked(by:)` (grouping) and `chunked(on:)` (projection) methods could be added that take delimiting `Signal` and `AsyncTimerSequence` inputs similar to `chunked(byCount:or:)`. However, it was decided that such functionality was likely to be underutilized and not worth the complication to the already broad surface area of `chunked` methods.
The naming of this family was considered to be `collect` which is used in APIs like `Combine`. This family of functions has distinct similarity to those APIs.
## Credits/Inspiration
This transformation function is a heavily inspired analog of the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Chunked.md)
https://developer.apple.com/documentation/foundation/timer
https://developer.apple.com/documentation/foundation/timer/timerpublisher
+157
View File
@@ -0,0 +1,157 @@
# Rate Limiting
* Proposal: [SAA-0014](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0014-rate-limits.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation:
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Debounce/AsyncDebounceSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestDebounce.swift)
]
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncThrottleSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestThrottle.swift)
]
* Decision Notes:
* Bugs:
## Introduction
When events can potentially happen faster than the desired consumption rate, there are multiple ways to handle the situation. One approach is to only emit values after a given period of time of inactivity, or "quiescence", has elapsed. This algorithm is commonly referred to as debouncing. A very close relative is an approach to emit values after a given period has elapsed. These emitted values can be reduced from the values encountered during the waiting period. This algorithm is commonly referred to as throttling.
## Proposed Solution
The debounce algorithm produces elements after a particular duration has passed between events. It transacts within a given tolerance applied to a clock. If values are produced by the base `AsyncSequence` during this quiet period, the debounce does not resume its next iterator until the period has elapsed with no values are produced or unless a terminal event is encountered.
The interface for this algorithm is available on all `AsyncSequence` types where the base type, iterator, and element are `Sendable`, since this algorithm will inherently create tasks to manage their timing of events. A shorthand implementation will be offered where the clock is the `ContinuousClock`, which allows for easy construction with `Duration` values.
```swift
extension AsyncSequence {
public func debounce<C: Clock>(
for interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
) -> AsyncDebounceSequence<Self, C>
public func debounce(
for interval: Duration,
tolerance: Duration? = nil
) -> AsyncDebounceSequence<Self, ContinuousClock>
}
```
This all boils down to a terse description of how to transform the asynchronous sequence over time.
```swift
fastEvents.debounce(for: .seconds(1))
```
In this case it transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second with no events to elapse before emitting a value.
The throttle algorithm produces elements such that at least a specific interval has elapsed between them. It transacts by measuring against a specific clock. If values are produced by the base `AsyncSequence` the throttle does not resume its next iterator until the period has elapsed or unless a terminal event is encountered.
The interface for this algorithm is available on all `AsyncSequence` types. Unlike other algorithms like `debounce`, the throttle algorithm does not need to create additional tasks or require any sort of tolerance because the interval is just measured. A shorthand implementation will be offered in conjunction where the clock is the `ContinuousClock`, which allows for easy construction with `Duration` values. An additional shorthand is offered to reduce the values such that it provides a "latest" or "earliest" value, representing the leading or trailing edge of a throttled region of production of events.
```swift
extension AsyncSequence {
public func throttle<C: Clock, Reduced>(
for interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> AsyncThrottleSequence<Self, C, Reduced>
public func throttle<Reduced>(
for interval: Duration,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> AsyncThrottleSequence<Self, ContinuousClock, Reduced>
public func throttle<C: Clock>(
for interval: C.Instant.Duration,
clock: C,
latest: Bool = true
) -> AsyncThrottleSequence<Self, C, Element>
public func throttle(
for interval: Duration,
latest: Bool = true
) -> AsyncThrottleSequence<Self, ContinuousClock, Element>
}
```
This all boils down to a terse description of how to transform the asynchronous sequence over time.
```swift
fastEvents.throttle(for: .seconds(1))
```
In this case, the throttle transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second to elapse before emitting a value.
## Detailed Design
### Debounce
The type that implements the algorithm for debounce emits the same element type as the base that it applies to. It also throws when the base type throws (and likewise does not throw when the base type does not throw).
```swift
public struct AsyncDebounceSequence<Base: AsyncSequence, C: Clock>: Sendable
where Base.Element: Sendable, Base: Sendable {
}
extension AsyncDebounceSequence: AsyncSequence {
public typealias Element = Base.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Base.Element?
}
public func makeAsyncIterator() -> Iterator
}
```
Since the stored types comprising `AsyncDebounceSequence` must be `Sendable`; `AsyncDebounceSequence` is unconditionally always `Sendable`. It is worth noting that the iterators are not required to be Sendable.
### Throttle
The type that implements the algorithm for throttle emits the same element type as the base that it applies to. It also throws when the base type throws (and likewise does not throw when the base type does not throw).
```swift
public struct AsyncThrottleSequence<Base: AsyncSequence, C: Clock, Reduced> {
}
extension AsyncThrottleSequence: AsyncSequence {
public typealias Element = Reduced
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Reduced?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncThrottleSequence: Sendable
where Base: Sendable, Element: Sendable { }
```
The `AsyncThrottleSequence` is conditionally `Sendable` if the base types comprising it are `Sendable`.
The time in which events are measured are from the previous emission if present. If a duration has elapsed between the last emission and the point in time the throttle is measured then that duration is counted as elapsed. The first element is considered not throttled because no interval can be constructed from the start to the first element.
## Alternatives Considered
An alternative form of `debounce` could exist similar to the reductions of `throttle`, where a closure would be invoked for each value being set as the latest, and reducing a new value to produce for the debounce.
It was considered to only provide the "latest" style APIs, however the reduction version grants more flexibility and can act as a funnel to the implementations of `latest`.
## Credits/Inspiration
The naming for debounce comes as a term of art; originally this term was inspired by electronic circuitry. When a physical switch closes a circuit it can easily have a "bouncing" behavior (also called chatter) that is caused by electrical contact resistance and the physical bounce of springs associated with switches. That phenomenon is often addressed with additional circuits to de-bounce (removing the bouncing) by ensuring a certain quiescence occurs.
http://reactivex.io/documentation/operators/debounce.html
https://developer.apple.com/documentation/combine/publishers/debounce/
http://reactivex.io/documentation/operators/sample.html
https://developer.apple.com/documentation/combine/publishers/throttle/
+196
View File
@@ -0,0 +1,196 @@
# Reductions
* Proposal: [SAA-0015](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0015-reductions.md)
* Author(s): [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncExclusiveReductionsSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestReductions.swift)
]
* Decision Notes:
* Bugs:
## Introduction
The family of algorithms for reduce are useful for converting a sequence or asynchronous sequence into a single value, but that can elide important intermediate information. The _reductions_ algorithm is often called "scan", but this name does not convey its heritage to the family of reducing.
There are two strategies that are usable for creating continuous reductions: exclusive reductions and inclusive reductions:
* Exclusive reductions take a value and incorporate values into that initial value. A common example is reductions by appending to an array.
* Inclusive reductions transact only on the values provided. A common example is adding numbers.
There are also common specializations of this algorithm that are worth offering as a shorthand. Particularly removing duplications that is a common task of reducing a sequence. When processing values over time it is definitely possible that the same value may occur in a row. When the distinctness of the presence value is not needed it is useful to consider the values over time that are differing from the last. Particularly this can be expressed as removing duplicate values either in the case as they are directly `Equatable` or by a predicate.
## Proposed Solution
Exclusive reductions come in two variants: transforming by application, or transformation via mutation. This replicates the same interface as `reduce(_:_:)` and `reduce(into:_:)`. Unlike the `reduce` algorithms, the `reductions` algorithm also comes in two flavors: throwing or non throwing transformations.
```swift
extension AsyncSequence {
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async -> Result
) -> AsyncExclusiveReductionsSequence<Self, Result>
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async -> Void
) -> AsyncExclusiveReductionsSequence<Self, Result>
}
extension AsyncSequence {
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async throws -> Result
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result>
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async throws -> Void
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result>
}
```
These APIs can be used to reduce an initial value progressively or reduce into an initial value via mutation. In practice, a common use case for reductions is to mutate a collection by appending values.
```swift
characters.reductions(into: "") { $0.append($1) }
```
If the characters being produced asynchronously are `"a", "b", "c"`, then the iteration of the reductions is `"a", "ab", "abc"`.
Inclusive reductions do not have an initial value and therefore do not need an additional variations beyond the throwing and non throwing flavors.
```swift
extension AsyncSequence {
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async -> Element
) -> AsyncInclusiveReductionsSequence<Self>
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async throws -> Element
) -> AsyncThrowingInclusiveReductionsSequence<Self>
}
```
This is often used for scenarios like a running tally or other similar cases.
```swift
numbers.reductions { $0 + $1 }
```
In the above example, if the numbers are a sequence of `1, 2, 3, 4`, the produced values would be `1, 3, 6, 10`.
The `removeDuplicates()` and `removeDuplicates(by:)` APIs serve this purpose of removing duplicate values that occur. These are special case optimizations in the family of the reductions APIs. These algorithms test against the previous value and if the latest iteration of the base `AsyncSequence` is the same as the last it invokes `next()` again. The resulting `AsyncRemoveDuplicatesSequence` will ensure that no duplicate values occur next to each other. This should not be confused with only emitting unique new values; where each value is tested against a collected set of values.
```swift
extension AsyncSequence where Element: Equatable {
public func removeDuplicates() -> AsyncRemoveDuplicatesSequence<Self>
}
extension AsyncSequence {
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async -> Bool
) -> AsyncRemoveDuplicatesSequence<Self>
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async throws -> Bool
) -> AsyncThrowingRemoveDuplicatesSequence<Self>
}
```
The `removeDuplicates` family comes in three variants. One variant is conditional upon the `Element` type being `Equatable`. This variation is a shorthand for writing `.removeDuplicates { $0 == $1 }`. The next variation is the closure version that allows for custom predicates to be applied. This algorithm allows for the cases where the elements themselves may not be equatable but portions of the element may be compared. Lastly is the variation that allows for comparison when the comparison method may throw.
## Detailed Design
#### Reductions
The exclusive reduction variants come in two distinct cases: non-throwing and throwing. These both have corresponding types to encompass that throwing behavior.
For non-throwing exclusive reductions, the element type of the sequence is the result of the reduction transform. `AsyncExclusiveReductionsSequence` will throw if the base asynchronous sequence throws, and will not throw if the base does not throws.
```swift
public struct AsyncExclusiveReductionsSequence<Base: AsyncSequence, Element> {
}
extension AsyncExclusiveReductionsSequence: AsyncSequence {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncExclusiveReductionsSequence: Sendable
where Base: Sendable, Element: Sendable { }
```
The sendability behavior of `AsyncExclusiveReductionsSequence` is such that when the base, base iterator, and element are `Sendable` then `AsyncExclusiveReductionsSequence` is `Sendable`.
```swift
public struct AsyncThrowingExclusiveReductionsSequence<Base: AsyncSequence, Element> {
}
extension AsyncThrowingExclusiveReductionsSequence: AsyncSequence {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async throws -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncThrowingExclusiveReductionsSequence: Sendable
where Base: Sendable, Element: Sendable { }
```
#### Remove Duplicates
In the cases where the `Element` type is `Equatable` or the non-trowing predicate variant these utilize the type `AsyncRemoveDuplicatesSequence`. The throwing predicate variant uses `AsyncThrowingRemoveDuplicatesSequence`. Both of these types are conditionally `Sendable` when the base, base element, and base iterator are `Sendable`
The `AsyncRemoveDuplicatesSequence` will rethrow if the base asynchronous sequence throws and will not throw if the base asynchronous sequence does not throw.
```swift
public struct AsyncRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator {
Iterator(iterator: base.makeAsyncIterator(), predicate: predicate)
}
}
extension AsyncRemoveDuplicatesSequence: Sendable
where Base: Sendable, Base.Element: Sendable { }
```
The `AsyncThrowingRemoveDuplicatesSequence` will rethrow if the base asynchronous sequence throws and still may throw if the base asynchronous sequence does not throw due to the predicate having the potential of throwing.
```swift
public struct AsyncThrowingRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async throws -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncThrowingRemoveDuplicatesSequence: Sendable
where Base: Sendable, Base.Element: Sendable { }
```
## Alternatives Considered
One alternate name for `reductions` was to name it `scan`; however the naming from the Swift Algorithms package offers considerably more inference to the heritage of what family of functions this algorithm belongs to.
## Credits/Inspiration
This transformation function is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Reductions.md)
@@ -0,0 +1,838 @@
# MultiProducerSingleConsumerAsyncChannel
* Proposal: [SAA-0016](0016-multi-producer-single-consumer-channel.md)
* Authors: [Franz Busch](https://github.com/FranzBusch)
* Status: **Implemented**
## Revision
- 2023/12/18: Migrate proposal from Swift Evolution to Swift Async Algorithms.
- 2023/12/19: Add element size dependent strategy
- 2024/05/19: Rename to multi producer single consumer channel
- 2024/05/28: Add unbounded strategy
- 2025/03/24: Adopt `~Copyable` for correct semantics and better performance.
## Introduction
[SE-0314](https://github.com/apple/swift-evolution/blob/main/proposals/0314-async-stream.md)
introduced new `Async[Throwing]Stream` types which act as root asynchronous
sequences. These two types allow bridging from synchronous callbacks such as
delegates to an asynchronous sequence. This proposal adds a new root primitive
with the goal of modeling asynchronous multi-producer-single-consumer systems.
## Motivation
After using the `AsyncSequence` protocol, the `Async[Throwing]Stream` types, and
the `Async[Throwing]Channel` types extensively over the past years, we learned
that there is a gap in the ecosystem for a type that provides strict
multi-producer-single-consumer guarantees with external backpressure support.
Additionally, any stream/channel like type needs to have a clear definition
about the following behaviors:
1. Backpressure
2. Multi/single consumer support
3. Downstream consumer termination
4. Upstream producer termination
The below sections are providing a detailed explanation of each of those.
### Backpressure
In general, backpressure is the mechanism that prevents a fast producer from
overwhelming a slow consumer. It helps the stability of the overall system by
regulating the flow of data between different components. Additionally, it
allows us to put an upper bound on the resource consumption of a system. In reality,
backpressure is used in almost all networked applications.
In Swift, asynchronous sequences also have the concept of internal backpressure.
This is modeled by the pull-based implementation where a consumer has to call
`next` on the `AsyncIterator`. In this model, there is no way for a consumer to
overwhelm a producer since the producer controls the rate of pulling elements.
However, the internal backpressure of an asynchronous sequence isn't the only
backpressure in play. There is also the source backpressure that is producing
the actual elements. For a backpressured system, it is important that every
component of such a system is aware of the backpressure of its consumer and its
producer.
Let's take a quick look at how our current root asynchronous sequences are handling
this.
`Async[Throwing]Stream` aims to support backpressure by providing a configurable
buffer and returning `Async[Throwing]Stream.Continuation.YieldResult` which
contains the current buffer depth from the `yield()` method. However, only
providing the current buffer depth on `yield()` is not enough to bridge a
backpressured system into an asynchronous sequence since this can only be used
as a "stop" signal, but we are missing a signal to indicate resuming the
production. The only viable backpressure strategy that can be implemented with
the current API is a timed backoff where we stop producing for some period of
time and then speculatively produce again. This is a very inefficient pattern
that produces high latencies and inefficient use of resources.
`Async[Throwing]Channel` is a multi-producer-multi-consumer channel that only
supports asynchronous producers. Additionally, the backpressure strategy is
fixed by a buffer size of 1 element per producer.
We are currently lacking a type that supports a configurable backpressure
strategy and both asynchronous and synchronous producers.
### Multi/single consumer support
The `AsyncSequence` protocol itself makes no assumptions about whether the
implementation supports multiple consumers or not. This allows the creation of
unicast and multicast asynchronous sequences. The difference between a unicast
and multicast asynchronous sequence is if they allow multiple iterators to be
created. `AsyncStream` does support the creation of multiple iterators and it
does handle multiple consumers correctly. On the other hand,
`AsyncThrowingStream` also supports multiple iterators but does `fatalError`
when more than one iterator has to suspend. The original proposal states:
> As with any sequence, iterating over an AsyncStream multiple times, or
creating multiple iterators and iterating over them separately, may produce an
unexpected series of values.
While that statement leaves room for any behavior, we learned that a clear distinction
of behavior for root asynchronous sequences is beneficial; especially when it comes to
how transformation algorithms are applied on top.
### Downstream consumer termination
Downstream consumer termination allows the producer to notify the consumer that
no more values are going to be produced. `Async[Throwing]Stream` does support
this by calling the `finish()` or `finish(throwing:)` methods of the
`Async[Throwing]Stream.Continuation`. However, `Async[Throwing]Stream` does not
handle the case that the `Continuation` may be `deinit`ed before one of the
finish methods is called. This currently leads to async streams that never
terminate.
### Upstream producer termination
Upstream producer termination is the inverse of downstream consumer termination,
where the producer is notified once the consumption has terminated. Currently,
`Async[Throwing]Stream` does expose the `onTermination` property on the
`Continuation`. The `onTermination` closure is invoked once the consumer has
terminated. The consumer can terminate in four separate cases:
1. The asynchronous sequence was `deinit`ed and no iterator was created.
2. The iterator was `deinit`ed and the asynchronous sequence is unicast.
3. The consuming task is canceled.
4. The asynchronous sequence returned `nil` or threw.
`Async[Throwing]Stream` currently invokes `onTermination` in all cases; however,
since `Async[Throwing]Stream` supports multiple consumers (as discussed in the
`Multi/single consumer support` section), a single consumer task being canceled
leads to the termination of all consumers. This is not expected from multicast
asynchronous sequences in general.
## Proposed solution
The above motivation lays out the expected behaviors for any consumer/producer
system and compares them to the behaviors of `Async[Throwing]Stream` and
`Async[Throwing]Channel`.
This section proposes a new type called `MultiProducerSingleConsumerAsyncChannel`
that implements all of the above-mentioned behaviors. Importantly, this proposed
solution is taking advantage of `~Copyable` types to model the
multi-producer-single-consumer behavior. While the current `AsyncSequence`
protocols are not supporting `~Copyable` types, we provide a way to convert the
proposed channel to an asynchronous sequence. This leaves us room to support any
potential future asynchronous streaming protocol that supports `~Copyable`.
### Creating a MultiProducerSingleConsumerAsyncChannel
You can create an `MultiProducerSingleConsumerAsyncChannel` instance using the
`makeChannel(of:backpressureStrategy:)` method. This method returns you the
channel and the source. The source can be used to send new values to the
asynchronous channel. The new API specifically provides a
multi-producer/single-consumer pattern.
```swift
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
// The channel and source can be extracted from the returned type
let channel = consume channelAndSource.channel
let source = consume channelAndSource.source
```
The new proposed APIs offer two different backpressure strategies:
- Watermark: Using a low and high watermark.
- Unbounded: Unbounded buffering of the channel. **Only** use this if the
production is limited through some other means.
The source is used to send values to the channel. It provides different APIs for
synchronous and asynchronous producers. All of the APIs are relaying the
backpressure of the channel. The synchronous multi-step APIs are the foundation
for all other APIs. Below is an example of how it can be used:
```swift
do {
let sendResult = try source.send(contentsOf: sequence)
switch sendResult {
case .produceMore:
// Trigger more production in the underlying system
case .enqueueCallback(let callbackHandle):
// There are enough values in the channel already. We need to enqueue
// a callback to get notified when we should produce more.
callbackHandle.enqueueCallback(onProduceMore: { result in
switch result {
case .success:
// Trigger more production in the underlying system
case .failure(let error):
// Terminate the underlying producer
}
})
}
} catch {
// `send(contentsOf:)` throws if the channel already terminated
}
```
The above API offers the most control and highest performance when bridging a
synchronous producer to a `MultiProducerSingleConsumerAsyncChannel`. First, you have
to send values using the `send(contentsOf:)` which returns a `SendResult`. The
result either indicates that more values should be produced or that a callback
should be enqueued by calling the `enqueueCallback(onProduceMore:)` method.
This callback is invoked once the backpressure strategy
decides that more values should be produced. This API aims to offer the most
flexibility with the greatest performance. The callback only has to be allocated
in the case where the producer needs to pause production.
Additionally, the above API is the building block for some higher-level and
easier-to-use APIs to send values to the channel. Below is an
example of the two higher-level APIs.
```swift
// Writing new values and providing a callback when to produce more
try source.send(contentsOf: sequence, onProduceMore: { result in
switch result {
case .success:
// Trigger more production
case .failure(let error):
// Terminate the underlying producer
}
})
// This method suspends until more values should be produced
try await source.send(contentsOf: sequence)
```
With the above APIs, we should be able to effectively bridge any system into a
`MultiProducerSingleConsumerAsyncChannel` regardless of whether the system is callback-based,
blocking, or asynchronous.
### Multi producer
To support multiple producers, the source offers a `copy` method to produce a new
source. The source is returned `sending`, so it is in a disconnected isolation
region from the original source, allowing it to be passed into a different isolation
region to concurrently produce elements.
```swift
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 5)
)
var channel = consume channelAndSource.channel
var source1 = consume channelAndSource.source
var source2 = source1.makeAdditionalSource()
group.addTask {
try await source1.send(1)
}
group.addTask() {
try await source2.send(2)
}
print(await channel.next()) // Prints either 1 or 2 depending on which child task runs first
print(await channel.next()) // Prints either 1 or 2 depending on which child task runs first
```
### Downstream consumer termination
> When reading the next two examples of termination behavior, keep in mind
that the newly proposed APIs are providing a strict single consumer channel.
Calling `finish()` terminates the downstream consumer. Below is an example of
this:
```swift
// Termination through calling finish
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
try await source.send(1)
source.finish()
print(await channel.next()) // Prints Optional(1)
print(await channel.next()) // Prints nil
```
If the channel has a failure type it can also be finished with an error.
```swift
// Termination through calling finish
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
throwing: SomeError.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
try await source.send(1)
source.finish(throwing: SomeError)
print(try await channel.next()) // Prints Optional(1)
print(try await channel.next()) // Throws SomeError
```
The other way to terminate the consumer is by deiniting the source. This has the
same effect as calling `finish()`. Since the source is a `~Copyable` type, this
will happen automatically when the source is last used or explicitly consumed.
```swift
// Termination through deiniting the source
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
try await source.send(1)
_ = consume source // Explicitly consume the source
print(await channel.next()) // Prints Optional(1)
print(await channel.next()) // Prints nil
```
### Upstream producer termination
The producer will get notified about termination through the `onTerminate`
callback. Termination of the producer happens in the following scenarios:
```swift
// Termination through task cancellation
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
source.setOnTerminationCallback { print("Terminated") }
let task = Task {
await channel.next()
}
task.cancel() // Prints Terminated
```
```swift
// Termination through deiniting the channel
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
source.setOnTerminationCallback { print("Terminated") }
_ = consume channel // Prints Terminated
```
```swift
// Termination through finishing the source and consuming the last element
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
source.setOnTerminationCallback { print("Terminated") }
_ = try await source.send(1)
source.finish()
print(await channel.next()) // Prints Optional(1)
await channel.next() // Prints Terminated
```
```swift
// Termination through deiniting the last source and consuming the last element
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source1 = consume channelAndSource.source
var source2 = source1.makeAdditionalSource()
source1.setOnTerminationCallback { print("Terminated") }
_ = try await source1.send(1)
_ = consume source1
_ = try await source2.send(2)
print(await channel.next()) // Prints Optional(1)
print(await channel.next()) // Prints Optional(2)
_ = consume source2
await channel.next() // Prints Terminated
```
Similar to the downstream consumer termination, trying to send more elements after the
producer has been terminated will result in an error thrown from the send methods.
## Detailed design
```swift
#if compiler(>=6.1)
/// An error that is thrown from the various `send` methods of the
/// ``MultiProducerSingleConsumerAsyncChannel/Source``.
///
/// This error is thrown when the channel is already finished when
/// trying to send new elements to the source.
public struct MultiProducerSingleConsumerAsyncChannelAlreadyFinishedError: Error { }
/// A multi-producer single-consumer channel.
///
/// The ``MultiProducerSingleConsumerAsyncChannel`` provides a ``MultiProducerSingleConsumerAsyncChannel/Source`` to
/// send values to the channel. The channel supports different back pressure strategies to control the
/// buffering and demand. The channel will buffer values until its backpressure strategy decides that the
/// producer have to wait.
///
/// This channel is also suitable for the single-producer single-consumer use-case
///
/// ## Using a MultiProducerSingleConsumerAsyncChannel
///
/// To use a ``MultiProducerSingleConsumerAsyncChannel`` you have to create a new channel with its source first by calling
/// the ``MultiProducerSingleConsumerAsyncChannel/makeChannel(of:throwing:BackpressureStrategy:)`` method.
/// Afterwards, you can pass the source to the producer and the channel to the consumer.
///
/// ```
/// let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
/// of: Int.self,
/// backpressureStrategy: .watermark(low: 2, high: 4)
/// )
///
/// // The channel and source can be extracted from the returned type
/// let channel = consume channelAndSource.channel
/// let source = consume channelAndSource.source
/// ```
///
/// ### Asynchronous producing
///
/// Values can be send to the source from asynchronous contexts using ``MultiProducerSingleConsumerAsyncChannel/Source/send(_:)-8eo96``
/// and ``MultiProducerSingleConsumerAsyncChannel/Source/send(contentsOf:)``. Backpressure results in calls
/// to the `send` methods to be suspended. Once more elements should be produced the `send` methods will be resumed.
///
/// ```
/// try await withThrowingTaskGroup(of: Void.self) { group in
/// group.addTask {
/// try await source.send(1)
/// try await source.send(2)
/// try await source.send(3)
/// }
///
/// for await element in channel {
/// print(element)
/// }
/// }
/// ```
///
/// ### Synchronous produceing
///
/// Values can also be send to the source from synchronous context. Backpressure is also exposed on the synchronous contexts; however,
/// it is up to the caller to decide how to properly translate the backpressure to underlying producer e.g. by blocking the thread.
///
/// ```swift
/// do {
/// let sendResult = try source.send(contentsOf: sequence)
///
/// switch sendResult {
/// case .produceMore:
/// // Trigger more production in the underlying system
///
/// case .enqueueCallback(let callbackHandle):
/// // There are enough values in the channel already. We need to enqueue
/// // a callback to get notified when we should produce more.
/// callbackHandle.enqueueCallback(onProduceMore: { result in
/// switch result {
/// case .success:
/// // Trigger more production in the underlying system
/// case .failure(let error):
/// // Terminate the underlying producer
/// }
/// })
/// }
/// } catch {
/// // `send(contentsOf:)` throws if the channel already terminated
/// }
/// ```
///
/// ### Multiple producers
///
/// To support multiple producers the source offers a ``Source/makeAdditionalSource()`` method to produce a new source.
///
/// ### Terminating the production of values
///
/// The consumer can be terminated through multiple ways:
/// - Calling ``Source/finish(throwing:)``.
/// - Deiniting all sources.
///
/// In both cases, if there are still elements buffered by the channel, then the consumer will receive
/// all buffered elements. Afterwards it will be terminated.
///
/// ### Observing termination of the consumer
///
/// When the consumer stops consumption by either deiniting the channel or the task calling ``next(isolation:)``
/// getting cancelled, the source will get notified about the termination if a termination callback has been set
/// before by calling ``Source/setOnTerminationCallback(_:)``.
@available(macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, *)
public struct MultiProducerSingleConsumerAsyncChannel<Element, Failure: Error>: ~Copyable {
/// A struct containing the initialized channel and source.
///
/// This struct can be deconstructed by consuming the individual
/// components from it.
///
/// ```swift
/// let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
/// of: Int.self,
/// backpressureStrategy: .watermark(low: 5, high: 10)
/// )
/// var channel = consume channelAndSource.channel
/// var source = consume channelAndSource.source
/// ```
@frozen
public struct ChannelAndStream : ~Copyable {
/// The channel.
public var channel: MultiProducerSingleConsumerAsyncChannel
/// The source.
public var source: Source
}
/// Initializes a new ``MultiProducerSingleConsumerAsyncChannel`` and an ``MultiProducerSingleConsumerAsyncChannel/Source``.
///
/// - Parameters:
/// - elementType: The element type of the channel.
/// - failureType: The failure type of the channel.
/// - backpressureStrategy: The backpressure strategy that the channel should use.
/// - Returns: A tuple containing the channel and its source. The source should be passed to the
/// producer while the channel should be passed to the consumer.
public static func makeChannel(
of elementType: Element.Type = Element.self,
throwing failureType: Failure.Type = Never.self,
backpressureStrategy: Source.BackpressureStrategy
) -> ChannelAndStream
/// Returns the next element.
///
/// If this method returns `nil` it indicates that no further values can ever
/// be returned. The channel automatically closes when all sources have been deinited.
///
/// If there are no elements and the channel has not been finished yet, this method will
/// suspend until an element is send to the channel.
///
/// If the task calling this method is cancelled this method will return `nil`.
///
/// - Parameter isolation: The callers isolation.
/// - Returns: The next buffered element.
public func next(isolation: isolated (any Actor)? = #isolation) async throws(Failure) -> Element?
}
@available(macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, *)
extension MultiProducerSingleConsumerAsyncChannel {
/// A struct to send values to the channel.
///
/// Use this source to provide elements to the channel by calling one of the `send` methods.
public struct Source: ~Copyable, Sendable {
/// A struct representing the backpressure of the channel.
public struct BackpressureStrategy: Sendable {
/// A backpressure strategy using a high and low watermark to suspend and resume production respectively.
///
/// - Parameters:
/// - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
/// - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
public static func watermark(low: Int, high: Int) -> BackpressureStrategy
/// A backpressure strategy using a high and low watermark to suspend and resume production respectively.
///
/// - Parameters:
/// - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
/// - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
/// - waterLevelForElement: A closure used to compute the contribution of each buffered element to the current water level.
///
/// - Note, `waterLevelForElement` will be called on each element when it is written into the source and when
/// it is consumed from the channel, so it is recommended to provide a function that runs in constant time.
public static func watermark(low: Int, high: Int, waterLevelForElement: @escaping @Sendable (borrowing Element) -> Int) -> BackpressureStrategy
/// An unbounded backpressure strategy.
///
/// - Important: Only use this strategy if the production of elements is limited through some other mean. Otherwise
/// an unbounded backpressure strategy can result in infinite memory usage and cause
/// your process to run out of memory.
public static func unbounded() -> BackpressureStrategy
}
/// A type that indicates the result of sending elements to the source.
public enum SendResult: ~Copyable, Sendable {
/// A handle that is returned when the channel's backpressure strategy indicated that production should
/// be suspended. Use this handle to enqueue a callback by calling the ``CallbackHandle/enqueueCallback(onProduceMore:)`` method.
///
/// - Important: ``CallbackHandle/enqueueCallback(onProduceMore:)`` and ``CallbackHandle/cancelCallback()`` must
/// only be called once.
public struct CallbackHandle: Sendable, Hashable {
/// Enqueues a callback that will be invoked once more elements should be produced.
///
/// - Important: Calling enqueue more than once is **not allowed**.
///
/// - Parameters:
/// - onProduceMore: The callback which gets invoked once more elements should be produced.
@inlinable
public mutating func enqueueCallback(
onProduceMore: sending @escaping (Result<Void, Error>) -> Void
)
/// Cancel an enqueued callback.
///
/// - Note: This methods supports being called before ``enqueueCallback(onProduceMore:)`` is called.
///
/// - Important: Calling enqueue more than once is **not allowed**.
@inlinable
public mutating func cancelCallback()
}
/// Indicates that more elements should be produced and send to the source.
case produceMore
/// Indicates that a callback should be enqueued.
case enqueueCallback(CallbackHandle)
}
/// A callback to invoke when the channel finished.
///
/// This is called after the last element has been consumed by the channel.
public func setOnTerminationCallback(_ callback: @escaping @Sendable () -> Void)
/// Creates a new source which can be used to send elements to the channel concurrently.
///
/// The channel will only automatically be finished if all existing sources have been deinited.
///
/// - Returns: A new source for sending elements to the channel.
public mutating func makeAdditionalSource() -> Source
/// Sends new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// - Parameter sequence: The elements to send to the channel.
/// - Returns: The result that indicates if more elements should be produced at this time.
public mutating func send<S>(
contentsOf sequence: consuming sending S
) throws -> SendResult where Element == S.Element, S: Sequence
/// Send the element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// - Parameter element: The element to send to the channel.
/// - Returns: The result that indicates if more elements should be produced at this time.
public mutating func send(_ element: sending consuming Element) throws -> SendResult
/// Send new elements to the channel and provide a callback which will be invoked once more elements should be produced.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then `onProduceMore` will be invoked with
/// a `Result.failure`.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
/// - onProduceMore: The callback which gets invoked once more elements should be produced. This callback might be
/// invoked during the call to ``send(contentsOf:onProduceMore:)``.
public mutating func send<S>(
contentsOf sequence: consuming sending S,
onProduceMore: @escaping @Sendable (Result<Void, Error>) -> Void
) where Element == S.Element, S: Sequence
/// Sends the element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then `onProduceMore` will be invoked with
/// a `Result.failure`.
///
/// - Parameters:
/// - element: The element to send to the channel.
/// - onProduceMore: The callback which gets invoked once more elements should be produced. This callback might be
/// invoked during the call to ``send(_:onProduceMore:)``.
public mutating func send(
_ element: consuming sending Element,
onProduceMore: @escaping @Sendable (Result<Void, Error>
) -> Void)
/// Send new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
public mutating func send<S>(
contentsOf sequence: consuming sending S
) async throws where Element == S.Element, S: Sequence
/// Send new element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - element: The element to send to the channel.
public mutating func send(_ element: consuming sending Element) async throws
/// Send the elements of the asynchronous sequence to the channel.
///
/// This method returns once the provided asynchronous sequence or the channel finished.
///
/// - Important: This method does not finish the source if consuming the upstream sequence terminated.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
public mutating func send<S>(
contentsOf sequence: consuming sending S
) async throws where Element: Sendable, Element == S.Element, S: Sendable, S: AsyncSequence
/// Indicates that the production terminated.
///
/// After all buffered elements are consumed the subsequent call to ``MultiProducerSingleConsumerAsyncChannel/next(isolation:)`` will return
/// `nil` or throw an error.
///
/// Calling this function more than once has no effect. After calling finish, the channel enters a terminal state and doesn't accept
/// new elements.
///
/// - Parameters:
/// - error: The error to throw, or `nil`, to finish normally.
public consuming func finish(throwing error: Failure? = nil)
}
}
@available(macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, *)
extension MultiProducerSingleConsumerAsyncChannel {
/// Converts the channel to an asynchronous sequence for consumption.
///
/// - Important: The returned asynchronous sequence only supports a single iterator to be created and
/// will fatal error at runtime on subsequent calls to `makeAsyncIterator`.
public consuming func asyncSequence() -> some (AsyncSequence<Element, Failure> & Sendable)
}
```
## Comparison to other root asynchronous primitives
### swift-async-algorithm: AsyncChannel
The `AsyncChannel` is a multi-consumer/multi-producer root asynchronous sequence
which can be used to communicate between two tasks. It only offers asynchronous
production APIs and has an effective buffer of one per producer. This means that
any producer will be suspended until its value has been consumed. `AsyncChannel`
can handle multiple consumers and resumes them in FIFO order.
### swift-nio: NIOAsyncSequenceProducer
The NIO team has created their own root asynchronous sequence with the goal to
provide a high-performance sequence that can be used to bridge a NIO `Channel`
inbound stream into Concurrency. The `NIOAsyncSequenceProducer` is a highly
generic and fully inlinable type and quite unwieldy to use. This proposal is
heavily inspired by the learnings from this type but tries to create a more
flexible and easier-to-use API that fits into the standard library.
## Future directions
### Adaptive backpressure strategy
The high/low watermark strategy is common in networking code; however, there are
other strategies such as an adaptive strategy that we could offer in the future.
An adaptive strategy regulates the backpressure based on the rate of
consumption and production. With the proposed new APIs, we can easily add further
strategies.
### Support `~Copyable` elements
In the future, we can extend the channel to support `~Copyable` elements. We
only need an underlying buffer primitive that can hold `~Copyable` types, and the
continuations need to support `~Copyable` elements as well. By making the
channel not directly conform to `AsyncSequence`, we can support this down the
road.
## Alternatives considered
### Provide an `onTermination` callback to the factory method
During development of the new APIs, I first tried to provide the `onTermination`
callback in the `makeChannel` method. However, that showed significant usability
problems in scenarios where one wants to store the source in a type and
reference `self` in the `onTermination` closure at the same time; hence, I kept
the current pattern of setting the `onTermination` closure on the source.
### Provide a `onConsumerCancellation` callback
During the pitch phase, it was raised that we should provide a
`onConsumerCancellation` callback which gets invoked once the asynchronous
channel notices that the consuming task got cancelled. This callback could be
used to customize how cancellation is handled by the channel, e.g. one could
imagine writing a few more elements to the channel before finishing it. Right now,
the channel immediately returns `nil` or throws a `CancellationError` when it
notices cancellation. This proposal decided not to provide this customization
because it opens up the possibility that asynchronous channels are not terminating
when implemented incorrectly. Additionally, asynchronous sequences are not the
only place where task cancellation leads to an immediate error being thrown i.e.
`Task.sleep()` does the same. Hence, the value of the asynchronous not
terminating immediately brings little value when the next call in the iterating
task might throw. However, the implementation is flexible enough to add this in
the future, and we can just default it to the current behaviour.
### Create a custom type for the `Result` of the `onProduceMore` callback
The `onProducerMore` callback takes a `Result<Void, Error>` which is used to
indicate if the producer should produce more or if the asynchronous channel
finished. We could introduce a new type for this, but the proposal decided
against it since it effectively is a result type.
### Use an initializer instead of factory methods
Instead of providing a `makeChannel` factory method, we could use an initializer
approach that takes a closure which gets the `Source` passed into. A similar API
has been offered with the `Continuation` based approach and
[SE-0388](https://github.com/apple/swift-evolution/blob/main/proposals/0388-async-stream-factory.md)
introduced new factory methods to solve some of the usability ergonomics with
the initializer based APIs.
### Provide the type on older compilers
To achieve maximum performance, the implementation is using `~Copyable` extensively.
On Swift versions before 6.1, there is a https://github.com/swiftlang/swift/issues/78048 when using; hence, this type
is only usable with Swift 6.1 and later compilers.
## Acknowledgements
- [Johannes Weiss](https://github.com/weissi) - For making me aware how
important this problem is and providing great ideas on how to shape the API.
- [Philippe Hausler](https://github.com/phausler) - For helping me designing the
APIs and continuously providing feedback
- [George Barnett](https://github.com/glbrntt) - For providing extensive code
reviews and testing the implementation.
- [Si Beaumont](https://github.com/simonjbeaumont) - For implementing the element size dependent strategy
+197
View File
@@ -0,0 +1,197 @@
# Share
* Proposal: [SAA-0015](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0016-share.md)
* Author(s): [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncShareSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestShare.swift)
]
* Decision Notes:
* Bugs:
## Introduction
Many of the AsyncSequence adopting types only permit a one singular consumption. However there are many times that the same produced values are useful in more than one place. Out of that mechanism there are a few approaches to share, distribute, and broadcast those values. This proposal will focus on one concept; sharing. Sharing is where each consumption independently can make forward progress and get the same values but do not replay from the beginning of time.
## Motivation
There are many potential usages for the sharing concept of AsyncSequences.
One such example is the case where a source of data as an asynchronous sequence needs to be consumed by updating UI, logging, and additionally a network connection. This particular case does not matter on which uses but instead that those uses are independent of each other. It would not be expected for networking to block or delay the updates to UI, nor should logging. This example case also illustrates that the isolation of each side might be different and that some of the sides may not tolerate coalescing or dropping values.
There are many other use cases that have been requested for this family of algorithms. Since the release of AsyncAlgorithms it has perhaps been the most popularly requested set of behaviors as additions to the package.
## Proposed solution
AsyncAlgorithms will introduce a new extension function on AsyncSequence that will provide a shareable asynchronous sequence that will produce the same values upon iteration from multiple instances of it's AsyncIterator. Those iterations can take place in multiple isolations.
When values from a differing isolation cannot be coalesced, the two options available are either awaiting (an exertion of back-pressure across the sequences) or buffering (an internal back-pressure to a buffer). Replaying the values from the beginning of the creation of the sequence is a distinctly different behavior that should be considered a different use case. This then leaves the behavioral characteristic of this particular operation of share as; sharing a buffer of values started from the initialization of a new iteration of the sequence. Control over that buffer should then have options to determine the behavior, similar to how AsyncStream allows that control. It should have options to be unbounded, buffering the oldest count of elements, or buffering the newest count of elements.
It is critical to identify that this is one algorithm in the family of algorithms for sharing values. It should not attempt to solve all behavioral requirements but instead serve a common set of them that make cohesive sense together. This proposal is not mutually exclusive to the other algorithms in the sharing family.
## Detailed design
A new extension will be added to return a `Sendable` `AsyncSequence`. This extension will take a buffering policy to identify how the buffer will be handled when iterations do not consume at the same rate.
The `Sendable` annotation identifies to the developer that this sequence can be shared and stored in an existental `any`.
```swift
extension AsyncSequence where Element: Sendable {
public func share(
bufferingPolicy: AsyncBufferSequencePolicy = .bounded(1)
) -> some AsyncSequence<Element, Failure> & Sendable
}
```
The buffer internally to the share algorithm will only extend back to the furthest element available but there will only be a singular buffer shared across all iterators. This ensures that with the application of the buffering policy the storage size is as minimal as possible while still allowing all iterations to avoid dropping values and keeping the memory usage in check. The signature reuses the existing `AsyncBufferSequencePolicy` type to specify the behavior around buffering either responding to how it should limit emitting to the buffer or what should happen when the buffer is exceeded.
## Runtime Behavior
The runtime behaviors fall into a few categories; ordering, iteration isolation, cancellation, and lifetimes. To understand the behaviors there are a terms useful to define. Each creation of the AsyncIterator of the sequence and invocation of next will be referred to a side of the share iteration. The back pressure to the system to fetch a new element or termination is referred to as demand. The limit which is the pending gate for awaiting until the buffer has been serviced used for the `AsyncBufferSequencePolicy.bounded(_ : Int)` policy. The last special definition is that of the extent which is specifically in this case the lifetime of the asynchronous sequence itself.
When the underlying type backing the share algorithm is constructed a new extent is created; this is used for tracking the reference lifetime under the hood and is used to both house the iteration but also to identify the point at which no more sides can be constructed. When no more sides can be constructed and no sides are left to iterate then the backing iteration is canceled. This prevents any un-referenced task backing the iteration to not be leaked by the algorith itself.
That construction then creates an initial shared state and buffer. No task is started initially; it is only upon the first demand that the task backing the iteration is started; this means on the first call to next a task is spun up servicing all potential sides. The order of which the sides are serviced is not specified and cannot be relied upon, however the order of delivery within a side is always guarenteed to be ordered. The singular task servicing the iteration will be the only place holding any sort of iterator from the base `AsyncSequence`; so that iterator is isolated and not sent from one isolation to another. That iteration first awaits any limit availability and then awaits for a demand given by a side. After-which it then awaits an element or terminal event from the iterator and enqueues the elements to the buffer.
The buffer itself is only held in one location, each side however has a cursor index into that buffer and when values are consumed it adjusts the indexes accordingly; leaving the buffer usage only as big as the largest deficit. This means that new sides that are started post initial start up will not have a "replay" effect; that is a similar but distinct algorithm and is not addressed by this proposal. Any buffer size sensitive systems that wish to adjust behavior should be aware that specifying a policy is a suggested step. However in common usage similar to other such systems servicing desktop and mobile applications the common behavior is often unbounded. Alternatively desktop or mobile applications will often want `.bounded(1)` since that enforces the slowest consumption to drive the forward progress at most 1 buffered element. All of the use cases have a reasonable default of `.bounded(1)`; mobile, deskop, and server side uses. Leaving this as the default parameter keeps the progressive disclosure of the beahviors - such that the easiest thing to write is correct for all uses, and then more advanced control can be adjusted by passing in a specific policy. This default argument diverges slightly from AsyncStream, but follows a similar behavior to that of Combine's `share`.
As previously stated, the isolation of the iteration of the upstream/base AsyncSequence is to a detached task, this ensures that individual sides can have independent cancellation. Those cancellations will have the effect of remvoing that side from the shared iteration and cleaning up accordingly (including adjusting the trimming of the internal buffer).
Representing concurrent access is difficult to express all potential examples but there are a few cases included with this proposal to illustrate some of the behaviors. If a more comprehensive behavioral analysis is needed, it is strongly suggested to try out the pending pull request to identify how specific behaviors work. Please keep in mind that the odering between tasks is not specified, only the order within one side of iteration.
Practically this all means that a given iteration may be "behind" another and can eventually catch up (provided it is within the buffer limit).
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .unbounded)
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
This example will print a possible ordering of the following:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 3
Task 2 4
Task 1 1
Task 1 2
Task 1 3
Task 1 4
```
The order of the interleaving of the prints are not guaranteed; however the order of the elements per iteration is. Likewise in this buffering case it is guaranteed that all values are represented in the output.
If the creation were instead altered to the following:
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingLatest(2))
```
The output would print the possible ordering of:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 3
Task 1 4
```
Some values are dropped due to the buffering policy, but eventually they reach consistency. Which similarly works for the following:
```
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingOldest(2))
```
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 1
Task 1 2
```
However in this particular case the newest values are the dropped elements.
The `.bounded(N)` policy enforces consumption to prevent any side from being beyond a given amount away from other sides' consumption.
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bounded(1))
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
Will have a potential ordering output of:
```
Task 2 0
Task 2 1
Task 1 0
Task 1 1
Task 2 2
Task 1 2
Task 1 3
Task 1 4
Task 2 3
Task 2 4
```
In that example output Task 2 can get element 0 and 1 but must await until task 1 has caught up to the specified buffering. This limit means that no additional iteration (and no values are then dropped) is made until the buffer count is below the specified value.
## Effect on API resilience
This is an additive API and no existing systems are changed, however it will introduce a few new types that will need to be maintained as ABI interfaces. Since the intent of this is to provide a mechanism to store AsyncSequences to a shared context the type must be exposed as ABI (for type sizing).
## Alternatives considered
It has been considered that the buffering policy would be nested inside the `AsyncShareSequence` type. However since this seems to be something that will be useful for other types it makes sense to use an existing type from a top level type. However if it is determined that a general form of a buffering policy would require additional behaviors this might be a debatable placement to move back to an interior type similar to AsyncStream.
+49
View File
@@ -0,0 +1,49 @@
# Map Error
* Proposal: [SAA-0017](0017-map-error.md)
* Authors: [Clive Liu](https://github.com/clive819) [Philippe Hausler](https://github.com/phausler)
* Review Manager: TBD
* Status: **Awaiting review**
## Introduction
Asynchronous sequences often participate in carefully crafted systems of algorithms spliced together. Some of those algorithms require that both the element type and the failure type are the same as each-other. In order to line those types up for the element we have the map algorithm today, however there are other times at which when using more specific APIs that transforming the Failure type is needed.
## Motivation
The `mapError` function empowers developers to elegantly transform errors within asynchronous sequences, enhancing code readability and maintainability.
Building failure-type-safe versions of zip or other algorithms will need to require that the associated Failure types are the same. Having an effecient and easy to use transformation routine to adjust the failure-types is then key to delivering or interfacing with those failure-type-safe algorithms.
## Proposed solution
A new extension and type will be added to transform the failure-types of AsyncSequences.
## Detailed design
The method will be applied to all AsyncSequences via an extension with the function name of `mapError`. This is spiritually related to the `mapError` method on `Result` and similar in functionality to other frameworks' methods of the similar naming. This will not return an opaque result since the type needs to be refined for `Sendable`; in that the `AsyncMapERrorSequence` is only `Sendable` when the base `AsyncSequence` is `Sendable`.
```swift
extension AsyncSequence {
public func mapError<MappedFailure: Error>(_ transform: @Sendable @escaping (Failure) async -> MappedFailure) -> AsyncMapErrorSequence<Element, MappedFailure>
}
public struct AsyncMapErrorSequence<Base: AsyncSequence, TransformedFailure: Error>: AsyncSequence { }
extension AsyncMapErrorSequence: Sendable where Base: Sendable { }
@available(*, unavailable)
extension AsyncMapErrorSequence.Iterator: Sendable {}
```
## Effect on API resilience
This cannot be back-deployed to 1.0 since it has a base requirement for the associated `Failure` and requires typed throws.
## Naming
The naming follows to current method naming of the Combine [mapError](https://developer.apple.com/documentation/combine/publisher/maperror(_:)) method and similarly the name of the method on `Result`
## Alternatives considered
It was initially considered that the return type would be opaque, however the only way to refine that as Sendable would be to have a disfavored overload; this ended up creating more ambiguity than it seemed worth.
+200
View File
@@ -0,0 +1,200 @@
# FlatMapLatest
* Proposal: [SAA-00nn](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/00nn-flatMapLatest.md)
* Author(s): [Peter Friese](https://github.com/peterfriese)
* Status: **Proposed**
* Implementation:
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/FlatMapLatest/AsyncFlatMapLatestSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestFlatMapLatest.swift)
]
* Decision Notes:
* Bugs:
## Introduction
When transforming elements of an asynchronous sequence into new asynchronous sequences, there are cases where only the results from the most recent transformation are relevant, and previous transformations should be abandoned. This is particularly common in reactive user interfaces where rapid user input triggers asynchronous operations, but only the result of the latest operation matters.
## Motivation
Consider a search-as-you-type interface where each keystroke triggers a network request:
```swift
let searchQueries = userInputField.textChanges
// Without flatMapLatest - all requests complete, wasting resources
for await query in searchQueries {
let results = try await searchAPI(query)
displayResults(results) // May display stale results
}
```
Without automatic cancellation, earlier requests continue to completion even though their results are no longer relevant. This wastes network bandwidth, server resources, and may display stale results to the user if a slower request completes after a faster one.
The `flatMapLatest` operator solves this by automatically cancelling iteration on the previous inner sequence whenever a new element arrives from the outer sequence:
```swift
let searchResults = searchQueries.flatMapLatest { query in
searchAPI(query)
}
for await result in searchResults {
displayResults(result) // Only latest results displayed
}
```
This pattern is broadly applicable:
- **Location-based queries**: Cancel previous location lookups when user moves
- **Dynamic configuration**: Restart data loading when settings change
- **Auto-save**: Only save the most recent changes when user types rapidly
- **Real-time data**: Switch to new data streams based on user selections
## Proposed Solution
The `flatMapLatest` algorithm transforms each element from the base `AsyncSequence` into a new inner `AsyncSequence` using a transform closure. When a new element is produced by the base sequence, iteration on the current inner sequence is cancelled, and iteration begins on the newly created sequence.
The interface is available on all `AsyncSequence` types where both the base and inner sequences are `Sendable` along with two disfavored refinements to account for variations of typed throws signatures:
```swift
extension AsyncSequence where Self: Sendable {
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable where T.Failure == Failure, T.Element: Sendable, Element: Sendable
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, Failure> & Sendable where T.Failure == Never, T.Element: Sendable, Element: Sendable
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable where Failure == Never, T.Element: Sendable, Element: Sendable
}
```
This provides a clean API for expressing switching behavior:
```swift
userActions.flatMapLatest { action in
performAction(action)
}
```
## Detailed Design
The type that implements the algorithm emits elements from the inner sequences. It throws when either the base type or any inner sequence throws.
Since both the base sequence and inner sequences must be `Sendable` (to support concurrent iteration and cancellation), `AsyncFlatMapLatestSequence` is unconditionally `Sendable`.
### Implementation Strategy
The implementation uses a state machine pattern to ensure thread-safe operation:
1. **Generation Tracking**: Each new inner sequence is assigned a generation number. Elements from stale generations are discarded.
2. **Explicit Cancellation**: When a new outer element arrives, the previous inner sequence's task is explicitly cancelled and its continuation is resumed with a cancellation error.
3. **Lock-Based Coordination**: A `Lock` protects the state machine from concurrent access.
4. **Continuation Management**: The storage manages continuations for both upstream (outer/inner sequences) and downstream (consumer) demand.
This approach eliminates race conditions where cancelled sequences could emit stale values.
### Behavioral Characteristics
**Switching**: When the base sequence produces a new element, the current inner sequence iteration is immediately cancelled. Any elements it would have produced are lost.
**Completion**: The sequence completes when:
1. The base sequence finishes producing elements, AND
2. The final inner sequence finishes producing elements
**Error Handling**: If either the base sequence or any inner sequence throws, the error is immediately propagated and all tasks are cancelled.
**Cancellation**: Cancelling the downstream iteration cancels both the base sequence iteration and the current inner sequence iteration.
## Example
```swift
let requests = AsyncStream<String> { continuation in
continuation.yield("query1")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("query2")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("query3")
continuation.finish()
}
let responses = requests.flatMapLatest { query in
AsyncStream<String> { continuation in
continuation.yield("\(query): loading")
try? await Task.sleep(for: .milliseconds(50))
continuation.yield("\(query): complete")
continuation.finish()
}
}
for await response in responses {
print(response)
}
// Output (may vary due to timing):
// query1: loading
// query2: loading
// query3: loading
// query3: complete
```
In this example, the earlier queries (query1 and query2) are cancelled before they complete, so only query3 produces its complete response.
## Effect on API Resilience
This is an additive API. No existing systems are changed. The new types introduced are:
- `AsyncFlatMapLatestSequence<Base, Inner>`: The sequence type
- Associated private types for the state machine implementation
These types will be part of the ABI surface area.
## Alternatives Considered
### Alternative Names
**`switchMap`**: Used in ReactiveX, but "switch" in Swift has strong association with control flow statements.
**`switchToLatest`**: Combine's terminology, but `flatMapLatest` is more discoverable alongside other `map` variants.
**`flatMap(...).latest()`**: Requires a hypothetical `flatMap` first, adding complexity.
### Delivering All Elements
An alternative behavior would buffer elements from cancelled sequences and deliver them later. However, this contradicts the core purpose of "latest" semantics and would be better served by a different operator.
### No Automatic Cancellation
Requiring manual cancellation would place significant burden on developers and be error-prone. The automatic cancellation is the key value proposition.
### Shorthand version for async sequences of async sequences
Mimicing combine a shorthand extension of `switchToLatest` could be added as an extension when the base AsyncSequence's element conforms to AsyncSequence. The implementation of which is trivial:
```swift
extension AsyncSequence where Self: Sendable {
func switchToLatest() -> some AsyncSequence<Element.Element, Failure> where Element: AsyncSequence, Element.Failure == Failure, Element.Element: Sendable, Element: Sendable {
flatMapLatest { $0 }
}
@_disfavoredOverload
func switchToLatest() -> some AsyncSequence<Element.Element, Failure> where Element: AsyncSequence, Element.Failure == Never, Element.Element: Sendable, Element: Sendable {
flatMapLatest { $0 }
}
@_disfavoredOverload
func switchToLatest() -> some AsyncSequence<Element.Element, Failure> where Element: AsyncSequence, Failure == Never, Element.Element: Sendable, Element: Sendable {
flatMapLatest { $0 }
}
}
```
## Comparison with Other Libraries
**ReactiveX**: ReactiveX has an [API definition of switchMap](https://reactivex.io/documentation/operators/flatmap.html) that performs the same operation for Observables, switching to the latest inner Observable.
**Combine**: Combine has an [API definition of switchToLatest()](https://developer.apple.com/documentation/combine/publisher/switchtolatest()) which subscribes to the most recent Publisher produced by an upstream Publisher of Publishers. `flatMapLatest` combines the map and switch operations into a single convenient operator.
**RxSwift**: RxSwift calls this operator `flatMapLatest`, which is the naming this proposal adopts.
+211
View File
@@ -0,0 +1,211 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
## Runtime Library Exception to the Apache 2.0 License: ##
As an exception, if you use this Software to compile your source code and
portions of this Software are embedded into the binary product as a result,
you may redistribute such product without providing attribution as would
otherwise be required by Sections 4(a), 4(b) and 4(d) of the License.
+177
View File
@@ -0,0 +1,177 @@
// swift-tools-version: 6.2
import PackageDescription
import CompilerPluginSupport
let AsyncAlgorithms_v1_0 = "AvailabilityMacro=AsyncAlgorithms 1.0:macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 6.0"
#if compiler(>=6.0) && swift(>=6.0) // 5.10 doesnt support visionOS availability
let AsyncAlgorithms_v1_1 =
"AvailabilityMacro=AsyncAlgorithms 1.1:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
let AsyncAlgorithms_v1_2 =
"AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
let AsyncAlgorithms_v1_3 =
"AvailabilityMacro=AsyncAlgorithms 1.3:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
#else
let AsyncAlgorithms_v1_1 = "AvailabilityMacro=AsyncAlgorithms 1.1:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0"
let AsyncAlgorithms_v1_2 = "AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0"
let AsyncAlgorithms_v1_3 = "AvailabilityMacro=AsyncAlgorithms 1.3:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0"
#endif
let availabilityMacros: [SwiftSetting] = [
.enableExperimentalFeature(
AsyncAlgorithms_v1_0
),
.enableExperimentalFeature(
AsyncAlgorithms_v1_1
),
.enableExperimentalFeature(
AsyncAlgorithms_v1_2
),
.enableExperimentalFeature(
AsyncAlgorithms_v1_3
),
]
let package = Package(
name: "swift-async-algorithms",
products: [
.library(name: "AsyncAlgorithms", targets: ["AsyncAlgorithms"]),
.library(name: "AsyncStreaming", targets: ["AsyncStreaming"]),
],
traits: [
.default(
enabledTraits: [
// "UnstableAsyncStreaming"
]
),
.trait(
name: "UnstableAsyncStreaming",
description: """
Enables source unstable async streaming components in the _AsyncStreaming
module. Do not rely on this module in API stable packages.
"""
),
],
targets: [
.target(
name: "AsyncAlgorithms",
dependencies: [
.product(name: "OrderedCollections", package: "swift-collections"),
.product(name: "DequeModule", package: "swift-collections"),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.target(
name: "AsyncStreaming",
dependencies: [
.product(name: "BasicContainers", package: "swift-collections"),
.product(name: "ContainersPreview", package: "swift-collections"),
.product(name: "DequeModule", package: "swift-collections"),
],
exclude: ["NNNN-async-streaming.md"],
swiftSettings: [
.enableExperimentalFeature("SuppressedAssociatedTypesWithDefaults"),
.enableExperimentalFeature("LifetimeDependence"),
.enableExperimentalFeature("Lifetimes"),
.enableUpcomingFeature("LifetimeDependence"),
.enableUpcomingFeature("NonisolatedNonsendingByDefault"),
.enableUpcomingFeature("InferIsolatedConformances"),
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("MemberImportVisibility"),
.enableUpcomingFeature("InternalImportsByDefault"),
]
),
.target(
name: "AsyncSequenceValidation",
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.systemLibrary(name: "_CAsyncSequenceValidationSupport"),
.target(
name: "AsyncAlgorithms_XCTest",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.testTarget(
name: "AsyncAlgorithmsTests",
dependencies: [
.target(name: "AsyncAlgorithms"),
.target(
name: "AsyncSequenceValidation",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.custom("freebsd"),
.openbsd,
.wasi,
])
),
.target(
name: "AsyncAlgorithms_XCTest",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.custom("freebsd"),
.openbsd,
.wasi,
])
),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.testTarget(
name: "AsyncStreamingTests",
dependencies: [
"AsyncStreaming",
.product(name: "BasicContainers", package: "swift-collections"),
.product(name: "ContainersPreview", package: "swift-collections"),
.product(name: "SpanPreview", package: "swift-collections"),
],
swiftSettings: [
.enableExperimentalFeature("SuppressedAssociatedTypesWithDefaults"),
.enableExperimentalFeature("LifetimeDependence"),
.enableExperimentalFeature("Lifetimes"),
.enableUpcomingFeature("LifetimeDependence"),
.enableUpcomingFeature("NonisolatedNonsendingByDefault"),
.enableUpcomingFeature("InferIsolatedConformances"),
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("MemberImportVisibility"),
.enableUpcomingFeature("InternalImportsByDefault"),
]
),
]
)
if Context.environment["SWIFTCI_USE_LOCAL_DEPS"] == nil {
package.dependencies += [
.package(
url: "https://github.com/apple/swift-collections.git",
from: "1.7.0",
traits: [.trait(name: "UnstableContainersPreview", condition: .when(traits: ["UnstableAsyncStreaming"]))]
)
]
} else {
package.dependencies += [
.package(path: "../swift-collections")
]
}
+42
View File
@@ -0,0 +1,42 @@
// swift-tools-version: 5.7
import PackageDescription
let package = Package(
name: "swift-async-algorithms",
platforms: [
.macOS("10.15"),
.iOS("13.0"),
.tvOS("13.0"),
.watchOS("6.0"),
],
products: [
.library(name: "AsyncAlgorithms", targets: ["AsyncAlgorithms"]),
.library(name: "AsyncSequenceValidation", targets: ["AsyncSequenceValidation"]),
.library(name: "_CAsyncSequenceValidationSupport", type: .static, targets: ["AsyncSequenceValidation"]),
.library(name: "AsyncAlgorithms_XCTest", targets: ["AsyncAlgorithms_XCTest"]),
],
dependencies: [
.package(url: "https://github.com/apple/swift-collections.git", from: "1.0.4"),
.package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
],
targets: [
.target(
name: "AsyncAlgorithms",
dependencies: [.product(name: "Collections", package: "swift-collections")]
),
.target(
name: "AsyncSequenceValidation",
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"]
),
.systemLibrary(name: "_CAsyncSequenceValidationSupport"),
.target(
name: "AsyncAlgorithms_XCTest",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"]
),
.testTarget(
name: "AsyncAlgorithmsTests",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation", "AsyncAlgorithms_XCTest"]
),
]
)
+104
View File
@@ -0,0 +1,104 @@
// swift-tools-version: 5.8
import PackageDescription
import CompilerPluginSupport
// Availability Macros
let availabilityMacros: [SwiftSetting] = [
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.0:macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 6.0"
),
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.1:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
),
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
),
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.3:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
),
]
let package = Package(
name: "swift-async-algorithms",
products: [
.library(name: "AsyncAlgorithms", targets: ["AsyncAlgorithms"])
],
targets: [
.target(
name: "AsyncAlgorithms",
dependencies: [
.product(name: "OrderedCollections", package: "swift-collections"),
.product(name: "DequeModule", package: "swift-collections"),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.target(
name: "AsyncSequenceValidation",
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.systemLibrary(name: "_CAsyncSequenceValidationSupport"),
.target(
name: "AsyncAlgorithms_XCTest",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.testTarget(
name: "AsyncAlgorithmsTests",
dependencies: [
.target(name: "AsyncAlgorithms"),
.target(
name: "AsyncSequenceValidation",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.openbsd,
.wasi,
])
),
.target(
name: "AsyncAlgorithms_XCTest",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.openbsd,
.wasi,
])
),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
]
)
if Context.environment["SWIFTCI_USE_LOCAL_DEPS"] == nil {
package.dependencies += [
.package(url: "https://github.com/apple/swift-collections.git", from: "1.1.0")
]
} else {
package.dependencies += [
.package(path: "../swift-collections")
]
}
+105
View File
@@ -0,0 +1,105 @@
# swift-async-algorithms
**Swift Async Algorithms** is an open-source package of asynchronous sequence and advanced algorithms that involve concurrency, along with their related types.
This package has three main goals:
- First-class integration with `async/await`
- Provide a home for time-based algorithms
- Be cross-platform and open source
## Motivation
AsyncAlgorithms is a package for algorithms that work with *values over time*. That includes those primarily about *time*, like `debounce` and `throttle`, but also algorithms about *order* like `combineLatest` and `merge`. Operations that work with multiple inputs (like `zip` does on `Sequence`) can be surprisingly complex to implement, with subtle behaviors and many edge cases to consider. A shared package can get these details correct, with extensive testing and documentation, for the benefit of all Swift apps.
The foundation for AsyncAlgorithms was included in Swift 5.5 from [AsyncSequence](https://github.com/apple/swift-evolution/blob/main/proposals/0298-asyncsequence.md). Swift 5.5 also brings the ability to use a natural `for/in` loop with `await` to process the values in an `AsyncSequence` and `Sequence`-equivalent API like `map` and `filter`. Structured concurrency allows us to write code where intermediate state is simply a local variable, `try` can be used directly on functions that `throw`, and generally treat the logic for asynchronous code similar to that of synchronous code.
This package is the home for these APIs. Development and API design take place on [GitHub](https://github.com/apple/swift-async-algorithms) and the [Swift Forums](https://forums.swift.org/c/related-projects/swift-async-algorithms).
## Contents
#### Combining asynchronous sequences
- [`chain(_:...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chain.md): Concatenates two or more asynchronous sequences with the same element type.
- [`combineLatest(_:...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/CombineLatest.md): Combines two or more asynchronous sequences into an asynchronous sequence producing a tuple of elements from those base asynchronous sequences that updates when any of the base sequences produce a value.
- [`merge(_:...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Merge.md): Merges two or more asynchronous sequence into a single asynchronous sequence producing the elements of all of the underlying asynchronous sequences.
- [`zip(_:...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Zip.md): Creates an asynchronous sequence of pairs built out of underlying asynchronous sequences.
- [`joined(separator:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Joined.md): Concatenated elements of an asynchronous sequence of asynchronous sequences, inserting the given separator between each element.
#### Creating asynchronous sequences
- [`async`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Lazy.md): Create an asynchronous sequence composed from a synchronous sequence.
- [`AsyncChannel`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Channel.md): An asynchronous sequence with back pressure sending semantics.
- [`AsyncThrowingChannel`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Channel.md): An asynchronous sequence with back pressure sending semantics that can emit failures.
#### Performance optimized asynchronous iterators
- [`AsyncBufferedByteIterator`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/BufferedBytes.md): A highly efficient iterator useful for iterating byte sequences derived from asynchronous read functions.
#### Other useful asynchronous sequences
- [`adjacentPairs()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/AdjacentPairs.md): Collects tuples of adjacent elements.
- [`chunks(...)` and `chunked(...)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Chunked.md): Collect values into chunks.
- [`compacted()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Compacted.md): Remove nil values from an asynchronous sequence.
- [`removeDuplicates()`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/RemoveDuplicates.md): Remove sequentially adjacent duplicate values.
- [`interspersed(with:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Intersperse.md): Place a value between every two elements of an asynchronous sequence.
#### Asynchronous Sequences that transact in time
- [`debounce(for:tolerance:clock:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Debounce.md): Emit values after a quiescence period has been reached.
- [`throttle(for:clock:reducing:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Throttle.md): Ensure a minimum interval has elapsed between events.
- [`AsyncTimerSequence`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Timer.md): Emit the value of now at a given interval repeatedly.
#### Obtaining all values from an asynchronous sequence
- [`RangeReplaceableCollection.init(_:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md): Creates a new instance of a collection containing the elements of an asynchronous sequence.
- [`Dictionary.init(uniqueKeysWithValues:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md): Creates a new dictionary from the key-value pairs in the given asynchronous sequence.
- [`Dictionary.init(_:uniquingKeysWith:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md): Creates a new dictionary from the key-value pairs in the given asynchronous sequence, using a combining closure to determine the value for any duplicate keys.
- [`Dictionary.init(grouping:by:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md): Creates a new dictionary whose keys are the groupings returned by the given closure and whose values are arrays of the elements that returned each key.
- [`SetAlgebra.init(_:)`](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md): Creates a new set from an asynchronous sequence of items.
#### Effects
Each algorithm has specific behavioral effects. For throwing effects these can either be if the sequence throws, does not throw, or rethrows errors. Sendability effects in some asynchronous sequences are conditional whereas others require the composed parts to all be sendable to satisfy a requirement of `Sendable`. The effects are [listed here](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Effects.md).
## Adding Swift Async Algorithms as a Dependency
To use the `AsyncAlgorithms` library in a SwiftPM project,
add the following line to the dependencies in your `Package.swift` file:
```swift
.package(url: "https://github.com/apple/swift-async-algorithms", from: "1.0.0"),
```
Include `"AsyncAlgorithms"` as a dependency for your executable target:
```swift
.target(name: "<target>", dependencies: [
.product(name: "AsyncAlgorithms", package: "swift-async-algorithms"),
]),
```
Finally, add `import AsyncAlgorithms` to your source code.
## Getting Started
⚠️ Please note that this package requires Xcode 14 on macOS hosts. Previous versions of Xcode do not contain the required Swift version.
### Building/Testing Using Xcode on macOS
1. In the `swift-async-algorithms` directory run `swift build` or `swift test` accordingly
### Building/Testing on Linux
1. Download the most recent development toolchain for your Linux distribution
2. Decompress the archive to a path in which the `swift` executable is in the binary search path environment variable (`$PATH`)
3. In the `swift-async-algorithms` directory run `swift build` or `swift test` accordingly
## Source Stability
The Swift Async Algorithms package has a goal of being source stable as soon as possible; version numbers will follow [Semantic Versioning](https://semver.org/). Source breaking changes to public API can only land in a new major version.
The public API of version 1.0 of the `swift-async-algorithms` package will consist of non-underscored declarations that are marked `public` in the `AsyncAlgorithms` module. Interfaces that aren't part of the public API may continue to change in any release, including patch releases.
Future minor versions of the package may introduce changes to these rules as needed.
We'd like this package to quickly embrace Swift language and toolchain improvements that are relevant to its mandate. Accordingly, from time to time, we expect that new versions of this package will require clients to upgrade to a more recent Swift toolchain release. Requiring a new Swift release will only require a minor version bump.
@@ -0,0 +1,93 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// An `AsyncSequence` that iterates over the adjacent pairs of the original
/// original `AsyncSequence`.
///
/// ```
/// for await (first, second) in (1...5).async.adjacentPairs() {
/// print("First: \(first), Second: \(second)")
/// }
///
/// // First: 1, Second: 2
/// // First: 2, Second: 3
/// // First: 3, Second: 4
/// // First: 4, Second: 5
/// ```
///
/// - Returns: An `AsyncSequence` where the element is a tuple of two adjacent elements
/// or the original `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func adjacentPairs() -> AsyncAdjacentPairsSequence<Self> {
AsyncAdjacentPairsSequence(self)
}
}
/// An `AsyncSequence` that iterates over the adjacent pairs of the original
/// `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncAdjacentPairsSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = (Base.Element, Base.Element)
@usableFromInline
let base: Base
@inlinable
init(_ base: Base) {
self.base = base
}
/// The iterator for an `AsyncAdjacentPairsSequence` instance.
@frozen
public struct Iterator: AsyncIteratorProtocol {
public typealias Element = (Base.Element, Base.Element)
@usableFromInline
var base: Base.AsyncIterator
@usableFromInline
internal var previousElement: Base.Element?
@inlinable
init(_ base: Base.AsyncIterator) {
self.base = base
}
@inlinable
public mutating func next() async rethrows -> (Base.Element, Base.Element)? {
if previousElement == nil {
previousElement = try await base.next()
}
guard let previous = previousElement, let next = try await base.next() else {
return nil
}
previousElement = next
return (previous, next)
}
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncAdjacentPairsSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncAdjacentPairsSequence.Iterator: Sendable {}
@@ -0,0 +1,36 @@
# ``AsyncAlgorithms``
**Swift Async Algorithms** is an open-source package of asynchronous sequence and advanced algorithms that involve concurrency, along with their related types.
## Overview
This package has three main goals:
- First-class integration with `async/await`
- Provide a home for time-based algorithms
- Be cross-platform and open source
## Topics
### Getting Started
- <doc:AdjacentPairs>
- <doc:BufferedBytes>
- <doc:Chain>
- <doc:Channel>
- <doc:Chunked>
- <doc:Collections>
- <doc:CombineLatest>
- <doc:Compacted>
- <doc:Debounce>
- <doc:Effects>
- <doc:Intersperse>
- <doc:Joined>
- <doc:Lazy>
- <doc:Merge>
- <doc:Reductions>
- <doc:RemoveDuplicates>
- <doc:Share>
- <doc:Throttle>
- <doc:Timer>
- <doc:Zip>
@@ -0,0 +1,49 @@
# AdjacentPairs
* Author(s): [László Teveli](https://github.com/tevelee)
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAdjacentPairsSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestAdjacentPairs.swift)]
The `adjacentPairs()` API serve the purpose of collecting adjacent values. This operation is available for any `AsyncSequence` by calling the `adjacentPairs()` method.
```swift
extension AsyncSequence {
public func adjacentPairs() -> AsyncAdjacentPairsSequence<Self>
}
```
## Detailed Design
The `adjacentPairs()` algorithm produces elements of tuple (size of 2), containing a pair of the original `Element` type.
The interface for this algorithm is available on all `AsyncSequence` types. The returned `AsyncAdjacentPairsSequence` conditionally conforms to `Sendable`.
Its iterator keeps track of the previous element returned in the `next()` function and updates it in every turn.
```swift
for await (first, second) in (1...5).async.adjacentPairs() {
print("First: \(first), Second: \(second)")
}
// First: 1, Second: 2
// First: 2, Second: 3
// First: 3, Second: 4
// First: 4, Second: 5
```
It composes well with the [Dictionary.init(_:uniquingKeysWith:)](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncAlgorithms.docc/Guides/Collections.md) API that deals with `AsyncSequence` of tuples.
```swift
Dictionary(uniqueKeysWithValues: url.lines.adjacentPairs())
```
## Alternatives Considered
This functionality is often written as a `zip` of a sequence together with itself, dropping its first element (`zip(source, source.dropFirst())`).
It's such a dominant use-case, the [swift-algorithms](https://github.com/apple/swift-algorithms) package also [introduced](https://github.com/apple/swift-algorithms/pull/119) it to its collection of algorithms.
## Credits/Inspiration
The synchronous counterpart in [swift-algorithms](https://github.com/apple/swift-algorithms/blob/main/Guides/AdjacentPairs.md).
@@ -0,0 +1,54 @@
# AsyncBufferedByteIterator
Provides a highly efficient iterator useful for iterating byte sequences derived from asynchronous read functions.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncBufferedByteIterator.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestBufferedByteIterator.swift)]
This type provides infrastructure for creating `AsyncSequence` types with an `Element` of `UInt8` backed by file descriptors or similar read sources.
```swift
struct AsyncBytes: AsyncSequence {
public typealias Element = UInt8
var handle: ReadableThing
internal init(_ readable: ReadableThing) {
handle = readable
}
public func makeAsyncIterator() -> AsyncBufferedByteIterator {
return AsyncBufferedByteIterator(capacity: 16384) { buffer in
// This runs once every 16384 invocations of next()
return try await handle.read(into: buffer)
}
}
}
```
## Detailed Design
```swift
public struct AsyncBufferedByteIterator: AsyncIteratorProtocol, Sendable {
public typealias Element = UInt8
public init(
capacity: Int,
readFunction: @Sendable @escaping (UnsafeMutableRawBufferPointer) async throws -> Int
)
public mutating func next() async throws -> UInt8?
}
```
For each invocation of `next`, the iterator will check if a buffer has been filled. If the buffer is filled with some amount of bytes, a fast path is taken to directly return a byte out of that buffer. If the buffer is not filled, the read function is invoked to acquire the next filled buffer, at which point it takes a byte out of that buffer.
If the read function returns `0`, indicating it didn't read any more bytes, the iterator is decided to be finished and no additional invocations to the read function are made.
If the read function throws, the error will be thrown by the iteration. Subsequent invocations to the iterator will then return `nil` without invoking the read function.
If the task is cancelled during the iteration, the iteration will check the cancellation only in passes where the read function is invoked, and will throw a `CancellationError`.
### Naming
This type was named precisely for what it does: it is an asynchronous iterator that buffers bytes.
@@ -0,0 +1,75 @@
# Chain
Chains two or more asynchronous sequences together sequentially.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChain2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChain3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChain.swift)]
Chains two or more asynchronous sequences together sequentially where the elements from the resulting asynchronous sequence are comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs.
This operation is available for all `AsyncSequence` types who share the same `Element` type.
```swift
let preamble = [
"// Some header to add as a preamble",
"//",
""
].async
let lines = chain(preamble, URL(fileURLWithPath: "/tmp/Sample.swift").lines)
for try await line in lines {
print(line)
}
```
The above example shows how two `AsyncSequence` types can be chained together. In this case it prepends a preamble to the `lines` content of the file.
## Detailed Design
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func chain<Base1: AsyncSequence, Base2: AsyncSequence>(_ s1: Base1, _ s2: Base2) -> AsyncChain2Sequence<Base1, Base2> where Base1.Element == Base2.Element
public func chain<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ s1: Base1, _ s2: Base2, _ s3: Base3) -> AsyncChain3Sequence<Base1, Base2, Base3>
public struct AsyncChain2Sequence<Base1: AsyncSequence, Base2: AsyncSequence> where Base1.Element == Base2.Element {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChain2Sequence: Sendable where Base1: Sendable, Base2: Sendable { }
extension AsyncChain2Sequence.Iterator: Sendable where Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable { }
public struct AsyncChain3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence> where Base1.Element == Base2.Element, Base1.Element == Base3.Element {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChain3Sequence: Sendable where Base1: Sendable, Base2: Sendable, Base3: Sendable { }
extension AsyncChain3Sequence.Iterator: Sendable where Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable, Base3.AsyncIterator: Sendable { }
```
The `chain(_:...)` function takes two or more sequences as arguments.
The resulting `AsyncChainSequence` type is an asynchronous sequence, with conditional conformance to `Sendable` when the arguments also conform to it.
When any of the asynchronous sequences being chained together come to their end of iteration, the `AsyncChainSequence` iteration proceeds to the next asynchronous sequence. When the last asynchronous sequence reaches the end of iteration, the `AsyncChainSequence` then ends its iteration.
At any point in time, if one of the comprising asynchronous sequences throws an error during iteration, the resulting `AsyncChainSequence` iteration will throw that error and end iteration. The throwing behavior of `AsyncChainSequence` is that it will throw when any of its comprising bases throw, and will not throw when all of its comprising bases do not throw.
### Naming
This function's and type's name match the term of art used in other languages and libraries.
This combinator function is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Chain.md).
@@ -0,0 +1,82 @@
# Channel
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Channels/AsyncChannel.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Channels/AsyncThrowingChannel.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChannel.swift)
]
## Introduction
`AsyncStream` introduced a mechanism to send buffered elements from a context that doesn't use Swift concurrency into one that does. That design only addressed a portion of the potential use cases; the missing portion was the back pressure excerpted across two concurrency domains.
## Proposed Solution
To achieve a system that supports back pressure and allows for the communication of more than one value from one task to another we are introducing a new type, the _channel_. The channel will be a reference-type asynchronous sequence with an asynchronous sending capability that awaits the consumption of iteration. Each value sent by the channel will await the consumption of that value by iteration. That awaiting behavior will allow for the affordance of back pressure applied from the consumption site to be transmitted to the production site. This means that the rate of production cannot exceed the rate of consumption, and that the rate of consumption cannot exceed the rate of production. Sending a terminal event to the channel will instantly resume all pending operations for every producers and consumers.
## Detailed Design
Similar to the `AsyncStream` and `AsyncThrowingStream` types, the type for sending elements via back pressure will come in two versions. These two versions will account for the throwing nature or non-throwing nature of the elements being produced.
Each type will have functions to send elements and to send terminal events.
```swift
public final class AsyncChannel<Element: Sendable>: AsyncSequence, Sendable {
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async -> Element?
}
public init(element elementType: Element.Type = Element.self)
public func send(_ element: Element) async
public func finish()
public func makeAsyncIterator() -> Iterator
}
public final class AsyncThrowingChannel<Element: Sendable, Failure: Error>: AsyncSequence, Sendable {
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async throws -> Element?
}
public init(element elementType: Element.Type = Element.self, failure failureType: Failure.Type = Failure.self)
public func send(_ element: Element) async
public func fail(_ error: Error) where Failure == Error
public func finish()
public func makeAsyncIterator() -> Iterator
}
```
Channels are intended to be used as communication types between tasks. Particularly when one task produces values and another task consumes said values. On the one hand, the back pressure applied by `send(_:)` via the suspension/resume ensures that the production of values does not exceed the consumption of values from iteration. This method suspends after enqueuing the event and is resumed when the next call to `next()` on the `Iterator` is made. On the other hand, the call to `finish()` or `fail(_:)` immediately resumes all the pending operations for every producers and consumers. Thus, every suspended `send(_:)` operations instantly resume, so as every suspended `next()` operations by producing a nil value, or by throwing an error, indicating the termination of the iterations. Further calls to `send(_:)` will immediately resume. The calls to `send(:)` and `next()` will immediately resume when their supporting task is cancelled, other operations from other tasks will remain active.
```swift
let channel = AsyncChannel<String>()
Task {
while let resultOfLongCalculation = doLongCalculations() {
await channel.send(resultOfLongCalculation)
}
channel.finish()
}
for await calculationResult in channel {
print(calculationResult)
}
```
The example above uses a task to perform intense calculations; each of which are sent to the other task via the `send(_:)` method. That call to `send(_:)` returns when the next iteration of the channel is invoked.
## Alternatives Considered
The use of the name "subject" was considered, due to its heritage as a name for a sync-to-async adapter type.
It was considered to make `AsyncChannel` and `AsyncThrowingChannel` actors, however due to the cancellation internals it would imply that these types would need to create new tasks to handle cancel events. The advantages of an actor in this particular case did not outweigh the impact of adjusting the implementations to be actors.
## Credits/Inspiration
`AsyncChannel` and `AsyncThrowingChannel` was heavily inspired from `Subject` but with the key difference that it uses Swift concurrency to apply back pressure.
https://developer.apple.com/documentation/combine/subject/
@@ -0,0 +1,314 @@
# Chunked
* Author(s): [Kevin Perry](https://github.com/kperryua)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedByGroupSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountAndSignalSequence.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunksOfCountSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChunk.swift)
]
## Introduction
Grouping of values from an asynchronous sequence is often useful for tasks that involve writing those values efficiently or useful to handle specific structured data inputs.
## Proposed Solution
Chunking operations can be broken down into a few distinct categories: grouping according to a binary predicate used to determine whether consecutive elements belong to the same group, projecting an element's property to determine the element's chunk membership, by discrete count, by another signal asynchronous sequence which indicates when the chunk should be delimited, or by a combination of count and signal.
### Grouping
Group chunks are determined by passing two consecutive elements to a closure which tests whether they are in the same group. When the `AsyncChunkedByGroupSequence` iterator receives the first element from the base sequence, it will immediately be added to a group. When it receives the second item, it tests whether the previous item and the current item belong to the same group. If they are not in the same group, then the iterator emits the first item's group and a new group is created containing the second item. Items declared to be in the same group accumulate until a new group is declared, or the iterator finds the end of the base sequence. When the base sequence terminates, the final group is emitted. If the base sequence throws an error, `AsyncChunkedByGroupSequence` will rethrow that error immediately and discard any current group.
```swift
extension AsyncSequence {
public func chunked<Collected: RangeReplaceableCollection>(
into: Collected.Type,
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, Collected>
where Collected.Element == Element
public func chunked(
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, [Element]>
}
```
Consider an example where an asynchronous sequence emits the following values: `10, 20, 30, 10, 40, 40, 10, 20`. Given the chunked operation to be defined as follows:
```swift
let chunks = numbers.chunked { $0 <= $1 }
for await numberChunk in chunks {
print(numberChunk)
}
```
That snippet will produce the following values:
```swift
[10, 20, 30]
[10, 40, 40]
[10, 20]
```
While `Array` is the default type for chunks, thanks to the overload that takes a `RangeReplaceableCollection` type, the same sample can be chunked into instances of `ContiguousArray`, or any other `RangeReplaceableCollection` instead.
```swift
let chunks = numbers.chunked(into: ContiguousArray.self) { $0 <= $1 }
for await numberChunk in chunks {
print(numberChunk)
}
```
That variant is the funnel method for the main implementation, which passes `[Element].self` in as the parameter.
### Projection
In some scenarios, chunks are determined not by comparing different elements, but by the element itself. This may be the case when the element has some sort of discriminator that can determine the chunk it belongs to. When two consecutive elements have different projections, the current chunk is emitted and a new chunk is created for the new element.
When the `AsyncChunkedOnProjectionSequence`'s iterator receives `nil` from the base sequence, it emits the final chunk. When the base sequence throws an error, the iterator discards the current chunk and rethrows that error.
Similarly to the `chunked(by:)` method this algorithm has an optional specification for the `RangeReplaceableCollection` which is used as the type of each chunk.
```swift
extension AsyncSequence {
public func chunked<Subject : Equatable, Collected: RangeReplaceableCollection>(
into: Collected.Type,
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, Collected>
public func chunked<Subject : Equatable>(
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, [Element]>
}
```
The following example shows how a sequence of names can be chunked together by their first characters.
```swift
let names = URL(fileURLWithPath: "/tmp/names.txt").lines
let groupedNames = names.chunked(on: \.first!)
for try await (firstLetter, names) in groupedNames {
print(firstLetter)
for name in names {
print(" ", name)
}
}
```
A special property of this kind of projection chunking is that when an asynchronous sequence's elements are known to be ordered, the output of the chunking asynchronous sequence is suitable for initializing dictionaries using the `AsyncSequence` initializer for `Dictionary`. This is because the projection can be easily designed to match the sorting characteristics and thereby guarantee that the output matches the pattern of an array of pairs of unique "keys" with the chunks as the "values".
In the example above, if the names are known to be ordered then you can take advantage of the uniqueness of each "first character" projection to initialize a `Dictionary` like so:
```swift
let names = URL(fileURLWithPath: "/tmp/names.txt").lines
let nameDirectory = try await Dictionary(uniqueKeysWithValues: names.chunked(on: \.first!))
```
### Count or Signal
Sometimes chunks are determined not by the elements themselves, but by external factors. This final category enables limiting chunks to a specific size and/or delimiting them by another asynchronous sequence which is referred to as a "signal". This particular chunking family is useful for scenarios where the elements are more efficiently processed as chunks than individual elements, regardless of their values.
This family is broken down into two sub-families of methods: ones that employ a signal plus an optional count (which return an `AsyncChunksOfCountOrSignalSequence`), and the ones that only deal with counts (which return an `AsyncChunksOfCountSequence`). Both sub-families have `Collected` as their element type, or `Array` if unspecified. These sub-families have rethrowing behaviors; if the base `AsyncSequence` can throw then the chunks sequence can also throw. Likewise if the base `AsyncSequence` cannot throw then the chunks sequence also cannot throw.
##### Count only
```swift
extension AsyncSequence {
public func chunks<Collected: RangeReplaceableCollection>(
ofCount count: Int,
into: Collected.Type
) -> AsyncChunksOfCountSequence<Self, Collected>
where Collected.Element == Element
public func chunks(
ofCount count: Int
) -> AsyncChunksOfCountSequence<Self, [Element]>
}
```
If a chunk size limit is specified via an `ofCount` parameter, the sequence will produce chunks of type `Collected` with at most the specified number of elements. When a chunk reaches the given size, the asynchronous sequence will emit it immediately.
For example, an asynchronous sequence of `UInt8` bytes can be chunked into at most 1024-byte `Data` instances like so:
```swift
let packets = bytes.chunks(ofCount: 1024, into: Data.self)
for try await packet in packets {
write(packet)
}
```
##### Signal only
```swift
extension AsyncSequence {
public func chunked<Signal, Collected: RangeReplaceableCollection>(
by signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal>
where Collected.Element == Element
public func chunked<Signal>(
by signal: Signal
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal>
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(
by timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>>
where Collected.Element == Element
public func chunked<C: Clock>(
by timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>>
}
```
If a signal asynchronous sequence is specified, the chunking asynchronous sequence emits chunks whenever the signal emits. The signals element values are ignored. If the chunking asynchronous sequence hasn't accumulated any elements since its previous emission, then no value is emitted in response to the signal.
Since time is a frequent method of signaling desired delineations of chunks, there is a pre-specialized set of overloads that take `AsyncTimerSequence`. These allow shorthand initialization by using `AsyncTimerSequence`'s static member initializers.
As an example, an asynchronous sequence of log messages can be chunked into arrays of logs in four second segments like so:
```swift
let fourSecondsOfLogs = logs.chunked(by: .repeating(every: .seconds(4)))
for await chunk in fourSecondsOfLogs {
send(chunk)
}
```
##### Count or Signal
```swift
extension AsyncSequence {
public func chunks<Signal, Collected: RangeReplaceableCollection>(
ofCount count: Int,
or signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal>
where Collected.Element == Element
public func chunks<Signal>(
ofCount count: Int,
or signal: Signal
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal>
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(
by timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>>
where Collected.Element == Element
public func chunked<C: Clock>(
by timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>>
}
```
If both count and signal are specified, the chunking asynchronous sequence emits chunks whenever *either* the chunk reaches the specified size *or* the signal asynchronous sequence emits. When a signal causes a chunk to be emitted, the accumulated element count is reset back to zero. When an `AsyncTimerSequence` is used as a signal, the timer is started from the moment `next()` is called for the first time on `AsyncChunksOfCountOrSignalSequence`'s iterator, and it emits on a regular cadence from that moment. Note that the scheduling of the timer's emission is unaffected by any chunks emitted based on count.
Like the example above, this code emits up to 1024-byte `Data` instances, but a chunk will also be emitted every second.
```swift
let packets = bytes.chunks(ofCount: 1024, or: .repeating(every: .seconds(1)), into: Data.self)
for try await packet in packets {
write(packet)
}
```
In any configuration of any of the chunking families, when the base asynchronous sequence terminates, one of two things will happen: 1) a partial chunk will be emitted, or 2) no chunk will be emitted (i.e. the iterator received no elements since the emission of the previous chunk). No elements from the base asynchronous sequence are ever discarded, except in the case of a thrown error.
## Detailed Design
### Grouping
```swift
public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Collected?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChunkedByGroupSequence: Sendable
where Base: Sendable, Base.Element: Sendable { }
extension AsyncChunkedByGroupSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable, Base.Element: Sendable { }
```
### Projection
```swift
public struct AsyncChunkedOnProjectionSequence<Base: AsyncSequence, Subject: Equatable, Collected: RangeReplaceableCollection>: AsyncSequence where Collected.Element == Base.Element {
public typealias Element = (Subject, Collected)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> (Subject, Collected)?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChunkedOnProjectionSequence: Sendable
where Base: Sendable, Base.Element: Sendable { }
extension AsyncChunkedOnProjectionSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable, Base.Element: Sendable, Subject: Sendable { }
```
### Count
```swift
public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Collected?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncChunksOfCountSequence : Sendable where Base : Sendable, Base.Element : Sendable { }
extension AsyncChunksOfCountSequence.Iterator : Sendable where Base.AsyncIterator : Sendable, Base.Element : Sendable { }
```
### Count or Signal
```swift
public struct AsyncChunksOfCountOrSignalSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection, Signal: AsyncSequence>: AsyncSequence, Sendable
where
Collected.Element == Base.Element,
Base: Sendable, Signal: Sendable,
Base.AsyncIterator: Sendable, Signal.AsyncIterator: Sendable,
Base.Element: Sendable, Signal.Element: Sendable {
public typealias Element = Collected
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async rethrows -> Collected?
}
public func makeAsyncIterator() -> Iterator
}
```
## Alternatives Considered
It was considered to make the chunked element to be an `AsyncSequence` instead of allowing collection into a `RangeReplaceableCollection` however it was determined that the throwing behavior of that would be complex to understand. If that hurdle could be overcome then that might be a future direction/consideration that would be worth exploring.
Variants of `chunked(by:)` (grouping) and `chunked(on:)` (projection) methods could be added that take delimiting `Signal` and `AsyncTimerSequence` inputs similar to `chunked(byCount:or:)`. However, it was decided that such functionality was likely to be underutilized and not worth the complication to the already broad surface area of `chunked` methods.
The naming of this family was considered to be `collect` which is used in APIs like `Combine`. This family of functions has distinct similarity to those APIs.
## Credits/Inspiration
This transformation function is a heavily inspired analog of the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Chunked.md)
@@ -0,0 +1,104 @@
# Collection Initializers
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Dictionary.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/RangeReplaceableCollection.swift),
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/SetAlgebra.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestDictionary.swift),
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestRangeReplaceableCollection.swift),
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestSetAlgebra.swift),
]
## Introduction
`Array`, `Dictionary` and `Set` are some of the most commonly-used data structures for storing collections of elements. Having a way to transition from an `AsyncSequence` to a collection is not only a useful shorthand but a powerful way of expressing direct intent for how to consume an `AsyncSequence`.
This type of functionality can be useful for examples and testing, and also for interfacing with existing APIs that expect a fully-formed collection before processing it.
## Proposed Solution
Three categories of initializers will be added to provide initializers for those three primary types: `Array`, `Dictionary` and `Set`. However these initializers can be written in a generic fashion such that they can apply to all similar collections.
`RangeReplaceableCollection` will gain a new initializer that constructs a collection given an `AsyncSequence`. This will allow for creating arrays from asynchronous sequences but also allow for creating types like `Data` or `ContiguousArray`. Because of the nature of asynchronous sequences, this initializer must be asynchronous and declare that it rethrows errors from the base asynchronous sequence.
```swift
extension RangeReplaceableCollection {
public init<Source: AsyncSequence>(
_ source: Source
) async rethrows
where Source.Element == Element
}
```
`Dictionary` will gain a family of new asynchronous, rethrowing initializers to parallel the existing `Sequence`-based initializers. The initializers will be asynchronous to facilitate uniquing keys and other tasks that may be asynchronous, in addition to the asynchronous initialization of the dictionaries.
```swift
extension Dictionary {
public init<S: AsyncSequence>(
uniqueKeysWithValues keysAndValues: S
) async rethrows
where S.Element == (Key, Value)
public init<S: AsyncSequence>(
_ keysAndValues: S,
uniquingKeysWith combine: (Value, Value) async throws -> Value
) async rethrows
where S.Element == (Key, Value)
public init<S: AsyncSequence>(
grouping values: S,
by keyForValue: (S.Element) async throws -> Key
) async rethrows
where Value == [S.Element]
}
```
`SetAlgebra` will gain a new asynchronous, rethrowing initializer that constructs a `SetAlgebra` type given an `AsyncSequence`. This will allow for creating sets from asynchronous sequences and allow for creating types like `OptionSet` types or `IndexSet`.
```swift
extension SetAlgebra {
public init<Source: AsyncSequence>(
_ source: Source
) async rethrows
where Source.Element == Element
}
```
## Detailed Design
Each of the initializers is intended for uses where the `AsyncSequence` being used for initialization is known to be finite. Common uses include:
* Reading from files via the `AsyncBytes` style sequences or `lines` accessors.
* Gathering elements produced by a `TaskGroup`.
* Accessing a prefix of an indefinite `AsyncSequence`.
Each of the initializers will use the `for`-`await`-`in`/`for`-`try`-`await`-`in` syntax to iterate the sequence directly in the initializer. In addition each initializer relies on the `AsyncSequence` being passed in to properly respect cancellation. In the cases where cancellation is a potential, developers should be ready to either check immediately or be ready for initialization based on a partial sequence, depending on the behavior of the `AsyncSequence` being used.
### RangeReplaceableCollection
```swift
let contents = try await Data(URL(fileURLWithPath: "/tmp/example.bin").resourceBytes)
```
### Dictionary
```swift
let table = await Dictionary(uniqueKeysWithValues: zip(keys, values))
```
### SetAlgebra
```swift
let allItems = await Set(items.prefix(10))
```
## Alternatives Considered
The spelling of these initializers could be expressed as a trailing conversion. However, that can lead to hard-to-read chains of operations. Functionally these all belong to the `reduce` family of functions, but due to source readability concerns they are more ergonomic for understanding what the code does by using the initializer patterns.
## Credits/Inspiration
The direct inspiration for each initialization is from their standard library counterparts.
@@ -0,0 +1,83 @@
# Combine Latest
Combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/CombineLatest/AsyncCombineLatest2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/CombineLatest/AsyncCombineLatest3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestCombineLatest.swift)]
```swift
let appleFeed = URL("http://www.example.com/ticker?symbol=AAPL").lines
let nasdaqFeed = URL("http://www.example.com/ticker?symbol=^IXIC").lines
for try await (apple, nasdaq) in combineLatest(appleFeed, nasdaqFeed) {
print("AAPL: \(apple) NASDAQ: \(nasdaq)")
}
```
Given some sample inputs the following combined events can be expected.
| Timestamp | appleFeed | nasdaqFeed | combined output |
| ----------- | --------- | ---------- | ----------------------------- |
| 11:40 AM | 173.91 | | |
| 12:25 AM | | 14236.78 | AAPL: 173.91 NASDAQ: 14236.78 |
| 12:40 AM | | 14218.34 | AAPL: 173.91 NASDAQ: 14218.34 |
| 1:15 PM | 173.00 | | AAPL: 173.00 NASDAQ: 14218.34 |
## Detailed Design
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func combineLatest<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncCombineLatest2Sequence<Base1, Base2>
public func combineLatest<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncCombineLatest3Sequence<Base1, Base2, Base3>
public struct AsyncCombineLatest2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable, Base2.Element: Sendable,
Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable {
public typealias Element = (Base1.Element, Base2.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
public struct AsyncCombineLatest3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable, Base3: Sendable
Base1.Element: Sendable, Base2.Element: Sendable, Base3.Element: Sendable
Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable, Base3.AsyncIterator: Sendable {
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The `combineLatest(_:...)` function takes two or more asynchronous sequences as arguments and produces an `AsyncCombineLatestSequence` which is an asynchronous sequence.
Since the bases comprising the `AsyncCombineLatestSequence` must be iterated concurrently to produce the latest value, those sequences must be able to be sent to child tasks. This means that a prerequisite of the bases must be that the base asynchronous sequences, their iterators, and the elements they produce must all be `Sendable`.
If any of the bases terminate before the first element is produced, then the `AsyncCombineLatestSequence` iteration can never be satisfied. So, if a base's iterator returns `nil` at the first iteration, then the `AsyncCombineLatestSequence` iterator immediately returns `nil` to signify a terminal state. In this particular case, any outstanding iteration of other bases will be cancelled. After the first element is produced ,this behavior is different since the latest values can still be satisfied by at least one base. This means that beyond the construction of the first tuple comprised of the returned elements of the bases, the terminal state of the `AsyncCombineLatestSequence` iteration will only be reached when all of the base iterations reach a terminal state.
The throwing behavior of `AsyncCombineLatestSequence` is that if any of the bases throw, then the composed asynchronous sequence throws on its iteration. If at any point (within the first iteration or afterwards), an error is thrown by any base, the other iterations are cancelled and the thrown error is immediately thrown to the consuming iteration.
### Naming
Since the inherent behavior of `combineLatest(_:...)` combines the latest values from multiple streams into a tuple the naming is intended to be quite literal. There are precedent terms of art in other frameworks and libraries (listed in the comparison section). Other naming takes the form of "withLatestFrom". This was disregarded since the "with" prefix is often most associated with the passing of a closure and some sort of contextual concept; `withUnsafePointer` or `withUnsafeContinuation` are prime examples.
### Comparison with other libraries
Combine latest often appears in libraries developed for processing events over time since the event ordering of a concept of "latest" only occurs when asynchrony is involved.
**ReactiveX** ReactiveX has an [API definition of CombineLatest](https://reactivex.io/documentation/operators/combinelatest.html) as a top level function for combining Observables.
**Combine** Combine has an [API definition of combineLatest](https://developer.apple.com/documentation/combine/publisher/combinelatest(_:)/) has an operator style method for combining Publishers.
@@ -0,0 +1,57 @@
# Compacted
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncCompactedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestCompacted.swift)
]
## Introduction
Just as it is common for `Sequence` types that contain optional values to need to `.compactMap { $0 }`, `AsyncSequence` types have the same use cases. This common task means that the type must employ a closure to test the optional value. This can be done more efficiently for both execution performance as well as API efficiency of typing.
## Proposed Solution
Similar to the Swift Algorithms package we propose that a new method be added to `AsyncSequence` to fit this need.
```swift
extension AsyncSequence {
public func compacted<Unwrapped>() -> AsyncCompactedSequence<Self, Unwrapped>
where Element == Unwrapped?
}
```
This is equivalent to writing `.compactMap { $0 }` from a behavioral standpoint but is easier to reason about and is more efficient since it does not need to execute or store a closure.
## Detailed Design
The `AsyncCompactedSequence` type from an effects standpoint works just like `AsyncCompactMapSequence`. When the base asynchronous sequence throws, the iteration of `AsyncCompactedSequence` can throw. Likewise if the base does not throw then the iteration of `AsyncCompactedSequence` does not throw. This type is conditionally `Sendable` when the base, base element, and base iterator are `Sendable.
```swift
public struct AsyncCompactedSequence<Base: AsyncSequence, Element>: AsyncSequence
where Base.Element == Element? {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator())
}
}
extension AsyncCompactedSequence: Sendable
where
Base: Sendable, Base.Element: Sendable,
Base.AsyncIterator: Sendable { }
extension AsyncCompactedSequence.Iterator: Sendable
where
Base: Sendable, Base.Element: Sendable,
Base.AsyncIterator: Sendable { }
```
## Credits/Inspiration
This transformation function is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Compacted.md)
@@ -0,0 +1,75 @@
# Debounce
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Debounce/AsyncDebounceSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestDebounce.swift)
]
## Introduction
When events can potentially happen faster than the desired consumption rate, there are multiple ways to handle the situation. One approach is to only emit values after a given period of time of inactivity, or "quiescence", has elapsed. This algorithm is commonly referred to as debouncing.
## Proposed Solution
The debounce algorithm produces elements after a particular duration has passed between events. It transacts within a given tolerance applied to a clock. If values are produced by the base `AsyncSequence` during this quiet period, the debounce does not resume its next iterator until the period has elapsed with no values are produced or unless a terminal event is encountered.
The interface for this algorithm is available on all `AsyncSequence` types where the base type, iterator, and element are `Sendable`, since this algorithm will inherently create tasks to manage their timing of events. A shorthand implementation will be offered where the clock is the `ContinuousClock`, which allows for easy construction with `Duration` values.
```swift
extension AsyncSequence {
public func debounce<C: Clock>(
for interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
) -> AsyncDebounceSequence<Self, C>
public func debounce(
for interval: Duration,
tolerance: Duration? = nil
) -> AsyncDebounceSequence<Self, ContinuousClock>
}
```
This all boils down to a terse description of how to transform the asynchronous sequence over time.
```swift
fastEvents.debounce(for: .seconds(1))
```
In this case it transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second with no events to elapse before emitting a value.
## Detailed Design
The type that implements the algorithm for debounce emits the same element type as the base that it applies to. It also throws when the base type throws (and likewise does not throw when the base type does not throw).
```swift
public struct AsyncDebounceSequence<Base: AsyncSequence, C: Clock>: Sendable
where Base.AsyncIterator: Sendable, Base.Element: Sendable, Base: Sendable {
}
extension AsyncDebounceSequence: AsyncSequence {
public typealias Element = Base.Element
public struct Iterator: AsyncIteratorProtocol, Sendable {
public mutating func next() async rethrows -> Base.Element?
}
public func makeAsyncIterator() -> Iterator
}
```
Since the stored types comprising `AsyncDebounceSequence` must be `Sendable`; `AsyncDebounceSequence` is unconditionally always `Sendable`.
## Alternatives Considered
An alternative form of `debounce` could exist similar to the reductions of `throttle`, where a closure would be invoked for each value being set as the latest, and reducing a new value to produce for the debounce.
## Credits/Inspiration
The naming for debounce comes as a term of art; originally this term was inspired by electronic circuitry. When a physical switch closes a circuit it can easily have a "bouncing" behavior (also called chatter) that is caused by electrical contact resistance and the physical bounce of springs associated with switches. That phenomenon is often addressed with additional circuits to de-bounce (removing the bouncing) by ensuring a certain quiescence occurs.
http://reactivex.io/documentation/operators/debounce.html
https://developer.apple.com/documentation/combine/publishers/debounce/
@@ -0,0 +1,63 @@
# Effects
Lists the effects of all async algorithms.
| Type | Throws | Sendability |
|-----------------------------------------------------|--------------|-------------|
| `AsyncAdjacentPairsSequence` | rethrows | Conditional |
| `AsyncBufferedByteIterator` | throws | Not Sendable|
| `AsyncBufferSequence` | rethrows | Conditional |
| `AsyncBufferSequence.Iterator` | rethrows | Not Sendable|
| `AsyncChain2Sequence` | rethrows | Conditional |
| `AsyncChain2Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncChain3Sequence` | rethrows | Conditional |
| `AsyncChain3Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncChannel` | non-throwing | Sendable |
| `AsyncChannel.Iterator` | non-throwing | Not Sendable|
| `AsyncChunkedByGroupSequence` | rethrows | Conditional |
| `AsyncChunkedByGroupSequence.Iterator` | rethrows | Not Sendable|
| `AsyncChunkedOnProjectionSequence` | rethrows | Conditional |
| `AsyncChunkedOnProjectionSequence.Iterator` | rethrows | Not Sendable|
| `AsyncChunksOfCountOrSignalSequence` | rethrows | Sendable |
| `AsyncChunksOfCountOrSignalSequence.Iterator` | rethrows | Not Sendable|
| `AsyncChunksOfCountSequence` | rethrows | Conditional |
| `AsyncChunksOfCountSequence.Iterator` | rethrows | Not Sendable|
| `AsyncCombineLatest2Sequence` | rethrows | Sendable |
| `AsyncCombineLatest2Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncCombineLatest3Sequence` | rethrows | Sendable |
| `AsyncCombineLatest3Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncCompactedSequence` | rethrows | Conditional |
| `AsyncCompactedSequence.Iterator` | rethrows | Not Sendable|
| `AsyncDebounceSequence` | rethrows | Sendable |
| `AsyncDebounceSequence.Iterator` | rethrows | Not Sendable|
| `AsyncExclusiveReductionsSequence` | rethrows | Conditional |
| `AsyncExclusiveReductionsSequence.Iterator` | rethrows | Not Sendable|
| `AsyncInclusiveReductionsSequence` | rethrows | Conditional |
| `AsyncInclusiveReductionsSequence.Iterator` | rethrows | Not Sendable|
| `AsyncInterspersedSequence` | rethrows | Conditional |
| `AsyncInterspersedSequence.Iterator` | rethrows | Not Sendable|
| `AsyncJoinedSequence` | rethrows | Conditional |
| `AsyncJoinedSequence.Iterator` | rethrows | Not Sendable|
| `AsyncSyncSequence` | non-throwing | Conditional |
| `AsyncSyncSequence.Iterator` | non-throwing | Not Sendable|
| `AsyncLimitBuffer` | non-throwing | Sendable |
| `AsyncMerge2Sequence` | rethrows | Sendable |
| `AsyncMerge2Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncMerge3Sequence` | rethrows | Sendable |
| `AsyncMerge3Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncRemoveDuplicatesSequence` | rethrows | Conditional |
| `AsyncRemoveDuplicatesSequence.Iterator` | rethrows | Not Sendable|
| `AsyncThrottleSequence` | rethrows | Conditional |
| `AsyncThrottleSequence.Iterator` | rethrows | Not Sendable|
| `AsyncThrowingChannel` | throws | Sendable |
| `AsyncThrowingChannel.Iterator` | throws | Not Sendable|
| `AsyncThrowingExclusiveReductionsSequence` | throws | Conditional |
| `AsyncThrowingExclusiveReductionsSequence.Iterator` | throws | Not Sendable|
| `AsyncThrowingInclusiveReductionsSequence` | throws | Conditional |
| `AsyncThrowingInclusiveReductionsSequence.Iterator` | throws | Not Sendable|
| `AsyncTimerSequence` | non-throwing | Sendable |
| `AsyncTimerSequence.Iterator` | non-throwing | Not Sendable|
| `AsyncZip2Sequence` | rethrows | Sendable |
| `AsyncZip2Sequence.Iterator` | rethrows | Not Sendable|
| `AsyncZip3Sequence` | rethrows | Sendable |
| `AsyncZip3Sequence.Iterator` | rethrows | Not Sendable|
@@ -0,0 +1,139 @@
# FlatMapLatest
* Author(s): [Peter Friese](https://github.com/peterfriese)
Transforms elements into asynchronous sequences, emitting elements from only the most recent inner sequence.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/FlatMapLatest/AsyncFlatMapLatestSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestFlatMapLatest.swift)]
```swift
let searchQuery = AsyncStream<String> { continuation in
// User types into search field
continuation.yield("swi")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("swift")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("swift async")
continuation.finish()
}
let searchResults = searchQuery.flatMapLatest { query in
performSearch(query) // Returns AsyncSequence<SearchResult>
}
for try await result in searchResults {
print(result) // Only shows results from "swift async"
}
```
## Introduction
When transforming elements of an asynchronous sequence into new asynchronous sequences, you often want to abandon previous sequences when new data arrives. This is particularly useful for scenarios like search-as-you-type, where each keystroke triggers a new search request and you only care about results from the most recent query.
The `flatMapLatest` operator solves this by cancelling iteration on the previous inner sequence whenever a new element arrives from the outer sequence.
## Proposed Solution
The `flatMapLatest` algorithm transforms each element from the base `AsyncSequence` into a new inner `AsyncSequence` using the provided `transform` closure. When a new element is produced by the base sequence, iteration on the current inner sequence is cancelled, and iteration begins on the newly created sequence.
```swift
extension AsyncSequence where Self: Sendable {
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> AsyncFlatMapLatestSequence<Self, T>
}
```
This creates a concise way to express switching behavior:
```swift
userInput.flatMapLatest { input in
fetchDataFromNetwork(input)
}
```
In this case, each new user input cancels any ongoing network request and starts a fresh one, ensuring only the latest data is delivered.
## Detailed Design
The type that implements the algorithm emits elements from the inner sequences. It throws when either the base type or any inner sequence throws.
```swift
public struct AsyncFlatMapLatestSequence<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>: AsyncSequence, Sendable
where Base.Element: Sendable, Inner.Element: Sendable {
public typealias Element = Inner.Element
public struct Iterator: AsyncIteratorProtocol, Sendable {
public func next() async throws -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The implementation uses a state machine to ensure thread-safe operation and generation tracking to prevent stale values from cancelled sequences.
### Behavior
- **Switching**: When a new element arrives from the base sequence, the current inner sequence is cancelled immediately
- **Completion**: The sequence completes when the base sequence finishes and the final inner sequence completes
- **Error Handling**: Errors from either the base or inner sequences are propagated immediately
- **Cancellation**: Cancelling iteration cancels both the base and current inner sequence
## Use Cases
### Search as You Type
```swift
let searchField = AsyncStream<String> { continuation in
// Emit search queries as user types
continuation.yield("s")
continuation.yield("sw")
continuation.yield("swift")
}
let results = searchField.flatMapLatest { query in
searchAPI(for: query)
}
```
Only the results for "swift" will be emitted, as earlier queries are cancelled.
### Location-Based Data
```swift
let locationUpdates = CLLocationManager.shared.locationUpdates
let nearbyPlaces = locationUpdates.flatMapLatest { location in
fetchNearbyPlaces(at: location)
}
```
Each location update triggers a new search, cancelling any ongoing requests for previous locations.
### Dynamic Configuration
```swift
let settings = userSettingsStream
let data = settings.flatMapLatest { config in
loadData(with: config)
}
```
When settings change, data loading is restarted with the new configuration.
## Comparison with Other Operators
**`map`**: Transforms elements synchronously without producing sequences.
**`flatMap`** (if it existed): Would emit elements from all inner sequences concurrently, not cancelling previous ones.
**`switchToLatest`** (Combine): Equivalent operator in Combine framework - `flatMapLatest` is the `AsyncSequence` equivalent.
## Comparison with Other Libraries
**ReactiveX** ReactiveX has an [API definition of switchMap](https://reactivex.io/documentation/operators/flatmap.html) which performs the same operation for Observables.
**Combine** Combine has an [API definition of switchToLatest()](https://developer.apple.com/documentation/combine/publisher/switchtolatest()) which flattens a publisher of publishers by subscribing to the most recent one.
@@ -0,0 +1,60 @@
# Intersperse
Places a given value in between each element of the asynchronous sequence.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Interspersed/AsyncInterspersedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/Interspersed/TestInterspersed.swift)]
```swift
let numbers = [1, 2, 3].async.interspersed(with: 0)
for await number in numbers {
print(number)
}
// prints 1 0 2 0 3
let empty = [].async.interspersed(with: 0)
// await Array(empty) == []
```
`interspersed(with:)` takes a separator value and inserts it in between every
element in the asynchronous sequence.
## Detailed Design
A new method is added to `AsyncSequence`:
```swift
extension AsyncSequence {
func interspersed(with separator: Element) -> AsyncInterspersedSequence<Self>
}
```
The new `AsyncInterspersedSequence` type represents the asynchronous sequence
when the separator is inserted between each element.
When the base asynchronous sequence can throw on iteration, `AsyncInterspersedSequence`
will throw on iteration. When the base does not throw, the iteration of
`AsyncInterspersedSequence` does not throw either.
`AsyncInterspersedSequence` is conditionally `Sendable` when the base asynchronous
sequence is `Sendable` and the element is also `Sendable`.
### Naming
This method’s and type’s name match the term of art used in other languages
and libraries.
This method is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Intersperse.md).
### Comparison with other languages
**[Haskell][Haskell]:** Has an `intersperse` function which takes an element
and a list and 'intersperses' that element between the elements of the list.
**[Rust][Rust]:** Has a function called `intersperse` to insert a particular
value between each element.
<!-- Link references for other languages -->
[Haskell]: https://hackage.haskell.org/package/base-4.14.0.0/docs/Data-List.html#v:intersperse
[Rust]: https://docs.rs/itertools/0.9.0/itertools/trait.Itertools.html#method.intersperse
@@ -0,0 +1,127 @@
# Joined
* Author(s): [Kevin Perry](https://github.com/kperryua)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncJoinedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestJoin.swift)
]
## Introduction
Concatenates an asynchronous sequence of asynchronous sequences that share an `Element` type together sequentially where the elements from the resulting asynchronous sequence are comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs. Similar to `chain()`, except the number of asynchronous sequences to concatenate is not known up front.
Optionally allows inserting the elements of a separator asynchronous sequence in between each of the other sequences.
```swift
let sequenceOfURLs: AsyncSequence<URL> = ...
let sequenceOfLines = sequenceOfURLs.map { $0.lines }
let joinedWithSeparator = sequenceOfLines.joined(separator: ["===================="].async)
for try await lineOrSeparator in joinedWithSeparator {
print(lineOrSeparator)
}
```
This example shows how an `AsyncSequence` of `URL`s can be turned into an `AsyncSequence` of the lines of each of those files in sequence, with a separator line in between each file.
## Proposed Solution
```swift
extension AsyncSequence where Element: AsyncSequence {
public func joined() -> AsyncJoinedSequence<Self> {
return AsyncJoinedSequence(self)
}
}
```
```swift
extension AsyncSequence where Element: AsyncSequence {
public func joined<Separator: AsyncSequence>(separator: Separator) -> AsyncJoinedBySeparatorSequence<Self, Separator> {
return AsyncJoinedBySeparatorSequence(self, separator: separator)
}
}
```
## Detailed Design
```swift
public struct AsyncJoinedSequence<Base: AsyncSequence>: AsyncSequence where Base.Element: AsyncSequence {
public typealias Element = Base.Element.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Base.Element.Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncJoinedSequence: Sendable
where
Base: Sendable,
Base.Element: Sendable,
Base.Element.Element: Sendable,
Base.AsyncIterator: Sendable,
Base.Element.AsyncIterator: Sendable { }
extension AsyncJoinedSequence.Iterator: Sendable
where
Base: Sendable,
Base.Element: Sendable,
Base.Element.Element: Sendable,
Base.AsyncIterator: Sendable,
Base.Element.AsyncIterator: Sendable { }
```
```swift
public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: AsyncSequence>: AsyncSequence
where Base.Element: AsyncSequence, Separator.Element == Base.Element.Element {
public typealias Element = Base.Element.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Base.Element.Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncJoinedBySeparatorSequence: Sendable
where
Base: Sendable,
Base.Element: Sendable,
Base.Element.Element: Sendable,
Base.AsyncIterator: Sendable,
Separator: Sendable,
Separator.AsyncIterator: Sendable,
Base.Element.AsyncIterator: Sendable { }
extension AsyncJoinedBySeparatorSequence.Iterator: Sendable
where
Base: Sendable,
Base.Element: Sendable,
Base.Element.Element: Sendable,
Base.AsyncIterator: Sendable,
Separator: Sendable,
Separator.AsyncIterator: Sendable,
Base.Element.AsyncIterator: Sendable { }
```
The resulting `AsyncJoinedSequence` or `AsyncJoinedBySeparatorSequence` type is an asynchronous sequence, with conditional conformance to `Sendable` when the arguments conform.
When any of the asynchronous sequences being joined together come to their end of iteration, the `Joined` sequence iteration proceeds to the separator asynchronous sequence, if any. When the separator asynchronous sequence terminates, or if no separator was specified, it proceeds on to the next asynchronous sequence. When the last asynchronous sequence reaches the end of iteration the `AsyncJoinedSequence` or `AsyncJoinedBySeparatorSequence` then ends its iteration. At any point in time if one of the comprising asynchronous sequences ever throws an error during iteration the `AsyncJoinedSequence` or `AsyncJoinedBySeparatorSequence` iteration will throw that error and end iteration.
## Future Directions
The Swift Algorithms package has [additional synchronous variants of `joined()`](https://github.com/apple/swift-algorithms/blob/main/Guides/Joined.md). It is conceivable to bring asynchronous variants of those over to `AsyncSequence`.
The variant that takes a single element as a separator is straightforward, but can be trivially replicated with `[element].async`. However, it may be beneficial for performance to reimplement this directly.
There is another variant that takes a closure that allows one to customize the separator based on the return value of a closure. That closure is passed each of the two consecutive asynchronous sequences (not the two neighboring elements in consecutive sequences). This variant arguably has the greatest utility when the sequence type conforms to `Collection`, allowing either the `count` or any arbitrary element to be obtained directly. With `AsyncSequence`, it is less likely that this function with provide a similar level of utility, so it has been omitted.
## Alternatives Considered
Because `joined()` is essentially identical to `flatMap { $0 }`, it was considered that this should be called `flatten()` instead. However, it is preferable to follow the lead of the Swift standard library's `joined()` method.
## Credits/Inspiration
The Swift standard library has functions on synchronous sequences ([`joined()`](https://developer.apple.com/documentation/swift/sequence/1641166-joined), [`joined(separator:)`](https://developer.apple.com/documentation/swift/sequence/2431985-joined)) that perform similar (but synchronous) tasks.
@@ -0,0 +1,37 @@
# AsyncSyncSequence
This operation is available for all `Sequence` types.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncSyncSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestLazy.swift)]
```swift
let numbers = [1, 2, 3, 4].async
let characters = "abcde".async
```
This transformation can be useful to test operations specifically available on `AsyncSequence` but also is useful
to combine with other `AsyncSequence` types to provide well known sources of data.
## Detailed Design
The `.async` property returns an `AsyncSyncSequence` that is generic upon the base `Sequence` it was constructed from.
```swift
extension Sequence {
public var async: AsyncSyncSequence<Self> { get }
}
public struct AsyncSyncSequence<Base: Sequence>: AsyncSequence {
...
}
extension AsyncSyncSequence: Sendable where Base: Sendable { }
extension AsyncSyncSequence.Iterator: Sendable where Base.Iterator: Sendable { }
```
### Naming
This property's and type's name match the naming approaches in the Swift standard library. The property is named with a
succinct name in inspiration from `.lazy`, and the type is named in reference to the lazy behavior of the constructed
`AsyncSequence`.
@@ -0,0 +1,83 @@
# Merge
Merges two or more asynchronous sequences sharing the same element type into one singular asynchronous sequence.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Merge/AsyncMerge2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Merge/AsyncMerge3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestMerge.swift)]
```swift
let appleFeed = URL(string: "http://www.example.com/ticker?symbol=AAPL")!.lines.map { "AAPL: " + $0 }
let nasdaqFeed = URL(string:"http://www.example.com/ticker?symbol=^IXIC")!.lines.map { "^IXIC: " + $0 }
for try await ticker in merge(appleFeed, nasdaqFeed) {
print(ticker)
}
```
Given some sample inputs the following merged events can be expected.
| Timestamp | appleFeed | nasdaqFeed | merged output |
| ----------- | --------- | ---------- | --------------- |
| 11:40 AM | 173.91 | | AAPL: 173.91 |
| 12:25 AM | | 14236.78 | ^IXIC: 14236.78 |
| 12:40 AM | | 14218.34 | ^IXIC: 14218.34 |
| 1:15 PM | 173.00 | | AAPL: 173.00 |
## Detailed Design
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func merge<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncMerge2Sequence<Base1, Base2>
public func merge<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncMerge3Sequence<Base1, Base2, Base3>
public struct AsyncMerge2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Sendable
where
Base1.Element == Base2.Element,
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable, Base2.Element: Sendable,
Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
public struct AsyncMerge3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where
Base1.Element == Base2.Element, Base1.Element == Base3.Element,
Base1: Sendable, Base2: Sendable, Base3: Sendable
Base1.Element: Sendable, Base2.Element: Sendable, Base3.Element: Sendable
Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable, Base3.AsyncIterator: Sendable {
public typealias Element = Base1.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The `merge(_:...)` function takes two or more asynchronous sequences as arguments and produces an `AsyncMergeSequence` which is an asynchronous sequence.
Since the bases comprising the `AsyncMergeSequence` must be iterated concurrently to produce the latest value, those sequences must be able to be sent to child tasks. This means that a prerequisite of the bases must be that the base asynchronous sequences, their iterators, and the elements they produce must be `Sendable`.
When iterating a `AsyncMergeSequence`, the sequence terminates when all of the base asynchronous sequences terminate, since this means there is no potential for any further elements to be produced.
The throwing behavior of `AsyncMergeSequence` is that if any of the bases throw, then the composed asynchronous sequence throws on its iteration. If at any point an error is thrown by any base, the other iterations are cancelled and the thrown error is immediately thrown to the consuming iteration.
### Naming
Since the inherent behavior of `merge(_:...)` merges values from multiple streams into a singular asynchronous sequence, the naming is intended to be quite literal. There are precedent terms of art in other frameworks and libraries (listed in the comparison section). Other naming takes the form of "withLatestFrom". This was disregarded since the "with" prefix is often most associated with the passing of a closure and some sort of contextual concept; `withUnsafePointer` or `withUnsafeContinuation` are prime examples.
### Comparison with other libraries
**ReactiveX** ReactiveX has an [API definition of Merge](https://reactivex.io/documentation/operators/merge.html) as a top level function for merging Observables.
**Combine** Combine has an [API definition of merge(with:)](https://developer.apple.com/documentation/combine/publisher/merge(with:)-7qt71/) as an operator style method for merging Publishers.
@@ -0,0 +1,131 @@
# Reductions
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncExclusiveReductionsSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestReductions.swift)
]
## Introduction
The family of algorithms for reduce are useful for converting a sequence or asynchronous sequence into a single value, but that can elide important intermediate information. The _reductions_ algorithm is often called "scan", but this name does not convey its heritage to the family of reducing.
There are two strategies that are usable for creating continuous reductions: exclusive reductions and inclusive reductions:
* Exclusive reductions take a value and incorporate values into that initial value. A common example is reductions by appending to an array.
* Inclusive reductions transact only on the values provided. A common example is adding numbers.
## Proposed Solution
Exclusive reductions come in two variants: transforming by application, or transformation via mutation. This replicates the same interface as `reduce(_:_:)` and `reduce(into:_:)`. Unlike the `reduce` algorithms, the `reductions` algorithm also comes in two flavors: throwing or non throwing transformations.
```swift
extension AsyncSequence {
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async -> Result
) -> AsyncExclusiveReductionsSequence<Self, Result>
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async -> Void
) -> AsyncExclusiveReductionsSequence<Self, Result>
}
extension AsyncSequence {
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async throws -> Result
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result>
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async throws -> Void
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result>
}
```
These APIs can be used to reduce an initial value progressively or reduce into an initial value via mutation. In practice, a common use case for reductions is to mutate a collection by appending values.
```swift
characters.reductions(into: "") { $0.append($1) }
```
If the characters being produced asynchronously are `"a", "b", "c"`, then the iteration of the reductions is `"a", "ab", "abc"`.
Inclusive reductions do not have an initial value and therefore do not need an additional variations beyond the throwing and non throwing flavors.
```swift
extension AsyncSequence {
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async -> Element
) -> AsyncInclusiveReductionsSequence<Self>
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async throws -> Element
) -> AsyncThrowingInclusiveReductionsSequence<Self>
}
```
This is often used for scenarios like a running tally or other similar cases.
```swift
numbers.reductions { $0 + $1 }
```
In the above example, if the numbers are a sequence of `1, 2, 3, 4`, the produced values would be `1, 3, 6, 10`.
## Detailed Design
The exclusive reduction variants come in two distinct cases: non-throwing and throwing. These both have corresponding types to encompass that throwing behavior.
For non-throwing exclusive reductions, the element type of the sequence is the result of the reduction transform. `AsyncExclusiveReductionsSequence` will throw if the base asynchronous sequence throws, and will not throw if the base does not throws.
```swift
public struct AsyncExclusiveReductionsSequence<Base: AsyncSequence, Element> {
}
extension AsyncExclusiveReductionsSequence: AsyncSequence {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncExclusiveReductionsSequence: Sendable
where Base: Sendable, Element: Sendable { }
extension AsyncExclusiveReductionsSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable, Element: Sendable { }
```
The sendability behavior of `AsyncExclusiveReductionsSequence` is such that when the base, base iterator, and element are `Sendable` then `AsyncExclusiveReductionsSequence` is `Sendable`.
```swift
public struct AsyncThrowingExclusiveReductionsSequence<Base: AsyncSequence, Element> {
}
extension AsyncThrowingExclusiveReductionsSequence: AsyncSequence {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async throws -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncThrowingExclusiveReductionsSequence: Sendable
where Base: Sendable, Element: Sendable { }
extension AsyncThrowingExclusiveReductionsSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable, Element: Sendable { }
```
## Alternatives Considered
One alternate name for `reductions` was to name it `scan`; however the naming from the Swift Algorithms package offers considerably more inference to the heritage of what family of functions this algorithm belongs to.
## Credits/Inspiration
This transformation function is a direct analog to the synchronous version [defined in the Swift Algorithms package](https://github.com/apple/swift-algorithms/blob/main/Guides/Reductions.md)
@@ -0,0 +1,93 @@
# RemoveDuplicates
* Author(s): [Kevin Perry](https://github.com/kperryua)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncRemoveDuplicatesSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestRemoveDuplicates.swift)
]
## Introduction
When processing values over time it is definitely possible that the same value may occur in a row. When the distinctness of the presence value is not needed it is useful to consider the values over time that are differing from the last. Particularly this can be expressed as removing duplicate values either in the case as they are directly `Equatable` or by a predicate.
## Proposed Solution
The `removeDuplicates()` and `removeDuplicates(by:)` APIs serve this purpose of removing duplicate values that occur. These algorithms test against the previous value and if the latest iteration of the base `AsyncSequence` is the same as the last it invokes `next()` again. The resulting `AsyncRemoveDuplicatesSequence` will ensure that no duplicate values occur next to each other. This should not be confused with only emitting unique new values; where each value is tested against a collected set of values.
```swift
extension AsyncSequence where Element: Equatable {
public func removeDuplicates() -> AsyncRemoveDuplicatesSequence<Self>
}
extension AsyncSequence {
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async -> Bool
) -> AsyncRemoveDuplicatesSequence<Self>
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async throws -> Bool
) -> AsyncThrowingRemoveDuplicatesSequence<Self>
}
```
The `removeDuplicates` family comes in three variants. One variant is conditional upon the `Element` type being `Equatable`. This variation is a shorthand for writing `.removeDuplicates { $0 == $1 }`. The next variation is the closure version that allows for custom predicates to be applied. This algorithm allows for the cases where the elements themselves may not be equatable but portions of the element may be compared. Lastly is the variation that allows for comparison when the comparison method may throw.
## Detailed Design
In the cases where the `Element` type is `Equatable` or the non-trowing predicate variant these utilize the type `AsyncRemoveDuplicatesSequence`. The throwing predicate variant uses `AsyncThrowingRemoveDuplicatesSequence`. Both of these types are conditionally `Sendable` when the base, base element, and base iterator are `Sendable`
The `AsyncRemoveDuplicatesSequence` will rethrow if the base asynchronous sequence throws and will not throw if the base asynchronous sequence does not throw.
```swift
public struct AsyncRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator {
Iterator(iterator: base.makeAsyncIterator(), predicate: predicate)
}
}
extension AsyncRemoveDuplicatesSequence: Sendable
where Base: Sendable, Base.Element: Sendable, Base.AsyncIterator: Sendable { }
extension AsyncRemoveDuplicatesSequence.Iterator: Sendable
where Base: Sendable, Base.Element: Sendable, Base.AsyncIterator: Sendable { }
```
The `AsyncThrowingRemoveDuplicatesSequence` will rethrow if the base asynchronous sequence throws and still may throw if the base asynchronous sequence does not throw due to the predicate having the potential of throwing.
```swift
public struct AsyncThrowingRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async throws -> Element?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncThrowingRemoveDuplicatesSequence: Sendable
where Base: Sendable, Base.Element: Sendable, Base.AsyncIterator: Sendable { }
extension AsyncThrowingRemoveDuplicatesSequence.Iterator: Sendable
where Base: Sendable, Base.Element: Sendable, Base.AsyncIterator: Sendable { }
```
## Alternatives Considered
An alternative algorithm for uniqueness was considered but was discounted since it does not directly belong to this particular family of methods.
The name of this method could be considered to belong to the `filter` family and could refer directly to the repetitions. There is distinct merit to considering the name to be `filterRepetitions` or `removeRepetitions`. Likewise the terminology of `drop` also has merit.
## Credits/Inspiration
The Combine framework has a [function for publishers](https://developer.apple.com/documentation/combine/publisher/removeduplicates()/) that performs a similar task.
@@ -0,0 +1,194 @@
# Share
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncShareSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestShare.swift)
]
## Introduction
Many of the AsyncSequence adopting types only permit a one singular consumption. However there are many times that the same produced values are useful in more than one place. Out of that mechanism there are a few approaches to share, distribute, and broadcast those values. This proposal will focus on one concept; sharing. Sharing is where each consumption independently can make forward progress and get the same values but do not replay from the beginning of time.
## Motivation
There are many potential usages for the sharing concept of AsyncSequences.
One such example is the case where a source of data as an asynchronous sequence needs to be consumed by updating UI, logging, and additionally a network connection. This particular case does not matter on which uses but instead that those uses are independent of each other. It would not be expected for networking to block or delay the updates to UI, nor should logging. This example case also illustrates that the isolation of each side might be different and that some of the sides may not tolerate coalescing or dropping values.
There are many other use cases that have been requested for this family of algorithms. Since the release of AsyncAlgorithms it has perhaps been the most popularly requested set of behaviors as additions to the package.
## Proposed solution
AsyncAlgorithms will introduce a new extension function on AsyncSequence that will provide a shareable asynchronous sequence that will produce the same values upon iteration from multiple instances of it's AsyncIterator. Those iterations can take place in multiple isolations.
When values from a differing isolation cannot be coalesced, the two options available are either awaiting (an exertion of back-pressure across the sequences) or buffering (an internal back-pressure to a buffer). Replaying the values from the beginning of the creation of the sequence is a distinctly different behavior that should be considered a different use case. This then leaves the behavioral characteristic of this particular operation of share as; sharing a buffer of values started from the initialization of a new iteration of the sequence. Control over that buffer should then have options to determine the behavior, similar to how AsyncStream allows that control. It should have options to be unbounded, buffering the oldest count of elements, or buffering the newest count of elements.
It is critical to identify that this is one algorithm in the family of algorithms for sharing values. It should not attempt to solve all behavioral requirements but instead serve a common set of them that make cohesive sense together. This proposal is not mutually exclusive to the other algorithms in the sharing family.
## Detailed design
A new extension will be added to return a `Sendable` `AsyncSequence`. This extension will take a buffering policy to identify how the buffer will be handled when iterations do not consume at the same rate.
The `Sendable` annotation identifies to the developer that this sequence can be shared and stored in an existental `any`.
```swift
extension AsyncSequence where Element: Sendable {
public func share(
bufferingPolicy: AsyncBufferSequencePolicy = .bounded(1)
) -> some AsyncSequence<Element, Failure> & Sendable
}
```
The buffer internally to the share algorithm will only extend back to the furthest element available but there will only be a singular buffer shared across all iterators. This ensures that with the application of the buffering policy the storage size is as minimal as possible while still allowing all iterations to avoid dropping values and keeping the memory usage in check. The signature reuses the existing `AsyncBufferSequencePolicy` type to specify the behavior around buffering either responding to how it should limit emitting to the buffer or what should happen when the buffer is exceeded.
## Runtime Behavior
The runtime behaviors fall into a few categories; ordering, iteration isolation, cancellation, and lifetimes. To understand the behaviors there are a terms useful to define. Each creation of the AsyncIterator of the sequence and invocation of next will be referred to a side of the share iteration. The back pressure to the system to fetch a new element or termination is referred to as demand. The limit which is the pending gate for awaiting until the buffer has been serviced used for the `AsyncBufferSequencePolicy.bounded(_ : Int)` policy. The last special definition is that of the extent which is specifically in this case the lifetime of the asynchronous sequence itself.
When the underlying type backing the share algorithm is constructed a new extent is created; this is used for tracking the reference lifetime under the hood and is used to both house the iteration but also to identify the point at which no more sides can be constructed. When no more sides can be constructed and no sides are left to iterate then the backing iteration is canceled. This prevents any un-referenced task backing the iteration to not be leaked by the algorith itself.
That construction then creates an initial shared state and buffer. No task is started initially; it is only upon the first demand that the task backing the iteration is started; this means on the first call to next a task is spun up servicing all potential sides. The order of which the sides are serviced is not specified and cannot be relied upon, however the order of delivery within a side is always guarenteed to be ordered. The singular task servicing the iteration will be the only place holding any sort of iterator from the base `AsyncSequence`; so that iterator is isolated and not sent from one isolation to another. That iteration first awaits any limit availability and then awaits for a demand given by a side. After-which it then awaits an element or terminal event from the iterator and enqueues the elements to the buffer.
The buffer itself is only held in one location, each side however has a cursor index into that buffer and when values are consumed it adjusts the indexes accordingly; leaving the buffer usage only as big as the largest deficit. This means that new sides that are started post initial start up will not have a "replay" effect; that is a similar but distinct algorithm and is not addressed by this proposal. Any buffer size sensitive systems that wish to adjust behavior should be aware that specifying a policy is a suggested step. However in common usage similar to other such systems servicing desktop and mobile applications the common behavior is often unbounded. Alternatively desktop or mobile applications will often want `.bounded(1)` since that enforces the slowest consumption to drive the forward progress at most 1 buffered element. All of the use cases have a reasonable default of `.bounded(1)`; mobile, deskop, and server side uses. Leaving this as the default parameter keeps the progressive disclosure of the beahviors - such that the easiest thing to write is correct for all uses, and then more advanced control can be adjusted by passing in a specific policy. This default argument diverges slightly from AsyncStream, but follows a similar behavior to that of Combine's `share`.
As previously stated, the isolation of the iteration of the upstream/base AsyncSequence is to a detached task, this ensures that individual sides can have independent cancellation. Those cancellations will have the effect of remvoing that side from the shared iteration and cleaning up accordingly (including adjusting the trimming of the internal buffer).
Representing concurrent access is difficult to express all potential examples but there are a few cases included with this proposal to illustrate some of the behaviors. If a more comprehensive behavioral analysis is needed, it is strongly suggested to try out the pending pull request to identify how specific behaviors work. Please keep in mind that the odering between tasks is not specified, only the order within one side of iteration.
Practically this all means that a given iteration may be "behind" another and can eventually catch up (provided it is within the buffer limit).
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .unbounded)
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
This example will print a possible ordering of the following:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 3
Task 2 4
Task 1 1
Task 1 2
Task 1 3
Task 1 4
```
The order of the interleaving of the prints are not guaranteed; however the order of the elements per iteration is. Likewise in this buffering case it is guaranteed that all values are represented in the output.
If the creation were instead altered to the following:
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingLatest(2))
```
The output would print the possible ordering of:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 3
Task 1 4
```
Some values are dropped due to the buffering policy, but eventually they reach consistency. Which similarly works for the following:
```
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingOldest(2))
```
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 1
Task 1 2
```
However in this particular case the newest values are the dropped elements.
The `.bounded(N)` policy enforces consumption to prevent any side from being beyond a given amount away from other sides' consumption.
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bounded(1))
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
Will have a potential ordering output of:
```
Task 2 0
Task 2 1
Task 1 0
Task 1 1
Task 2 2
Task 1 2
Task 1 3
Task 1 4
Task 2 3
Task 2 4
```
In that example output Task 2 can get element 0 and 1 but must await until task 1 has caught up to the specified buffering. This limit means that no additional iteration (and no values are then dropped) is made until the buffer count is below the specified value.
## Effect on API resilience
This is an additive API and no existing systems are changed, however it will introduce a few new types that will need to be maintained as ABI interfaces. Since the intent of this is to provide a mechanism to store AsyncSequences to a shared context the type must be exposed as ABI (for type sizing).
## Alternatives considered
It has been considered that the buffering policy would be nested inside the `AsyncShareSequence` type. However since this seems to be something that will be useful for other types it makes sense to use an existing type from a top level type. However if it is determined that a general form of a buffering policy would require additional behaviors this might be a debatable placement to move back to an interior type similar to AsyncStream.
@@ -0,0 +1,90 @@
# Throttle
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncThrottleSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestThrottle.swift)
]
## Introduction
When events can potentially happen faster than the desired consumption rate, there are multiple ways to handle the situation. One approach is to emit values after a given period has elapsed. These emitted values can be reduced from the values encountered during the waiting period. This algorithm is commonly referred to as throttling.
## Proposed Solution
The throttle algorithm produces elements such that at least a specific interval has elapsed between them. It transacts by measuring against a specific clock. If values are produced by the base `AsyncSequence` the throttle does not resume its next iterator until the period has elapsed or unless a terminal event is encountered.
The interface for this algorithm is available on all `AsyncSequence` types. Unlike other algorithms like `debounce`, the throttle algorithm does not need to create additional tasks or require any sort of tolerance because the interval is just measured. A shorthand implementation will be offered in conjunction where the clock is the `ContinuousClock`, which allows for easy construction with `Duration` values. An additional shorthand is offered to reduce the values such that it provides a "latest" or "earliest" value, representing the leading or trailing edge of a throttled region of production of events.
```swift
extension AsyncSequence {
public func throttle<C: Clock, Reduced>(
for interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> AsyncThrottleSequence<Self, C, Reduced>
public func throttle<Reduced>(
for interval: Duration,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> AsyncThrottleSequence<Self, ContinuousClock, Reduced>
public func throttle<C: Clock>(
for interval: C.Instant.Duration,
clock: C,
latest: Bool = true
) -> AsyncThrottleSequence<Self, C, Element>
public func throttle(
for interval: Duration,
latest: Bool = true
) -> AsyncThrottleSequence<Self, ContinuousClock, Element>
}
```
This all boils down to a terse description of how to transform the asynchronous sequence over time.
```swift
fastEvents.throttle(for: .seconds(1))
```
In this case, the throttle transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second to elapse before emitting a value.
## Detailed Design
The type that implements the algorithm for throttle emits the same element type as the base that it applies to. It also throws when the base type throws (and likewise does not throw when the base type does not throw).
```swift
public struct AsyncThrottleSequence<Base: AsyncSequence, C: Clock, Reduced> {
}
extension AsyncThrottleSequence: AsyncSequence {
public typealias Element = Reduced
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Reduced?
}
public func makeAsyncIterator() -> Iterator
}
extension AsyncThrottleSequence: Sendable
where Base: Sendable, Element: Sendable { }
extension AsyncThrottleSequence.Iterator: Sendable
where Base.AsyncIterator: Sendable { }
```
The `AsyncThrottleSequence` and its `Iterator` are conditionally `Sendable` if the base types comprising it are `Sendable`.
The time in which events are measured are from the previous emission if present. If a duration has elapsed between the last emission and the point in time the throttle is measured then that duration is counted as elapsed. The first element is considered not throttled because no interval can be constructed from the start to the first element.
## Alternatives Considered
It was considered to only provide the "latest" style APIs, however the reduction version grants more flexibility and can act as a funnel to the implementations of `latest`.
## Credits/Inspiration
http://reactivex.io/documentation/operators/sample.html
https://developer.apple.com/documentation/combine/publishers/throttle/
@@ -0,0 +1,51 @@
# Timer
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncTimerSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestTimer.swift)
]
## Introduction
Producing elements at regular intervals can be useful for composing with other algorithms. These can range from invoking code at specific times to using those regular intervals as a delimiter of events. There are other cases this exists in APIs however those do not currently interact with Swift concurrency. These existing APIs are ones like `Timer` or `DispatchTimer` but are bound to internal clocks that are not extensible.
## Proposed Solution
We propose to add a new type; `AsyncTimerSequence` which utilizes the new `Clock`, `Instant` and `Duration` types. This allows the interaction of the timer to custom implementations of types adopting `Clock`.
This asynchronous sequence will produce elements of the clock's `Instant` type after the interval has elapsed. That instant will be the `now` at the time that the sleep has resumed. For each invocation to `next()` the `AsyncTimerSequence.Iterator` will calculate the next deadline to resume and pass that and the tolerance to the clock. If at any point in time the task executing that iteration is cancelled the iteration will return `nil` from the call to `next()`.
```swift
public struct AsyncTimerSequence<C: Clock>: AsyncSequence {
public typealias Element = C.Instant
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async -> C.Instant?
}
public init(
interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
)
public func makeAsyncIterator() -> Iterator
}
extension AsyncTimerSequence where C == SuspendingClock {
public static func repeating(every interval: Duration, tolerance: Duration? = nil) -> AsyncTimerSequence<SuspendingClock>
}
extension AsyncTimerSequence: Sendable { }
extension AsyncTimerSequence.Iterator: Sendable { }
```
Since all the types comprising `AsyncTimerSequence` are `Sendable` these types are also `Sendable`.
## Credits/Inspiration
https://developer.apple.com/documentation/foundation/timer
https://developer.apple.com/documentation/foundation/timer/timerpublisher
@@ -0,0 +1,85 @@
# Zip
Combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Zip/AsyncZip2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Zip/AsyncZip3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestZip.swift)]
```swift
let appleFeed = URL(string: "http://www.example.com/ticker?symbol=AAPL")!.lines
let nasdaqFeed = URL(string: "http://www.example.com/ticker?symbol=^IXIC")!.lines
for try await (apple, nasdaq) in zip(appleFeed, nasdaqFeed) {
print("APPL: \(apple) NASDAQ: \(nasdaq)")
}
```
Given some sample inputs the following zipped events can be expected.
| Timestamp | appleFeed | nasdaqFeed | combined output |
| ----------- | --------- | ---------- | ----------------------------- |
| 11:40 AM | 173.91 | | |
| 12:25 AM | | 14236.78 | AAPL: 173.91 NASDAQ: 14236.78 |
| 12:40 AM | | 14218.34 | |
| 1:15 PM | 173.00 | | AAPL: 173.00 NASDAQ: 14218.34 |
## Detailed Design
This function family and the associated family of return types are prime candidates for variadic generics. Until that proposal is accepted, these will be implemented in terms of two- and three-base sequence cases.
```swift
public func zip<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncZip2Sequence<Base1, Base2>
public func zip<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncZip3Sequence<Base1, Base2, Base3>
public struct AsyncZip2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable, Base2.Element: Sendable,
Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable {
public typealias Element = (Base1.Element, Base2.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
public struct AsyncZip3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where
Base1: Sendable, Base2: Sendable, Base3: Sendable
Base1.Element: Sendable, Base2.Element: Sendable, Base3.Element: Sendable
Base1.AsyncIterator: Sendable, Base2.AsyncIterator: Sendable, Base3.AsyncIterator: Sendable {
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async rethrows -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The `zip(_:...)` function takes two or more asynchronous sequences as arguments with the resulting `AsyncZipSequence` which is an asynchronous sequence.
Each iteration of an `AsyncZipSequence` will await for all base iterators to produce a value. This iteration will be done concurrently to produce a singular tuple result. If any of the base iterations terminates by returning `nil` from its iteration, the `AsyncZipSequence` iteration is immediately considered unsatisfiable and returns `nil` and all iterations of other bases will be cancelled. If any iteration of the bases throws an error, then the other iterations concurrently running are cancelled and the produced error is rethrown, terminating the iteration.
`AsyncZipSequence` requires that the iterations are done concurrently. This means that the base sequences, their elements, and iterators must all be `Sendable`. That makes `AsyncZipSequence` inherently `Sendable`.
The source of throwing of `AsyncZipSequence` is determined by its bases. That means that if any base can throw an error then the iteration of the `AsyncZipSequence` can throw. If no bases can throw, then the `AsyncZipSequence` does not throw.
### Naming
The `zip(_:...)` function takes its name from the Swift standard library function of the same name. The `AsyncZipSequence` family of types take their name from the same family from the standard library for the type returned by `zip(_:_:)`. The one difference is that this asynchronous version allows for the affordance of recognizing the eventual variadic generic need of expanding a zip of more than just two sources.
It is common in some libraries to have a `ZipMap` or some other combination of `zip` and `map`. This is a common usage pattern, but leaving a singular type for composition feels considerably more approachable.
### Comparison with other libraries
**Swift** The swift standard library has an [API definition of zip](https://developer.apple.com/documentation/swift/1541125-zip) as a top level function for combining two sequences.
**ReactiveX** ReactiveX has an [API definition of Zip](https://reactivex.io/documentation/operators/zip.html) as a top level function for combining Observables.
**Combine** Combine has an [API definition of zip](https://developer.apple.com/documentation/combine/publisher/zip(_:)/) as an operator style method for combining Publishers.
@@ -0,0 +1,143 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// An `AsyncIterator` that provides a convenient and high-performance implementation
/// of a common architecture for `AsyncSequence` of `UInt8`, otherwise known as byte streams.
///
/// Bytes are read into an internal buffer of `capacity` bytes via the
/// `readFunction`. Invoking `next()` returns bytes from the internal buffer until it's
/// empty, and then suspends and awaits another invocation of `readFunction` to
/// refill. If `readFunction` returns 0 (indicating nothing was read), `next()` will
/// return `nil` from then on. Cancellation is checked before each invocation of
/// `readFunction`, which means that many calls to `next()` will not check for
/// cancellation.
///
/// A typical use of `AsyncBufferedByteIterator` looks something like this:
///
/// struct AsyncBytes: AsyncSequence {
/// public typealias Element = UInt8
/// var handle: ReadableThing
///
/// internal init(_ readable: ReadableThing) {
/// handle = readable
/// }
///
/// public func makeAsyncIterator() -> AsyncBufferedByteIterator {
/// return AsyncBufferedByteIterator(capacity: 16384) { buffer in
/// // This runs once every 16384 invocations of next()
/// return try await handle.read(into: buffer)
/// }
/// }
/// }
///
///
@available(AsyncAlgorithms 1.0, *)
public struct AsyncBufferedByteIterator: AsyncIteratorProtocol {
public typealias Element = UInt8
@usableFromInline var buffer: _AsyncBytesBuffer
/// Creates an asynchronous buffered byte iterator with a specified capacity and read function.
///
/// - Parameters:
/// - capacity: The maximum number of bytes that a single invocation of `readFunction` may produce.
/// This is the allocated capacity of the backing buffer for iteration; the value must be greater than 0.
/// - readFunction: The function for refilling the buffer.
public init(
capacity: Int,
readFunction: @Sendable @escaping (UnsafeMutableRawBufferPointer) async throws -> Int
) {
buffer = _AsyncBytesBuffer(capacity: capacity, readFunction: readFunction)
}
/// Reads a byte out of the buffer if available. When no bytes are available, this will trigger
/// the read function to reload the buffer and then return the next byte from that buffer.
@inlinable @inline(__always)
public mutating func next() async throws -> UInt8? {
return try await buffer.next()
}
}
@available(*, unavailable)
extension AsyncBufferedByteIterator: Sendable {}
@available(AsyncAlgorithms 1.0, *)
@frozen @usableFromInline
internal struct _AsyncBytesBuffer {
@usableFromInline
final class Storage {
fileprivate let buffer: UnsafeMutableRawBufferPointer
init(
capacity: Int
) {
precondition(capacity > 0)
buffer = UnsafeMutableRawBufferPointer.allocate(
byteCount: capacity,
alignment: MemoryLayout<AnyObject>.alignment
)
}
deinit {
buffer.deallocate()
}
}
@usableFromInline internal let storage: Storage
@usableFromInline internal var nextPointer: UnsafeRawPointer
@usableFromInline internal var endPointer: UnsafeRawPointer
internal let readFunction: @Sendable (UnsafeMutableRawBufferPointer) async throws -> Int
internal var finished = false
@usableFromInline init(
capacity: Int,
readFunction: @Sendable @escaping (UnsafeMutableRawBufferPointer) async throws -> Int
) {
let s = Storage(capacity: capacity)
self.readFunction = readFunction
storage = s
nextPointer = UnsafeRawPointer(s.buffer.baseAddress!)
endPointer = nextPointer
}
@inline(never) @usableFromInline
internal mutating func reloadBufferAndNext() async throws -> UInt8? {
if finished {
return nil
}
try Task.checkCancellation()
do {
let readSize: Int = try await readFunction(storage.buffer)
if readSize == 0 {
finished = true
nextPointer = endPointer
return nil
}
nextPointer = UnsafeRawPointer(storage.buffer.baseAddress!)
endPointer = nextPointer + readSize
} catch {
finished = true
nextPointer = endPointer
throw error
}
return try await next()
}
@inlinable @inline(__always)
internal mutating func next() async throws -> UInt8? {
if _fastPath(nextPointer != endPointer) {
let byte = nextPointer.load(fromByteOffset: 0, as: UInt8.self)
nextPointer = nextPointer + 1
return byte
}
return try await reloadBufferAndNext()
}
}
@@ -0,0 +1,94 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// Returns a new asynchronous sequence that iterates over the two given asynchronous sequences, one
/// followed by the other.
///
/// - Parameters:
/// - s1: The first asynchronous sequence.
/// - s2: The second asynchronous sequence.
/// - Returns: An asynchronous sequence that iterates first over the elements of `s1`, and
/// then over the elements of `s2`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chain<Base1: AsyncSequence, Base2: AsyncSequence>(
_ s1: Base1,
_ s2: Base2
) -> AsyncChain2Sequence<Base1, Base2> where Base1.Element == Base2.Element {
AsyncChain2Sequence(s1, s2)
}
/// A concatenation of two asynchronous sequences with the same element type.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncChain2Sequence<Base1: AsyncSequence, Base2: AsyncSequence> where Base1.Element == Base2.Element {
@usableFromInline
let base1: Base1
@usableFromInline
let base2: Base2
@usableFromInline
init(_ base1: Base1, _ base2: Base2) {
self.base1 = base1
self.base2 = base2
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain2Sequence: AsyncSequence {
public typealias Element = Base1.Element
/// The iterator for a `AsyncChain2Sequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base1: Base1.AsyncIterator?
@usableFromInline
var base2: Base2.AsyncIterator?
@usableFromInline
init(_ base1: Base1.AsyncIterator, _ base2: Base2.AsyncIterator) {
self.base1 = base1
self.base2 = base2
}
@inlinable
public mutating func next() async rethrows -> Element? {
do {
if let value = try await base1?.next() {
return value
} else {
base1 = nil
}
return try await base2?.next()
} catch {
base1 = nil
base2 = nil
throw error
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base1.makeAsyncIterator(), base2.makeAsyncIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain2Sequence: Sendable where Base1: Sendable, Base2: Sendable {}
@available(*, unavailable)
extension AsyncChain2Sequence.Iterator: Sendable {}
@@ -0,0 +1,111 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// Returns a new asynchronous sequence that iterates over the three given asynchronous sequences, one
/// followed by the other.
///
/// - Parameters:
/// - s1: The first asynchronous sequence.
/// - s2: The second asynchronous sequence.
/// - s3: The third asynchronous sequence.
/// - Returns: An asynchronous sequence that iterates first over the elements of `s1`, and
/// then over the elements of `s2`, and then over the elements of `s3`
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chain<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(
_ s1: Base1,
_ s2: Base2,
_ s3: Base3
) -> AsyncChain3Sequence<Base1, Base2, Base3> {
AsyncChain3Sequence(s1, s2, s3)
}
/// A concatenation of three asynchronous sequences with the same element type.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncChain3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>
where Base1.Element == Base2.Element, Base1.Element == Base3.Element {
@usableFromInline
let base1: Base1
@usableFromInline
let base2: Base2
@usableFromInline
let base3: Base3
@usableFromInline
init(_ base1: Base1, _ base2: Base2, _ base3: Base3) {
self.base1 = base1
self.base2 = base2
self.base3 = base3
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain3Sequence: AsyncSequence {
public typealias Element = Base1.Element
/// The iterator for a `AsyncChain3Sequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base1: Base1.AsyncIterator?
@usableFromInline
var base2: Base2.AsyncIterator?
@usableFromInline
var base3: Base3.AsyncIterator?
@usableFromInline
init(_ base1: Base1.AsyncIterator, _ base2: Base2.AsyncIterator, _ base3: Base3.AsyncIterator) {
self.base1 = base1
self.base2 = base2
self.base3 = base3
}
@inlinable
public mutating func next() async rethrows -> Element? {
do {
if let value = try await base1?.next() {
return value
} else {
base1 = nil
}
if let value = try await base2?.next() {
return value
} else {
base2 = nil
}
return try await base3?.next()
} catch {
base1 = nil
base2 = nil
base3 = nil
throw error
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base1.makeAsyncIterator(), base2.makeAsyncIterator(), base3.makeAsyncIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain3Sequence: Sendable where Base1: Sendable, Base2: Sendable, Base3: Sendable {}
@available(*, unavailable)
extension AsyncChain3Sequence.Iterator: Sendable {}
@@ -0,0 +1,132 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2021 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection`
/// type by testing if elements belong in the same group.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked<Collected: RangeReplaceableCollection>(
into: Collected.Type,
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, Collected> where Collected.Element == Element {
AsyncChunkedByGroupSequence(self, grouping: belongInSameGroup)
}
/// Creates an asynchronous sequence that creates chunks by testing if elements belong in the same group.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked(
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, [Element]> {
chunked(into: [Element].self, by: belongInSameGroup)
}
}
/// An `AsyncSequence` that chunks by testing if two elements belong to the same group.
///
/// Group chunks are determined by passing two consecutive elements to a closure which tests
/// whether they are in the same group. When the `AsyncChunkedByGroupSequence` iterator
/// receives the first element from the base sequence, it will immediately be added to a group. When
/// it receives the second item, it tests whether the previous item and the current item belong to the
/// same group. If they are not in the same group, then the iterator emits the first item's group and a
/// new group is created containing the second item. Items declared to be in the same group
/// accumulate until a new group is declared, or the iterator finds the end of the base sequence.
/// When the base sequence terminates, the final group is emitted. If the base sequence throws an
/// error, `AsyncChunkedByGroupSequence` will rethrow that error immediately and discard
/// any current group.
///
/// let numbers = [10, 20, 30, 10, 40, 40, 10, 20].async
/// let chunks = numbers.chunked { $0 <= $1 }
/// for await numberChunk in chunks {
/// print(numberChunk)
/// }
/// // prints
/// // [10, 20, 30]
/// // [10, 40, 40]
/// // [10, 20]
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
/// The iterator for a `AsyncChunkedByGroupSequence` instance.
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base: Base.AsyncIterator
@usableFromInline
let grouping: @Sendable (Base.Element, Base.Element) -> Bool
@usableFromInline
init(base: Base.AsyncIterator, grouping: @escaping @Sendable (Base.Element, Base.Element) -> Bool) {
self.base = base
self.grouping = grouping
}
@usableFromInline
var hangingNext: Base.Element?
@inlinable
public mutating func next() async rethrows -> Collected? {
var firstOpt = hangingNext
if firstOpt == nil {
firstOpt = try await base.next()
} else {
hangingNext = nil
}
guard let first = firstOpt else {
return nil
}
var result: Collected = .init()
result.append(first)
var prev = first
while let next = try await base.next() {
guard grouping(prev, next) else {
hangingNext = next
break
}
result.append(next)
prev = next
}
return result
}
}
@usableFromInline
let base: Base
@usableFromInline
let grouping: @Sendable (Base.Element, Base.Element) -> Bool
@usableFromInline
init(_ base: Base, grouping: @escaping @Sendable (Base.Element, Base.Element) -> Bool) {
self.base = base
self.grouping = grouping
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base: base.makeAsyncIterator(), grouping: grouping)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunkedByGroupSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncChunkedByGroupSequence.Iterator: Sendable {}
@@ -0,0 +1,115 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2021 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type on the uniqueness of a given subject.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked<Subject: Equatable, Collected: RangeReplaceableCollection>(
into: Collected.Type,
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, Collected> {
AsyncChunkedOnProjectionSequence(self, projection: projection)
}
/// Creates an asynchronous sequence that creates chunks on the uniqueness of a given subject.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked<Subject: Equatable>(
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, [Element]> {
chunked(into: [Element].self, on: projection)
}
}
/// An `AsyncSequence` that chunks on a subject when it differs from the last element.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunkedOnProjectionSequence<
Base: AsyncSequence,
Subject: Equatable,
Collected: RangeReplaceableCollection
>: AsyncSequence where Collected.Element == Base.Element {
public typealias Element = (Subject, Collected)
/// The iterator for a `AsyncChunkedOnProjectionSequence` instance.
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base: Base.AsyncIterator
@usableFromInline
let projection: @Sendable (Base.Element) -> Subject
@usableFromInline
init(base: Base.AsyncIterator, projection: @escaping @Sendable (Base.Element) -> Subject) {
self.base = base
self.projection = projection
}
@usableFromInline
var hangingNext: (Subject, Base.Element)?
@inlinable
public mutating func next() async rethrows -> (Subject, Collected)? {
var firstOpt = hangingNext
if firstOpt == nil {
let nextOpt = try await base.next()
if let next = nextOpt {
firstOpt = (projection(next), next)
}
} else {
hangingNext = nil
}
guard let first = firstOpt else {
return nil
}
var result: Collected = .init()
result.append(first.1)
while let next = try await base.next() {
let subj = projection(next)
guard subj == first.0 else {
hangingNext = (subj, next)
break
}
result.append(next)
}
return (first.0, result)
}
}
@usableFromInline
let base: Base
@usableFromInline
let projection: @Sendable (Base.Element) -> Subject
@usableFromInline
init(_ base: Base, projection: @escaping @Sendable (Base.Element) -> Subject) {
self.base = base
self.projection = projection
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base: base.makeAsyncIterator(), projection: projection)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunkedOnProjectionSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncChunkedOnProjectionSequence.Iterator: Sendable {}
@@ -0,0 +1,178 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2021 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type of a given count or when a signal `AsyncSequence` produces an element.
@available(AsyncAlgorithms 1.0, *)
public func chunks<Signal, Collected: RangeReplaceableCollection>(
ofCount count: Int,
or signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: count, signal: signal)
}
/// Creates an asynchronous sequence that creates chunks of a given count or when a signal `AsyncSequence` produces an element.
@available(AsyncAlgorithms 1.0, *)
public func chunks<Signal>(
ofCount count: Int,
or signal: Signal
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal> {
chunks(ofCount: count, or: signal, into: [Element].self)
}
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type when a signal `AsyncSequence` produces an element.
@available(AsyncAlgorithms 1.0, *)
public func chunked<Signal, Collected: RangeReplaceableCollection>(
by signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: nil, signal: signal)
}
/// Creates an asynchronous sequence that creates chunks when a signal `AsyncSequence` produces an element.
@available(AsyncAlgorithms 1.0, *)
public func chunked<Signal>(by signal: Signal) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal> {
chunked(by: signal, into: [Element].self)
}
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type of a given count or when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunks<C: Clock, Collected: RangeReplaceableCollection>(
ofCount count: Int,
or timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: count, signal: timer)
}
/// Creates an asynchronous sequence that creates chunks of a given count or when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunks<C: Clock>(
ofCount count: Int,
or timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>> {
chunks(ofCount: count, or: timer, into: [Element].self)
}
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(
by timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: nil, signal: timer)
}
/// Creates an asynchronous sequence that creates chunks when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunked<C: Clock>(
by timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>> {
chunked(by: timer, into: [Element].self)
}
}
/// An `AsyncSequence` that chunks elements into collected `RangeReplaceableCollection` instances by either count or a signal from another `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunksOfCountOrSignalSequence<
Base: AsyncSequence,
Collected: RangeReplaceableCollection,
Signal: AsyncSequence
>: AsyncSequence, Sendable
where
Collected.Element == Base.Element,
Base: Sendable,
Signal: Sendable,
Base.Element: Sendable,
Signal.Element: Sendable
{
public typealias Element = Collected
enum Either {
case element(Base.Element)
case terminal
case signal
}
/// The iterator for a `AsyncChunksOfCountOrSignalSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
typealias EitherMappedBase = AsyncMapSequence<Base, Either>
typealias EitherMappedSignal = AsyncMapSequence<Signal, Either>
typealias ChainedBase = AsyncChain2Sequence<EitherMappedBase, AsyncSyncSequence<[Either]>>
typealias Merged = AsyncMerge2Sequence<ChainedBase, EitherMappedSignal>
let count: Int?
var iterator: Merged.AsyncIterator
var terminated = false
init(iterator: Merged.AsyncIterator, count: Int?) {
self.count = count
self.iterator = iterator
}
public mutating func next() async rethrows -> Collected? {
guard !terminated else {
return nil
}
var result: Collected?
while let next = try await iterator.next() {
switch next {
case .element(let element):
if result == nil {
result = Collected()
}
result!.append(element)
if result?.count == count {
return result
}
case .terminal:
terminated = true
return result
case .signal:
if result != nil {
return result
}
}
}
return result
}
}
let base: Base
let signal: Signal
let count: Int?
init(_ base: Base, count: Int?, signal: Signal) {
if let count = count {
precondition(count > 0)
}
self.base = base
self.count = count
self.signal = signal
}
public func makeAsyncIterator() -> Iterator {
return Iterator(
iterator: merge(
chain(base.map { Either.element($0) }, [.terminal].async),
signal.map { _ in Either.signal }
).makeAsyncIterator(),
count: count
)
}
}
@available(*, unavailable)
extension AsyncChunksOfCountOrSignalSequence.Iterator: Sendable {}
@@ -0,0 +1,99 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2021 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` of a given count.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunks<Collected: RangeReplaceableCollection>(
ofCount count: Int,
into: Collected.Type
) -> AsyncChunksOfCountSequence<Self, Collected> where Collected.Element == Element {
AsyncChunksOfCountSequence(self, count: count)
}
/// Creates an asynchronous sequence that creates chunks of a given count.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunks(ofCount count: Int) -> AsyncChunksOfCountSequence<Self, [Element]> {
chunks(ofCount: count, into: [Element].self)
}
}
/// An `AsyncSequence` that chunks elements into `RangeReplaceableCollection` instances of at least a given count.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
/// The iterator for a `AsyncChunksOfCountSequence` instance.
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base: Base.AsyncIterator
@usableFromInline
let count: Int
@usableFromInline
init(base: Base.AsyncIterator, count: Int) {
self.base = base
self.count = count
}
@inlinable
public mutating func next() async rethrows -> Collected? {
guard let first = try await base.next() else {
return nil
}
if count == 1 {
return Collected(CollectionOfOne(first))
}
var result: Collected = .init()
result.append(first)
while let next = try await base.next() {
result.append(next)
if result.count == count {
break
}
}
return result
}
}
@usableFromInline
let base: Base
@usableFromInline
let count: Int
@usableFromInline
init(_ base: Base, count: Int) {
precondition(count > 0)
self.base = base
self.count = count
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base: base.makeAsyncIterator(), count: count)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunksOfCountSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunksOfCountSequence.Iterator: Sendable where Base.AsyncIterator: Sendable, Base.Element: Sendable {}
@@ -0,0 +1,76 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns a new `AsyncSequence` that iterates over every non-nil element from the
/// original `AsyncSequence`.
///
/// Produces the same result as `c.compactMap { $0 }`.
///
/// - Returns: An `AsyncSequence` where the element is the unwrapped original
/// element and iterates over every non-nil element from the original
/// `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func compacted<Unwrapped>() -> AsyncCompactedSequence<Self, Unwrapped>
where Element == Unwrapped? {
AsyncCompactedSequence(self)
}
}
/// An `AsyncSequence` that iterates over every non-nil element from the original
/// `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncCompactedSequence<Base: AsyncSequence, Element>: AsyncSequence
where Base.Element == Element? {
@usableFromInline
let base: Base
@inlinable
init(_ base: Base) {
self.base = base
}
/// The iterator for an `AsyncCompactedSequence` instance.
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base: Base.AsyncIterator
@inlinable
init(_ base: Base.AsyncIterator) {
self.base = base
}
@inlinable
public mutating func next() async rethrows -> Element? {
while let wrapped = try await base.next() {
guard let some = wrapped else { continue }
return some
}
return nil
}
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncCompactedSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncCompactedSequence.Iterator: Sendable {}
@@ -0,0 +1,139 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given closure.
///
/// Starting from the initial value, this combines each element of the base
/// asynchronous sequence with the running result and emits each intermediate
/// result. The initial value itself is not emitted, so the first element of
/// the returned sequence is the result of combining the initial value with
/// the first element of the base sequence. If the base sequence is empty, the
/// returned sequence is also empty.
///
/// - Parameters:
/// - initial: The value to use as the initial value.
/// - transform: A closure that combines the previously-reduced result and
/// the next element in the receiving asynchronous sequence, which it returns.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async -> Result
) -> AsyncExclusiveReductionsSequence<Self, Result> {
reductions(into: initial) { result, element in
result = await transform(result, element)
}
}
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given closure.
///
/// Starting from the initial value, this combines each element of the base
/// asynchronous sequence with the running result and emits each intermediate
/// result. The initial value itself is not emitted, so the first element of
/// the returned sequence is the result of combining the initial value with
/// the first element of the base sequence. If the base sequence is empty, the
/// returned sequence is also empty.
///
/// - Parameters:
/// - initial: The value to use as the initial value.
/// - transform: A closure that combines the previously-reduced result and
/// the next element in the receiving asynchronous sequence, mutating the
/// previous result instead of returning a value.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async -> Void
) -> AsyncExclusiveReductionsSequence<Self, Result> {
AsyncExclusiveReductionsSequence(self, initial: initial, transform: transform)
}
}
/// An asynchronous sequence of applying a transform to the element of an asynchronous sequence and the
/// previously transformed result.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncExclusiveReductionsSequence<Base: AsyncSequence, Element> {
@usableFromInline
let base: Base
@usableFromInline
let initial: Element
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async -> Void
@inlinable
init(_ base: Base, initial: Element, transform: @Sendable @escaping (inout Element, Base.Element) async -> Void) {
self.base = base
self.initial = initial
self.transform = transform
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncExclusiveReductionsSequence: AsyncSequence {
/// The iterator for an `AsyncExclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.AsyncIterator
@usableFromInline
var current: Element?
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async -> Void
@inlinable
init(
_ iterator: Base.AsyncIterator,
initial: Element,
transform: @Sendable @escaping (inout Element, Base.Element) async -> Void
) {
self.iterator = iterator
self.current = initial
self.transform = transform
}
@inlinable
public mutating func next() async rethrows -> Element? {
guard var result = current else { return nil }
let value = try await iterator.next()
guard let value = value else {
current = nil
return nil
}
await transform(&result, value)
current = result
return result
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), initial: initial, transform: transform)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncExclusiveReductionsSequence: Sendable where Base: Sendable, Element: Sendable {}
@available(*, unavailable)
extension AsyncExclusiveReductionsSequence.Iterator: Sendable {}
@@ -0,0 +1,108 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given closure.
///
/// This can be seen as applying the reduce function to each element and
/// producing an asynchronous sequence consisting of the initial value followed
/// by these results.
///
/// ```
/// let runningTotal = [1, 2, 3, 4].async.reductions(+)
/// print(await Array(runningTotal))
///
/// // prints [1, 3, 6, 10]
/// ```
///
/// - Parameter transform: A closure that combines the previously reduced
/// result and the next element in the receiving sequence, and returns
/// the result.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async -> Element
) -> AsyncInclusiveReductionsSequence<Self> {
AsyncInclusiveReductionsSequence(self, transform: transform)
}
}
/// An asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using a given closure.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncInclusiveReductionsSequence<Base: AsyncSequence> {
@usableFromInline
let base: Base
@usableFromInline
let transform: @Sendable (Base.Element, Base.Element) async -> Base.Element
@inlinable
init(_ base: Base, transform: @Sendable @escaping (Base.Element, Base.Element) async -> Base.Element) {
self.base = base
self.transform = transform
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncInclusiveReductionsSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal var iterator: Base.AsyncIterator
@usableFromInline
internal var element: Base.Element?
@usableFromInline
internal let transform: @Sendable (Base.Element, Base.Element) async -> Base.Element
@inlinable
init(
_ iterator: Base.AsyncIterator,
transform: @Sendable @escaping (Base.Element, Base.Element) async -> Base.Element
) {
self.iterator = iterator
self.transform = transform
}
@inlinable
public mutating func next() async rethrows -> Base.Element? {
guard let previous = element else {
element = try await iterator.next()
return element
}
guard let next = try await iterator.next() else { return nil }
element = await transform(previous, next)
return element
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), transform: transform)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncInclusiveReductionsSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncInclusiveReductionsSequence.Iterator: Sendable {}
@@ -0,0 +1,154 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Element: AsyncSequence {
/// Concatenate an `AsyncSequence` of `AsyncSequence` elements with a separator.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func joined<Separator: AsyncSequence>(
separator: Separator
) -> AsyncJoinedBySeparatorSequence<Self, Separator> {
return AsyncJoinedBySeparatorSequence(self, separator: separator)
}
}
/// An `AsyncSequence` that concatenates `AsyncSequence` elements with a separator.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: AsyncSequence>: AsyncSequence
where Base.Element: AsyncSequence, Separator.Element == Base.Element.Element {
public typealias Element = Base.Element.Element
public typealias AsyncIterator = Iterator
/// The iterator for an `AsyncJoinedBySeparatorSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
enum State {
@usableFromInline
enum SeparatorState {
case initial(Separator)
case partialAsync(Separator.AsyncIterator, ContiguousArray<Element>)
case cached(ContiguousArray<Element>)
case partialCached(ContiguousArray<Element>.Iterator, ContiguousArray<Element>)
@usableFromInline
func startSeparator() -> SeparatorState {
switch self {
case .initial(let separatorSequence):
return .partialAsync(separatorSequence.makeAsyncIterator(), [])
case .cached(let array):
return .partialCached(array.makeIterator(), array)
default:
fatalError("Invalid separator sequence state")
}
}
@usableFromInline
func next() async rethrows -> (Element?, SeparatorState) {
switch self {
case .partialAsync(var separatorIterator, var cache):
guard let next = try await separatorIterator.next() else {
return (nil, .cached(cache))
}
cache.append(next)
return (next, .partialAsync(separatorIterator, cache))
case .partialCached(var cacheIterator, let cache):
guard let next = cacheIterator.next() else {
return (nil, .cached(cache))
}
return (next, .partialCached(cacheIterator, cache))
default:
fatalError("Invalid separator sequence state")
}
}
}
case initial(Base.AsyncIterator, Separator)
case sequence(Base.AsyncIterator, Base.Element.AsyncIterator, SeparatorState)
case separator(Base.AsyncIterator, SeparatorState, Base.Element)
case terminal
}
@usableFromInline
var state: State
@usableFromInline
init(_ iterator: Base.AsyncIterator, separator: Separator) {
state = .initial(iterator, separator)
}
@inlinable
public mutating func next() async rethrows -> Base.Element.Element? {
do {
switch state {
case .terminal:
return nil
case .initial(var outerIterator, let separator):
guard let innerSequence = try await outerIterator.next() else {
state = .terminal
return nil
}
let innerIterator = innerSequence.makeAsyncIterator()
state = .sequence(outerIterator, innerIterator, .initial(separator))
return try await next()
case .sequence(var outerIterator, var innerIterator, let separatorState):
if let item = try await innerIterator.next() {
state = .sequence(outerIterator, innerIterator, separatorState)
return item
}
guard let nextInner = try await outerIterator.next() else {
state = .terminal
return nil
}
state = .separator(outerIterator, separatorState.startSeparator(), nextInner)
return try await next()
case .separator(let iterator, let separatorState, let nextBase):
let (itemOpt, newSepState) = try await separatorState.next()
guard let item = itemOpt else {
state = .sequence(iterator, nextBase.makeAsyncIterator(), newSepState)
return try await next()
}
state = .separator(iterator, newSepState, nextBase)
return item
}
} catch {
state = .terminal
throw error
}
}
}
@usableFromInline
let base: Base
@usableFromInline
let separator: Separator
@usableFromInline
init(_ base: Base, separator: Separator) {
self.base = base
self.separator = separator
}
@inlinable
public func makeAsyncIterator() -> Iterator {
return Iterator(base.makeAsyncIterator(), separator: separator)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncJoinedBySeparatorSequence: Sendable
where Base: Sendable, Base.Element: Sendable, Base.Element.Element: Sendable, Separator: Sendable {}
@available(*, unavailable)
extension AsyncJoinedBySeparatorSequence.Iterator: Sendable {}
@@ -0,0 +1,101 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Element: AsyncSequence {
/// Concatenate an `AsyncSequence` of `AsyncSequence` elements
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func joined() -> AsyncJoinedSequence<Self> {
return AsyncJoinedSequence(self)
}
}
/// An `AsyncSequence` that concatenates`AsyncSequence` elements
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncJoinedSequence<Base: AsyncSequence>: AsyncSequence where Base.Element: AsyncSequence {
public typealias Element = Base.Element.Element
public typealias AsyncIterator = Iterator
/// The iterator for an `AsyncJoinedSequence` instance.
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
enum State {
case initial(Base.AsyncIterator)
case sequence(Base.AsyncIterator, Base.Element.AsyncIterator)
case terminal
}
@usableFromInline
var state: State
@inlinable
init(_ iterator: Base.AsyncIterator) {
state = .initial(iterator)
}
@inlinable
public mutating func next() async rethrows -> Base.Element.Element? {
do {
switch state {
case .terminal:
return nil
case .initial(var outerIterator):
guard let innerSequence = try await outerIterator.next() else {
state = .terminal
return nil
}
let innerIterator = innerSequence.makeAsyncIterator()
state = .sequence(outerIterator, innerIterator)
return try await next()
case .sequence(var outerIterator, var innerIterator):
if let item = try await innerIterator.next() {
state = .sequence(outerIterator, innerIterator)
return item
}
guard let nextInner = try await outerIterator.next() else {
state = .terminal
return nil
}
state = .sequence(outerIterator, nextInner.makeAsyncIterator())
return try await next()
}
} catch {
state = .terminal
throw error
}
}
}
@usableFromInline
let base: Base
@usableFromInline
init(_ base: Base) {
self.base = base
}
@inlinable
public func makeAsyncIterator() -> Iterator {
return Iterator(base.makeAsyncIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncJoinedSequence: Sendable
where Base: Sendable, Base.Element: Sendable, Base.Element.Element: Sendable {}
@available(*, unavailable)
extension AsyncJoinedSequence.Iterator: Sendable {}
@@ -0,0 +1,106 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.0)
@available(AsyncAlgorithms 1.2, *)
extension AsyncSequence {
/// Converts any failure into a new error.
///
/// - Parameter transform: A closure that takes the failure as a parameter and returns a new error.
/// - Returns: An asynchronous sequence that maps the error thrown into the one produced by the transform closure.
///
/// Use the ``mapError(_:)`` operator when you need to replace one error type with another.
@available(AsyncAlgorithms 1.2, *)
public func mapError<MappedError: Error>(
_ transform: @Sendable @escaping (Self.Failure) -> MappedError
) -> some AsyncSequence<Self.Element, MappedError> {
AsyncMapErrorSequence(base: self, transform: transform)
}
/// Converts any failure into a new error.
///
/// - Parameter transform: A closure that takes the failure as a parameter and returns a new error.
/// - Returns: An asynchronous sequence that maps the error thrown into the one produced by the transform closure.
///
/// Use the ``mapError(_:)`` operator when you need to replace one error type with another.
@available(AsyncAlgorithms 1.2, *)
public func mapError<MappedError: Error>(
_ transform: @Sendable @escaping (Self.Failure) -> MappedError
) -> (some AsyncSequence<Self.Element, MappedError> & Sendable) where Self: Sendable, Self.Element: Sendable {
AsyncMapErrorSequence(base: self, transform: transform)
}
}
/// An asynchronous sequence that converts any failure into a new error.
@available(AsyncAlgorithms 1.2, *)
struct AsyncMapErrorSequence<Base: AsyncSequence, MappedError: Error> {
typealias Element = Base.Element
typealias Failure = MappedError
private let base: Base
private let transform: @Sendable (Base.Failure) async -> MappedError
init(
base: Base,
transform: @Sendable @escaping (Base.Failure) async -> MappedError
) {
self.base = base
self.transform = transform
}
}
@available(AsyncAlgorithms 1.2, *)
extension AsyncMapErrorSequence: AsyncSequence {
/// The iterator that produces elements of the map sequence.
struct Iterator: AsyncIteratorProtocol {
typealias Element = Base.Element
private var base: Base.AsyncIterator
private let transform: @Sendable (Base.Failure) async -> MappedError
init(
base: Base.AsyncIterator,
transform: @Sendable @escaping (Base.Failure) async -> MappedError
) {
self.base = base
self.transform = transform
}
mutating func next() async throws(MappedError) -> Element? {
try await self.next(isolation: nil)
}
mutating func next(isolation actor: isolated (any Actor)?) async throws(MappedError) -> Element? {
do {
return try await base.next(isolation: actor)
} catch {
throw await transform(error)
}
}
}
func makeAsyncIterator() -> Iterator {
Iterator(
base: base.makeAsyncIterator(),
transform: transform
)
}
}
@available(AsyncAlgorithms 1.2, *)
extension AsyncMapErrorSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncMapErrorSequence.Iterator: Sendable {}
#endif
@@ -0,0 +1,163 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Element: Equatable {
/// Creates an asynchronous sequence that omits repeated elements.
@available(AsyncAlgorithms 1.0, *)
public func removeDuplicates() -> AsyncRemoveDuplicatesSequence<Self> {
AsyncRemoveDuplicatesSequence(self) { lhs, rhs in
lhs == rhs
}
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that omits repeated elements by testing them with a predicate.
@available(AsyncAlgorithms 1.0, *)
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async -> Bool
) -> AsyncRemoveDuplicatesSequence<Self> {
return AsyncRemoveDuplicatesSequence(self, predicate: predicate)
}
/// Creates an asynchronous sequence that omits repeated elements by testing them with an error-throwing predicate.
@available(AsyncAlgorithms 1.0, *)
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async throws -> Bool
) -> AsyncThrowingRemoveDuplicatesSequence<Self> {
return AsyncThrowingRemoveDuplicatesSequence(self, predicate: predicate)
}
}
/// An asynchronous sequence that omits repeated elements by testing them with a predicate.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncRemoveDuplicatesSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.AsyncIterator
@usableFromInline
let predicate: @Sendable (Element, Element) async -> Bool
@usableFromInline
var last: Element?
@usableFromInline
init(iterator: Base.AsyncIterator, predicate: @escaping @Sendable (Element, Element) async -> Bool) {
self.iterator = iterator
self.predicate = predicate
}
@inlinable
public mutating func next() async rethrows -> Element? {
guard let last = last else {
last = try await iterator.next()
return last
}
while let element = try await iterator.next() {
if await !predicate(last, element) {
self.last = element
return element
}
}
return nil
}
}
@usableFromInline
let base: Base
@usableFromInline
let predicate: @Sendable (Element, Element) async -> Bool
init(_ base: Base, predicate: @escaping @Sendable (Element, Element) async -> Bool) {
self.base = base
self.predicate = predicate
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(iterator: base.makeAsyncIterator(), predicate: predicate)
}
}
/// An asynchronous sequence that omits repeated elements by testing them with an error-throwing predicate.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncThrowingRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncThrowingRemoveDuplicatesSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.AsyncIterator
@usableFromInline
let predicate: @Sendable (Element, Element) async throws -> Bool
@usableFromInline
var last: Element?
@usableFromInline
init(iterator: Base.AsyncIterator, predicate: @escaping @Sendable (Element, Element) async throws -> Bool) {
self.iterator = iterator
self.predicate = predicate
}
@inlinable
public mutating func next() async throws -> Element? {
guard let last = last else {
last = try await iterator.next()
return last
}
while let element = try await iterator.next() {
if try await !predicate(last, element) {
self.last = element
return element
}
}
return nil
}
}
@usableFromInline
let base: Base
@usableFromInline
let predicate: @Sendable (Element, Element) async throws -> Bool
init(_ base: Base, predicate: @escaping @Sendable (Element, Element) async throws -> Bool) {
self.base = base
self.predicate = predicate
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(iterator: base.makeAsyncIterator(), predicate: predicate)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncRemoveDuplicatesSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingRemoveDuplicatesSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncRemoveDuplicatesSequence.Iterator: Sendable {}
@available(*, unavailable)
extension AsyncThrowingRemoveDuplicatesSequence.Iterator: Sendable {}
@@ -0,0 +1,726 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.2)
import Synchronization
import DequeModule
@available(AsyncAlgorithms 1.1, *)
extension AsyncSequence
where Element: Sendable, Self: SendableMetatype, AsyncIterator: SendableMetatype {
/// Creates a shared async sequence that allows multiple concurrent iterations over a single source.
///
/// The `share` method transforms an async sequence into a shareable sequence that can be safely
/// iterated by multiple concurrent tasks. This is useful when you want to broadcast elements from
/// a single source to multiple consumers without duplicating work or creating separate iterations.
///
/// Each element from the source sequence is delivered to all active iterators.
/// Elements are buffered according to the specified buffering policy to handle timing differences
/// between consumers.
///
/// The base sequence is iterated in it's own task to ensure that cancellation is not polluted from
/// one side of iteration to another.
///
/// ## Example Usage
///
/// ```swift
/// let numbers = [1, 2, 3, 4, 5].async.map {
/// try? await Task.sleep(for: .seconds(1))
/// return $0
/// }
///
/// let shared = numbers.share()
///
/// // Multiple tasks can iterate concurrently
/// let consumer1 = Task {
/// for await value in shared {
/// print("Consumer 1: \(value)")
/// }
/// }
///
/// let consumer2 = Task {
/// for await value in shared {
/// print("Consumer 2: \(value)")
/// }
/// }
///
/// await consumer1.value
/// await consumer2.value
/// ```
///
/// - Parameter bufferingPolicy: The policy controlling how elements are enqueued to the shared buffer. Defaults to `.bounded(1)`.
/// - `.bounded(n)`: Limits the buffer to `n` elements, applying backpressure to the source when that limit is reached
/// - `.bufferingOldest(n)`: Keeps the oldest `n` elements, discarding newer ones when full
/// - `.bufferingNewest(n)`: Keeps the newest `n` elements, discarding older ones when full
/// - `.unbounded`: Allows unlimited buffering (use with caution)
///
/// - Returns: A sendable async sequence that can be safely shared across multiple concurrent tasks.
///
public func share(
bufferingPolicy: AsyncBufferSequencePolicy = .bounded(1)
) -> some AsyncSequence<Element, Failure> & Sendable {
// The iterator is transferred to the isolation of the iterating task
// this has to be done "unsafely" since we cannot annotate the transfer
// however since iterating an AsyncSequence types twice has been defined
// as invalid and one creation of the iterator is virtually a consuming
// operation so this is safe at runtime.
// The general principal of `.share()` is to provide a mecahnism for non-
// shared AsyncSequence types to be shared. The parlance for those is
// that the base AsyncSequence type is not Sendable. If the iterator
// is not marked as `nonisolated(unsafe)` the compiler will claim that
// the value is "Capture of 'iterator' with non-Sendable type 'Self.AsyncIterator' in a '@Sendable' closure;"
// Since the closure returns a disconnected non-sendable value there is no
// distinct problem here and the compiler just needs to be informed
// that the diagnostic is overly pessimistic.
nonisolated(unsafe) let iterator = makeAsyncIterator()
return AsyncShareSequence<Self>(
{
iterator
},
bufferingPolicy: bufferingPolicy
)
}
}
// An async sequence that enables safe concurrent sharing of a single source sequence.
//
// `AsyncShareSequence` wraps a base async sequence and allows multiple concurrent iterators
// to consume elements from the same source. It handles all the complexity of coordinating
// between multiple consumers, buffering elements, and managing the lifecycle of the underlying
// iteration.
//
// ## Key Features
//
// **Single Source Iteration**: The base sequence's iterator is created and consumed only once
// **Concurrent Safe**: Multiple tasks can safely iterate simultaneously
// **Configurable Buffering**: Supports various buffering strategies for different use cases
// **Automatic Cleanup**: Properly manages resources and cancellation across all consumers
//
// ## Internal Architecture
//
// The implementation uses several key components:
// `Side`: Represents a single consumer's iteration state
// `Iteration`: Coordinates all consumers and manages the shared buffer
// `Extent`: Manages the overall lifecycle and cleanup
//
// This type is typically not used directly; instead, use the `share()` method on any
// async sequence that meets the sendability requirements.
@available(AsyncAlgorithms 1.1, *)
struct AsyncShareSequence<Base: AsyncSequence>: Sendable
where Base.Element: Sendable, Base: SendableMetatype, Base.AsyncIterator: SendableMetatype {
// Represents a single consumer's connection to the shared sequence.
//
// Each iterator of the shared sequence creates its own `Side` instance, which tracks
// that consumer's position in the shared buffer and manages its continuation for
// async iteration. The `Side` automatically registers itself with the central
// `Iteration` coordinator and cleans up when deallocated.
//
// ## Lifecycle
//
// **Creation**: Automatically registers with the iteration coordinator
// **Usage**: Tracks buffer position and manages async continuations
// **Cleanup**: Automatically unregisters and cancels pending operations on deinit
final class Side {
// Tracks the state of a single consumer's iteration.
//
// - `continuation`: The continuation waiting for the next element (nil if not waiting)
// - `position`: The consumer's current position in the shared buffer
struct State {
var continuation: UnsafeContinuation<Result<Element?, Failure>, Never>?
var position = 0
// Creates a new state with the position adjusted by the given offset.
//
// This is used when the shared buffer is trimmed to maintain correct
// relative positioning for this consumer.
//
// - Parameter adjustment: The number of positions to subtract from the current position
// - Returns: A new `State` with the adjusted position
func offset(_ adjustment: Int) -> State {
State(continuation: continuation, position: position - adjustment)
}
}
let iteration: Iteration
let id: Int
init(_ iteration: Iteration) {
self.iteration = iteration
id = iteration.registerSide()
}
deinit {
iteration.unregisterSide(id)
}
func next(isolation actor: isolated (any Actor)?) async throws(Failure) -> Element? {
try await iteration.next(isolation: actor, id: id)
}
}
// The central coordinator that manages the shared iteration state.
//
// `Iteration` is responsible for:
// Managing the single background task that consumes the source sequence
// Coordinating between multiple consumer sides
// Buffering elements according to the specified policy
// Handling backpressure and flow control
// Managing cancellation and cleanup
//
// ## Thread Safety
//
// All operations are synchronized using a `Mutex` to ensure thread-safe access
// to the shared state across multiple concurrent consumers.
final class Iteration: Sendable {
// Represents the state of the background task that consumes the source sequence.
//
// The iteration task goes through several states during its lifecycle:
// `pending`: Initial state, holds the factory to create the iterator
// `starting`: Transitional state while the task is being created
// `running`: Active state with a running background task
// `cancelled`: Terminal state when the iteration has been cancelled
enum IteratingTask {
case pending(@Sendable () -> sending Base.AsyncIterator)
case starting
case running(Task<Void, Never>)
case cancelled
var isStarting: Bool {
switch self {
case .starting: true
default: false
}
}
func cancel() {
switch self {
case .running(let task):
task.cancel()
default:
break
}
}
}
// The complete shared state for coordinating all aspects of the shared iteration.
//
// This state is protected by a mutex and contains all the information needed
// to coordinate between multiple consumers, manage buffering, and control
// the background iteration task.
struct State: Sendable {
// Defines how elements are stored and potentially discarded in the shared buffer.
//
// `unbounded`: Store all elements without limit (may cause memory growth)
// `bufferingOldest(Int)`: Keep only the oldest N elements, ignore newer ones when full
// `bufferingNewest(Int)`: Keep only the newest N elements, discard older ones when full
enum StoragePolicy: Sendable {
case unbounded
case bufferingOldest(Int)
case bufferingNewest(Int)
}
var generation = 0
var sides = [Int: Side.State]()
var iteratingTask: IteratingTask
private(set) var buffer = Deque<Element>()
private(set) var finished = false
private(set) var failure: Failure?
var cancelled = false
var limit: UnsafeContinuation<Bool, Never>?
var demand: UnsafeContinuation<Void, Never>?
let storagePolicy: StoragePolicy
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
self.iteratingTask = .pending(iteratorFactory)
switch bufferingPolicy.policy {
case .bounded: self.storagePolicy = .unbounded
case .bufferingOldest(let bound): self.storagePolicy = .bufferingOldest(bound)
case .bufferingNewest(let bound): self.storagePolicy = .bufferingNewest(bound)
case .unbounded: self.storagePolicy = .unbounded
}
}
// Removes elements from the front of the buffer that all consumers have already processed.
//
// This method finds the minimum position across all active consumers and removes
// that many elements from the front of the buffer. It then adjusts all consumer
// positions to account for the removed elements, maintaining their relative positions.
//
// This optimization prevents the buffer from growing indefinitely when all consumers
// are keeping pace with each other.
mutating func trimBuffer() {
if let minimumIndex = sides.values.map({ $0.position }).min(), minimumIndex > 0 {
buffer.removeFirst(minimumIndex)
sides = sides.mapValues {
$0.offset(minimumIndex)
}
}
}
// Private state machine transitions for the emission of a given value.
//
// This method ensures the continuations are properly consumed when emitting values
// and returns those continuations for resumption.
private mutating func _emit<T>(
_ value: T,
limit: Int
) -> (T, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) {
let belowLimit = buffer.count < limit || limit == 0
defer {
if belowLimit {
self.limit = nil
}
demand = nil
}
guard case .cancelled = iteratingTask else {
return (value, belowLimit ? self.limit : nil, demand, false)
}
return (value, belowLimit ? self.limit : nil, demand, true)
}
// Internal state machine transitions for the emission of a given value.
//
// This method ensures the continuations are properly consumed when emitting values
// and returns those continuations for resumption.
//
// If no limit is specified it interprets that as an unbounded limit.
mutating func emit<T>(
_ value: T,
limit: Int?
) -> (T, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) {
return _emit(value, limit: limit ?? .max)
}
// Adds an element to the buffer according to the configured storage policy.
//
// The behavior depends on the storage policy:
// **Unbounded**: Always appends the element
// **Buffering Oldest**: Appends only if under the limit, otherwise ignores the element
// **Buffering Newest**: Appends if under the limit, otherwise removes the oldest and appends
//
// - Parameter element: The element to add to the buffer
mutating func enqueue(_ element: Element) {
let count = buffer.count
switch storagePolicy {
case .unbounded:
buffer.append(element)
case .bufferingOldest(let limit):
if count < limit {
buffer.append(element)
}
case .bufferingNewest(let limit):
if count < limit {
buffer.append(element)
} else if count > 0 {
buffer.removeFirst()
buffer.append(element)
}
}
}
mutating func finish() {
finished = true
}
mutating func fail(_ error: Failure) {
finished = true
failure = error
}
}
let state: Mutex<State>
let limit: Int?
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
state = Mutex(State(iteratorFactory, bufferingPolicy: bufferingPolicy))
switch bufferingPolicy.policy {
case .bounded(let limit):
self.limit = limit
default:
self.limit = nil
}
}
func cancel() {
let (task, limitContinuation, demand, cancelled) = state.withLock {
state -> (IteratingTask?, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) in
guard state.sides.count == 0 else {
state.cancelled = true
return state.emit(nil, limit: limit)
}
defer {
state.iteratingTask = .cancelled
state.cancelled = true
}
return state.emit(state.iteratingTask, limit: limit)
}
task?.cancel()
limitContinuation?.resume(returning: cancelled)
demand?.resume()
}
func registerSide() -> Int {
state.withLock { state in
defer { state.generation += 1 }
state.sides[state.generation] = Side.State()
return state.generation
}
}
func unregisterSide(_ id: Int) {
let (side, continuation, cancelled, iteratingTaskToCancel) = state.withLock {
state -> (Side.State?, UnsafeContinuation<Bool, Never>?, Bool, IteratingTask?) in
let side = state.sides.removeValue(forKey: id)
state.trimBuffer()
let cancelRequested = state.sides.count == 0 && state.cancelled
guard let limit, state.buffer.count < limit else {
guard case .cancelled = state.iteratingTask else {
defer {
if cancelRequested {
state.iteratingTask = .cancelled
}
}
return (side, nil, false, cancelRequested ? state.iteratingTask : nil)
}
return (side, nil, true, nil)
}
defer { state.limit = nil }
guard case .cancelled = state.iteratingTask else {
defer {
if cancelRequested {
state.iteratingTask = .cancelled
}
}
return (side, state.limit, false, cancelRequested ? state.iteratingTask : nil)
}
return (side, state.limit, true, nil)
}
if let continuation {
continuation.resume(returning: cancelled)
}
if let side {
side.continuation?.resume(returning: .success(nil))
}
if let iteratingTaskToCancel {
iteratingTaskToCancel.cancel()
}
}
func iterate() async -> Bool {
if let limit, limit > 0 {
let cancelled = await withUnsafeContinuation { (continuation: UnsafeContinuation<Bool, Never>) in
let (resume, cancelled) = state.withLock { state -> (UnsafeContinuation<Bool, Never>?, Bool) in
guard state.buffer.count >= limit else {
assert(state.limit == nil)
guard case .cancelled = state.iteratingTask else {
return (continuation, false)
}
return (continuation, true)
}
state.limit = continuation
guard case .cancelled = state.iteratingTask else {
return (nil, false)
}
return (nil, true)
}
if let resume {
resume.resume(returning: cancelled)
}
}
if cancelled {
return false
}
}
// await a demand
await withUnsafeContinuation { (continuation: UnsafeContinuation<Void, Never>) in
let hasPendingDemand = state.withLock { state in
for (_, side) in state.sides {
if side.continuation != nil {
return true
}
}
state.demand = continuation
return false
}
if hasPendingDemand {
continuation.resume()
}
}
return state.withLock { state in
switch state.iteratingTask {
case .cancelled:
return false
default:
return true
}
}
}
func cancel(id: Int) {
unregisterSide(id) // doubly unregistering is idempotent but has a side effect of emitting nil if present
}
struct Resumption {
let continuation: UnsafeContinuation<Result<Element?, Failure>, Never>
let result: Result<Element?, Failure>
func resume() {
continuation.resume(returning: result)
}
}
func emit(_ result: Result<Element?, Failure>) {
let (resumptions, limitContinuation, demandContinuation, cancelled) = state.withLock {
state -> ([Resumption], UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) in
var resumptions = [Resumption]()
switch result {
case .success(let element):
if let element {
state.enqueue(element)
} else {
state.finish()
}
case .failure(let failure):
state.fail(failure)
}
for (id, side) in state.sides {
if let continuation = side.continuation {
if side.position < state.buffer.count {
resumptions.append(Resumption(continuation: continuation, result: .success(state.buffer[side.position])))
state.sides[id]?.position += 1
state.sides[id]?.continuation = nil
} else if state.finished {
state.sides[id]?.continuation = nil
if let failure = state.failure {
resumptions.append(Resumption(continuation: continuation, result: .failure(failure)))
} else {
resumptions.append(Resumption(continuation: continuation, result: .success(nil)))
}
}
}
}
state.trimBuffer()
return state.emit(resumptions, limit: limit)
}
if let limitContinuation {
limitContinuation.resume(returning: cancelled)
}
if let demandContinuation {
demandContinuation.resume()
}
for resumption in resumptions {
resumption.resume()
}
}
private func nextIteration(
_ id: Int
) async -> Result<AsyncShareSequence<Base>.Element?, AsyncShareSequence<Base>.Failure> {
return await withTaskCancellationHandler {
await withUnsafeContinuation { continuation in
let (res, limitContinuation, demandContinuation, cancelled) = state.withLock {
state -> (
Result<Element?, Failure>?, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool
) in
guard let side = state.sides[id] else {
return state.emit(.success(nil), limit: limit)
}
if side.position < state.buffer.count {
// There's an element available at this position
let element = state.buffer[side.position]
state.sides[id]?.position += 1
state.trimBuffer()
return state.emit(.success(element), limit: limit)
} else {
// Position is beyond the buffer
if let failure = state.failure {
return state.emit(.failure(failure), limit: limit)
} else if state.finished {
return state.emit(.success(nil), limit: limit)
} else {
state.sides[id]?.continuation = continuation
return state.emit(nil, limit: limit)
}
}
}
if let limitContinuation {
limitContinuation.resume(returning: cancelled)
}
if let demandContinuation {
demandContinuation.resume()
}
if let res {
continuation.resume(returning: res)
}
}
} onCancel: {
cancel(id: id)
}
}
private func iterationLoop(factory: @Sendable () -> sending Base.AsyncIterator) async {
var iterator = factory()
do {
while await iterate() {
if let element = try await iterator.next() {
emit(.success(element))
} else {
emit(.success(nil))
}
}
} catch {
emit(.failure(error as! Failure))
}
}
func next(isolation actor: isolated (any Actor)?, id: Int) async throws(Failure) -> Element? {
let (factory, cancelled) = state.withLock { state -> ((@Sendable () -> sending Base.AsyncIterator)?, Bool) in
switch state.iteratingTask {
case .pending(let factory):
state.iteratingTask = .starting
return (factory, false)
case .cancelled:
return (nil, true)
default:
return (nil, false)
}
}
if cancelled { return nil }
if let factory {
let task: Task<Void, Never>
// for the fancy dance of availability and canImport see the comment on the next check for details
#if swift(>=6.2)
if #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) {
task = Task(name: "Share Iteration") { [factory, self] in
await iterationLoop(factory: factory)
}
} else {
task = Task.detached(name: "Share Iteration") { [factory, self] in
await iterationLoop(factory: factory)
}
}
#else
task = Task.detached { [factory, self] in
await iterationLoop(factory: factory)
}
#endif
// Known Issue: there is a very small race where the task may not get a priority escalation during startup
// this unfortuantely cannot be avoided since the task should ideally not be formed within the critical
// region of the state. Since that could lead to potential deadlocks in low-core-count systems.
// That window is relatively small and can be revisited if a suitable proof of safe behavior can be
// determined.
state.withLock { state in
precondition(state.iteratingTask.isStarting)
state.iteratingTask = .running(task)
}
}
// withTaskPriorityEscalationHandler is only available for the '26 releases and the 6.2 version of
// the _Concurrency library. This menas for Darwin based OSes we have to have a fallback at runtime,
// and for non-darwin OSes we need to verify against the ability to import that version.
// Using this priority escalation means that the base task can avoid being detached.
//
// This is disabled for now until the 9999 availability is removed from `withTaskPriorityEscalationHandler`
#if false // TODO: remove when this is resolved
guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else {
return try await nextIteration(id).get()
}
return try await withTaskPriorityEscalationHandler {
return await nextIteration(id)
} onPriorityEscalated: { old, new in
let task = state.withLock { state -> Task<Void, Never>? in
switch state.iteratingTask {
case .running(let task):
return task
default:
return nil
}
}
task?.escalatePriority(to: new)
}.get()
#else
return try await nextIteration(id).get()
#endif
}
}
// Manages the lifecycle of the shared iteration.
//
// `Extent` serves as the ownership boundary for the shared sequence. When the
// `AsyncShareSequence` itself is deallocated, the `Extent` ensures that the
// background iteration task is properly cancelled and all resources are cleaned up.
//
// This design allows multiple iterators to safely reference the same underlying
// iteration coordinator while ensuring proper cleanup when the shared sequence
// is no longer needed.
final class Extent: Sendable {
let iteration: Iteration
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
iteration = Iteration(iteratorFactory, bufferingPolicy: bufferingPolicy)
}
deinit {
iteration.cancel()
}
}
let extent: Extent
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
extent = Extent(iteratorFactory, bufferingPolicy: bufferingPolicy)
}
}
@available(AsyncAlgorithms 1.1, *)
extension AsyncShareSequence: AsyncSequence {
typealias Element = Base.Element
typealias Failure = Base.Failure
struct Iterator: AsyncIteratorProtocol {
let side: Side
init(_ iteration: Iteration) {
side = Side(iteration)
}
mutating func next() async rethrows -> Element? {
try await side.next(isolation: nil)
}
mutating func next(isolation actor: isolated (any Actor)?) async throws(Failure) -> Element? {
try await side.next(isolation: actor)
}
}
func makeAsyncIterator() -> Iterator {
Iterator(extent.iteration)
}
}
#endif
@@ -0,0 +1,75 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension Sequence {
/// An asynchronous sequence containing the same elements as this sequence,
/// but on which operations, such as `map` and `filter`, are
/// implemented asynchronously.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public var async: AsyncSyncSequence<Self> {
AsyncSyncSequence(self)
}
}
/// An asynchronous sequence composed from a synchronous sequence.
///
/// Asynchronous lazy sequences can be used to interface existing or pre-calculated
/// data to interoperate with other asynchronous sequences and algorithms based on
/// asynchronous sequences.
///
/// This functions similarly to `LazySequence` by accessing elements sequentially
/// in the iterator's `next()` method.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncSyncSequence<Base: Sequence>: AsyncSequence {
public typealias Element = Base.Element
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.Iterator?
@usableFromInline
init(_ iterator: Base.Iterator) {
self.iterator = iterator
}
@inlinable
public mutating func next() async -> Base.Element? {
guard !Task.isCancelled, let value = iterator?.next() else {
iterator = nil
return nil
}
return value
}
}
@usableFromInline
let base: Base
@usableFromInline
init(_ base: Base) {
self.base = base
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncSyncSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncSyncSequence.Iterator: Sendable {}
@@ -0,0 +1,140 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2021 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle<C: Clock, Reduced>(
for interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> _AsyncThrottleSequence<Self, C, Reduced> {
_AsyncThrottleSequence(self, interval: interval, clock: clock, reducing: reducing)
}
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle<Reduced>(
for interval: Duration,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> _AsyncThrottleSequence<Self, ContinuousClock, Reduced> {
_throttle(for: interval, clock: .continuous, reducing: reducing)
}
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle<C: Clock>(
for interval: C.Instant.Duration,
clock: C,
latest: Bool = true
) -> _AsyncThrottleSequence<Self, C, Element> {
_throttle(for: interval, clock: clock) { previous, element in
guard latest else {
return previous ?? element
}
return element
}
}
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle(
for interval: Duration,
latest: Bool = true
) -> _AsyncThrottleSequence<Self, ContinuousClock, Element> {
_throttle(for: interval, clock: .continuous, latest: latest)
}
}
/// A rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public struct _AsyncThrottleSequence<Base: AsyncSequence, C: Clock, Reduced> {
let base: Base
let interval: C.Instant.Duration
let clock: C
let reducing: @Sendable (Reduced?, Base.Element) async -> Reduced
init(
_ base: Base,
interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Base.Element) async -> Reduced
) {
self.base = base
self.interval = interval
self.clock = clock
self.reducing = reducing
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension _AsyncThrottleSequence: AsyncSequence {
public typealias Element = Reduced
/// The iterator for an `AsyncThrottleSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
var base: Base.AsyncIterator
var last: C.Instant?
let interval: C.Instant.Duration
let clock: C
let reducing: @Sendable (Reduced?, Base.Element) async -> Reduced
init(
_ base: Base.AsyncIterator,
interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Base.Element) async -> Reduced
) {
self.base = base
self.interval = interval
self.clock = clock
self.reducing = reducing
}
public mutating func next() async rethrows -> Reduced? {
var reduced: Reduced?
let start = last ?? clock.now
repeat {
guard let element = try await base.next() else {
if reduced != nil, let last {
// ensure the rate of elements never exceeds the given interval
let amount = interval - last.duration(to: clock.now)
if amount > .zero {
try? await clock.sleep(until: clock.now.advanced(by: amount), tolerance: nil)
}
}
// the last value is unable to have any subsequent
// values so always return the last reduction
return reduced
}
let reduction = await reducing(reduced, element)
let now = clock.now
if start.duration(to: now) >= interval || last == nil {
last = now
return reduction
} else {
reduced = reduction
}
} while true
}
}
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), interval: interval, clock: clock, reducing: reducing)
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension _AsyncThrottleSequence: Sendable where Base: Sendable, Element: Sendable {}
@available(*, unavailable)
extension _AsyncThrottleSequence.Iterator: Sendable {}
@@ -0,0 +1,150 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given error-throwing closure.
///
/// Starting from the initial value, this combines each element of the base
/// asynchronous sequence with the running result and emits each intermediate
/// result. The initial value itself is not emitted, so the first element of
/// the returned sequence is the result of combining the initial value with
/// the first element of the base sequence. If the base sequence is empty, the
/// returned sequence is also empty.
///
/// - Parameters:
/// - initial: The value to use as the initial value.
/// - transform: A closure that combines the previously reduced result and
/// the next element in the receiving asynchronous sequence and returns
/// the result. If the closure throws an error, the sequence throws.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async throws -> Result
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result> {
reductions(into: initial) { result, element in
result = try await transform(result, element)
}
}
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given error-throwing closure.
///
/// Starting from the initial value, this combines each element of the base
/// asynchronous sequence with the running result and emits each intermediate
/// result. The initial value itself is not emitted, so the first element of
/// the returned sequence is the result of combining the initial value with
/// the first element of the base sequence. If the base sequence is empty, the
/// returned sequence is also empty.
///
/// - Parameters:
/// - initial: The value to use as the initial value.
/// - transform: A closure that combines the previously reduced result and
/// the next element in the receiving asynchronous sequence, mutating the
/// previous result instead of returning a value. If the closure throws an
/// error, the sequence throws.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async throws -> Void
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result> {
AsyncThrowingExclusiveReductionsSequence(self, initial: initial, transform: transform)
}
}
/// An asynchronous sequence of applying an error-throwing transform to the element of
/// an asynchronous sequence and the previously transformed result.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncThrowingExclusiveReductionsSequence<Base: AsyncSequence, Element> {
@usableFromInline
let base: Base
@usableFromInline
let initial: Element
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async throws -> Void
@inlinable
init(
_ base: Base,
initial: Element,
transform: @Sendable @escaping (inout Element, Base.Element) async throws -> Void
) {
self.base = base
self.initial = initial
self.transform = transform
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingExclusiveReductionsSequence: AsyncSequence {
/// The iterator for an `AsyncThrowingExclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.AsyncIterator
@usableFromInline
var current: Element?
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async throws -> Void
@inlinable
init(
_ iterator: Base.AsyncIterator,
initial: Element,
transform: @Sendable @escaping (inout Element, Base.Element) async throws -> Void
) {
self.iterator = iterator
self.current = initial
self.transform = transform
}
@inlinable
public mutating func next() async throws -> Element? {
guard var result = current else { return nil }
let value = try await iterator.next()
guard let value = value else {
current = nil
return nil
}
do {
try await transform(&result, value)
current = result
return result
} catch {
current = nil
throw error
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), initial: initial, transform: transform)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingExclusiveReductionsSequence: Sendable where Base: Sendable, Element: Sendable {}
@available(*, unavailable)
extension AsyncThrowingExclusiveReductionsSequence.Iterator: Sendable {}
@@ -0,0 +1,112 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given error-throwing closure.
///
/// This can be seen as applying the reduce function to each element and
/// providing the initial value followed by these results as an asynchronous sequence.
///
/// ```
/// let runningTotal = [1, 2, 3, 4].async.reductions(+)
/// print(await Array(runningTotal))
///
/// // prints [1, 3, 6, 10]
/// ```
///
/// - Parameter transform: A closure that combines the previously reduced
/// result and the next element in the receiving sequence. If the closure
/// throws an error, the sequence throws.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async throws -> Element
) -> AsyncThrowingInclusiveReductionsSequence<Self> {
AsyncThrowingInclusiveReductionsSequence(self, transform: transform)
}
}
/// An asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using a given error-throwing closure.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncThrowingInclusiveReductionsSequence<Base: AsyncSequence> {
@usableFromInline
let base: Base
@usableFromInline
let transform: @Sendable (Base.Element, Base.Element) async throws -> Base.Element
@inlinable
init(_ base: Base, transform: @Sendable @escaping (Base.Element, Base.Element) async throws -> Base.Element) {
self.base = base
self.transform = transform
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInclusiveReductionsSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncThrowingInclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal var iterator: Base.AsyncIterator?
@usableFromInline
internal var element: Base.Element?
@usableFromInline
internal let transform: @Sendable (Base.Element, Base.Element) async throws -> Base.Element
@inlinable
init(
_ iterator: Base.AsyncIterator,
transform: @Sendable @escaping (Base.Element, Base.Element) async throws -> Base.Element
) {
self.iterator = iterator
self.transform = transform
}
@inlinable
public mutating func next() async throws -> Base.Element? {
guard let previous = element else {
element = try await iterator?.next()
return element
}
guard let next = try await iterator?.next() else { return nil }
do {
element = try await transform(previous, next)
} catch {
iterator = nil
throw error
}
return element
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), transform: transform)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInclusiveReductionsSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncThrowingInclusiveReductionsSequence.Iterator: Sendable {}
@@ -0,0 +1,91 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// An `AsyncSequence` that produces elements at regular intervals.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public struct AsyncTimerSequence<C: Clock>: AsyncSequence {
public typealias Element = C.Instant
/// The iterator for an `AsyncTimerSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
var clock: C?
let interval: C.Instant.Duration
let tolerance: C.Instant.Duration?
var last: C.Instant?
init(interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.clock = clock
self.interval = interval
self.tolerance = tolerance
}
public mutating func next() async -> C.Instant? {
guard let clock = self.clock else {
return nil
}
let next = (self.last ?? clock.now).advanced(by: self.interval)
do {
try await clock.sleep(until: next, tolerance: self.tolerance)
} catch {
self.clock = nil
return nil
}
let now = clock.now
self.last = next
return now
}
}
let clock: C
let interval: C.Instant.Duration
let tolerance: C.Instant.Duration?
/// Create an `AsyncTimerSequence` with a given repeating interval.
public init(interval: C.Instant.Duration, tolerance: C.Instant.Duration? = nil, clock: C) {
self.clock = clock
self.interval = interval
self.tolerance = tolerance
}
public func makeAsyncIterator() -> Iterator {
Iterator(interval: interval, tolerance: tolerance, clock: clock)
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncTimerSequence {
/// Create an `AsyncTimerSequence` with a given repeating interval.
public static func repeating(
every interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
) -> AsyncTimerSequence<C> {
return AsyncTimerSequence(interval: interval, tolerance: tolerance, clock: clock)
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncTimerSequence where C == SuspendingClock {
/// Create an `AsyncTimerSequence` with a given repeating interval.
public static func repeating(
every interval: Duration,
tolerance: Duration? = nil
) -> AsyncTimerSequence<SuspendingClock> {
return AsyncTimerSequence(interval: interval, tolerance: tolerance, clock: SuspendingClock())
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncTimerSequence: Sendable {}
@available(*, unavailable)
extension AsyncTimerSequence.Iterator: Sendable {}
@@ -0,0 +1,144 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Self: Sendable {
/// Creates an asynchronous sequence that buffers elements.
///
/// The buffering behaviour is dictated by the policy:
/// - bounded: will buffer elements until the limit is reached. Then it will suspend the upstream async sequence.
/// - unbounded: will buffer elements without limit.
/// - bufferingNewest: will buffer elements until the limit is reached. Then it will discard the oldest elements.
/// - bufferingOldest: will buffer elements until the limit is reached. Then it will discard the newest elements.
///
/// - Parameter policy: A policy that drives the behaviour of the ``AsyncBufferSequence``
/// - Returns: An asynchronous sequence that buffers elements up to a given limit.
@available(AsyncAlgorithms 1.0, *)
public func buffer(
policy: AsyncBufferSequencePolicy
) -> AsyncBufferSequence<Self> {
AsyncBufferSequence<Self>(base: self, policy: policy)
}
}
/// A policy dictating the buffering behaviour of an ``AsyncBufferSequence``
@available(AsyncAlgorithms 1.0, *)
public struct AsyncBufferSequencePolicy: Sendable {
enum _Policy {
case bounded(Int)
case unbounded
case bufferingNewest(Int)
case bufferingOldest(Int)
}
let policy: _Policy
/// A policy for buffering elements until the limit is reached.
/// Then consumption of the upstream `AsyncSequence` will be paused until elements are consumed from the buffer.
/// If the limit is zero then no buffering policy is applied.
public static func bounded(_ limit: Int) -> Self {
precondition(limit >= 0, "The limit should be positive or equal to 0.")
return Self(policy: .bounded(limit))
}
/// A policy for buffering elements without limit.
public static var unbounded: Self {
return Self(policy: .unbounded)
}
/// A policy for buffering elements until the limit is reached.
/// After the limit is reached and a new element is produced by the upstream, the oldest buffered element will be discarded.
/// If the limit is zero then no buffering policy is applied.
public static func bufferingLatest(_ limit: Int) -> Self {
precondition(limit >= 0, "The limit should be positive or equal to 0.")
return Self(policy: .bufferingNewest(limit))
}
/// A policy for buffering elements until the limit is reached.
/// After the limit is reached and a new element is produced by the upstream, the latest buffered element will be discarded.
/// If the limit is zero then no buffering policy is applied.
public static func bufferingOldest(_ limit: Int) -> Self {
precondition(limit >= 0, "The limit should be positive or equal to 0.")
return Self(policy: .bufferingOldest(limit))
}
}
/// An `AsyncSequence` that buffers elements in regard to a policy.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncBufferSequence<Base: AsyncSequence & Sendable>: AsyncSequence {
// Internal implementation note:
// This type origianlly had no requirement that the element is actually Sendable. However,
// that is technically an implementation detail hole in the safety of the system, it needs
// to specify that the element is actually Sendable since the draining mechanism passes
// through the isolation that is in nature sending but cannot be marked as such for the
// isolated next method.
// In practice the users of this type are safe from isolation crossing since the Element
// is as sendable as it is required by the base sequences the buffer is constructed from.
enum StorageType {
case transparent(Base.AsyncIterator)
case bounded(storage: BoundedBufferStorage<Base>)
case unbounded(storage: UnboundedBufferStorage<Base>)
}
public typealias Element = Base.Element
public typealias AsyncIterator = Iterator
let base: Base
let policy: AsyncBufferSequencePolicy
init(
base: Base,
policy: AsyncBufferSequencePolicy
) {
self.base = base
self.policy = policy
}
public func makeAsyncIterator() -> Iterator {
let storageType: StorageType
switch self.policy.policy {
case .bounded(...0), .bufferingNewest(...0), .bufferingOldest(...0):
storageType = .transparent(self.base.makeAsyncIterator())
case .bounded(let limit):
storageType = .bounded(storage: BoundedBufferStorage(base: self.base, limit: limit))
case .unbounded:
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .unlimited))
case .bufferingNewest(let limit):
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .bufferingNewest(limit)))
case .bufferingOldest(let limit):
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .bufferingOldest(limit)))
}
return Iterator(storageType: storageType)
}
public struct Iterator: AsyncIteratorProtocol {
var storageType: StorageType
public mutating func next() async rethrows -> Element? {
switch self.storageType {
case .transparent(var iterator):
let element = try await iterator.next()
self.storageType = .transparent(iterator)
return element
case .bounded(let storage):
return try await storage.next().wrapped?._rethrowGet()
case .unbounded(let storage):
return try await storage.next().wrapped?._rethrowGet()
}
}
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncBufferSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncBufferSequence.Iterator: Sendable {}
@@ -0,0 +1,329 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
import DequeModule
@available(AsyncAlgorithms 1.0, *)
struct BoundedBufferStateMachine<Base: AsyncSequence> {
typealias Element = Base.Element
typealias SuspendedProducer = UnsafeContinuation<Void, Never>
typealias SuspendedConsumer = UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>
// We are using UnsafeTransfer here since we have to get the elements from the task
// into the consumer task. This is a transfer but we cannot prove this to the compiler at this point
// since next is not marked as transferring the return value.
fileprivate enum State {
case initial(base: Base)
case buffering(
task: Task<Void, Never>,
buffer: Deque<Result<UnsafeTransfer<Element>, Error>>,
suspendedProducer: SuspendedProducer?,
suspendedConsumer: SuspendedConsumer?
)
case modifying
case finished(buffer: Deque<Result<UnsafeTransfer<Element>, Error>>)
}
private var state: State
private let limit: Int
init(base: Base, limit: Int) {
self.state = .initial(base: base)
self.limit = limit
}
var task: Task<Void, Never>? {
switch self.state {
case .buffering(let task, _, _, _):
return task
default:
return nil
}
}
mutating func taskStarted(task: Task<Void, Never>) {
switch self.state {
case .initial:
self.state = .buffering(task: task, buffer: [], suspendedProducer: nil, suspendedConsumer: nil)
case .buffering:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
preconditionFailure("Invalid state.")
}
}
mutating func shouldSuspendProducer() -> Bool {
switch state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(_, let buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if there are free slots, we should directly request the next element
return buffer.count >= self.limit
case .buffering(_, _, .none, .some):
// we have an awaiting consumer, we should not suspended the producer, we should
// directly request the next element
return false
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There is already a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return false
}
}
enum ProducerSuspendedAction {
case none
case resumeProducer
}
mutating func producerSuspended(continuation: SuspendedProducer) -> ProducerSuspendedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(let task, let buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if the buffer is available we resume the producer so it can we can request the next element
// otherwise we confirm the suspension
guard buffer.count < limit else {
self.state = .buffering(
task: task,
buffer: buffer,
suspendedProducer: continuation,
suspendedConsumer: nil
)
return .none
}
return .resumeProducer
case .buffering(_, let buffer, .none, .some):
// we have an awaiting consumer, we can resume the producer so the next element can be requested
precondition(
buffer.isEmpty,
"Invalid state. The buffer should be empty as we have an awaiting consumer already."
)
return .resumeProducer
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There is already a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .resumeProducer
}
}
enum ElementProducedAction {
case none
case resumeConsumer(continuation: SuspendedConsumer, result: UnsafeTransfer<Result<Base.Element, Error>?>)
}
mutating func elementProduced(element: Element) -> ElementProducedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(let task, var buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// we have to stack the new element or suspend the producer if the buffer is full
precondition(
buffer.count < limit,
"Invalid state. The buffer should be available for stacking a new element."
)
self.state = .modifying
buffer.append(.success(.init(element)))
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .none
case .buffering(let task, let buffer, .none, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it with the element and exit
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .resumeConsumer(continuation: suspendedConsumer, result: UnsafeTransfer(.success(element)))
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There should not be a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
}
}
enum FinishAction {
case none
case resumeConsumer(
continuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>?
)
}
mutating func finish(error: Error?) -> FinishAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(_, var buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if we have an error we stack it in the buffer so it can be consumed later
if let error {
buffer.append(.failure(error))
}
self.state = .finished(buffer: buffer)
return .none
case .buffering(_, let buffer, .none, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .finished(buffer: [])
return .resumeConsumer(continuation: suspendedConsumer)
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There should not be a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
}
}
enum NextAction {
case startTask(base: Base)
case suspend
case returnResult(producerContinuation: UnsafeContinuation<Void, Never>?, result: Result<Element, Error>?)
}
mutating func next() -> NextAction {
switch state {
case .initial(let base):
return .startTask(base: base)
case .buffering(_, let buffer, .none, .none) where buffer.isEmpty:
// we are idle, we must suspend the consumer
return .suspend
case .buffering(let task, var buffer, let suspendedProducer, .none):
// we have values in the buffer, we unstack the oldest one and resume a potential suspended producer
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .returnResult(producerContinuation: suspendedProducer, result: result.map { $0.wrapped })
case .buffering(_, _, _, .some):
preconditionFailure("Invalid states. There is already a suspended consumer.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(producerContinuation: nil, result: nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(producerContinuation: nil, result: result.map { $0.wrapped })
}
}
enum NextSuspendedAction {
case none
case returnResult(producerContinuation: UnsafeContinuation<Void, Never>?, result: Result<Element, Error>?)
}
mutating func nextSuspended(continuation: SuspendedConsumer) -> NextSuspendedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(let task, let buffer, .none, .none) where buffer.isEmpty:
// we are idle, we confirm the suspension of the consumer
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: continuation)
return .none
case .buffering(let task, var buffer, let suspendedProducer, .none):
// we have values in the buffer, we unstack the oldest one and resume a potential suspended producer
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .returnResult(producerContinuation: suspendedProducer, result: result.map { $0.wrapped })
case .buffering(_, _, _, .some):
preconditionFailure("Invalid states. There is already a suspended consumer.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(producerContinuation: nil, result: nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(producerContinuation: nil, result: result.map { $0.wrapped })
}
}
enum InterruptedAction {
case none
case resumeProducerAndConsumer(
task: Task<Void, Never>,
producerContinuation: UnsafeContinuation<Void, Never>?,
consumerContinuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>?
)
}
mutating func interrupted() -> InterruptedAction {
switch self.state {
case .initial:
self.state = .finished(buffer: [])
return .none
case .buffering(let task, _, let suspendedProducer, let suspendedConsumer):
self.state = .finished(buffer: [])
return .resumeProducerAndConsumer(
task: task,
producerContinuation: suspendedProducer,
consumerContinuation: suspendedConsumer
)
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
self.state = .finished(buffer: [])
return .none
}
}
}
@available(AsyncAlgorithms 1.0, *)
extension BoundedBufferStateMachine: Sendable where Base: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension BoundedBufferStateMachine.State: Sendable where Base: Sendable {}
@@ -0,0 +1,158 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Sendable {
private let stateMachine: ManagedCriticalState<BoundedBufferStateMachine<Base>>
init(base: Base, limit: Int) {
self.stateMachine = ManagedCriticalState(BoundedBufferStateMachine(base: base, limit: limit))
}
func next() async -> UnsafeTransfer<Result<Base.Element, Error>?> {
return await withTaskCancellationHandler {
let action: BoundedBufferStateMachine<Base>.NextAction? = self.stateMachine.withCriticalRegion {
stateMachine in
let action = stateMachine.next()
switch action {
case .startTask(let base):
self.startTask(stateMachine: &stateMachine, base: base)
return nil
case .suspend:
return action
case .returnResult:
return action
}
}
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .suspend:
break
case .returnResult(let producerContinuation, let result):
producerContinuation?.resume()
return UnsafeTransfer(result)
case .none:
break
}
return await withUnsafeContinuation {
(continuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.nextSuspended(continuation: continuation)
}
switch action {
case .none:
break
case .returnResult(let producerContinuation, let result):
producerContinuation?.resume()
continuation.resume(returning: UnsafeTransfer(result))
}
}
} onCancel: {
self.interrupted()
}
}
private func startTask(
stateMachine: inout BoundedBufferStateMachine<Base>,
base: Base
) {
let task = Task {
do {
var iterator = base.makeAsyncIterator()
loop: while true {
let shouldSuspend = self.stateMachine.withCriticalRegion { stateMachine -> Bool in
return stateMachine.shouldSuspendProducer()
}
if shouldSuspend {
await withUnsafeContinuation { (continuation: UnsafeContinuation<Void, Never>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.producerSuspended(continuation: continuation)
}
switch action {
case .none:
break
case .resumeProducer:
continuation.resume()
}
}
}
guard let element = try await iterator.next() else {
// the upstream is finished
break loop
}
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.elementProduced(element: element)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation, let result):
continuation.resume(returning: result)
}
}
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: nil)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(nil))
}
} catch {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: error)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(Result<Base.Element, Error>.failure(error)))
}
}
}
stateMachine.taskStarted(task: task)
}
func interrupted() {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.interrupted()
}
switch action {
case .none:
break
case .resumeProducerAndConsumer(let task, let producerContinuation, let consumerContinuation):
task.cancel()
producerContinuation?.resume()
consumerContinuation?.resume(returning: UnsafeTransfer(nil))
}
}
deinit {
self.interrupted()
}
}
@@ -0,0 +1,259 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
import DequeModule
@available(AsyncAlgorithms 1.0, *)
struct UnboundedBufferStateMachine<Base: AsyncSequence> {
typealias Element = Base.Element
typealias SuspendedConsumer = UnsafeContinuation<UnsafeTransfer<Result<Element, Error>?>, Never>
enum Policy {
case unlimited
case bufferingNewest(Int)
case bufferingOldest(Int)
}
// We are using UnsafeTransfer here since we have to get the elements from the task
// into the consumer task. This is a transfer but we cannot prove this to the compiler at this point
// since next is not marked as transferring the return value.
fileprivate enum State {
case initial(base: Base)
case buffering(
task: Task<Void, Never>,
buffer: Deque<Result<UnsafeTransfer<Element>, Error>>,
suspendedConsumer: SuspendedConsumer?
)
case modifying
case finished(buffer: Deque<Result<UnsafeTransfer<Element>, Error>>)
}
private var state: State
private let policy: Policy
init(base: Base, policy: Policy) {
self.state = .initial(base: base)
self.policy = policy
}
var task: Task<Void, Never>? {
switch self.state {
case .buffering(let task, _, _):
return task
default:
return nil
}
}
mutating func taskStarted(task: Task<Void, Never>) {
switch self.state {
case .initial:
self.state = .buffering(task: task, buffer: [], suspendedConsumer: nil)
case .buffering:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
preconditionFailure("Invalid state.")
}
}
enum ElementProducedAction {
case none
case resumeConsumer(
continuation: SuspendedConsumer,
result: UnsafeTransfer<Result<Element, Error>?>
)
}
mutating func elementProduced(element: Element) -> ElementProducedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(let task, var buffer, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// we have to apply the policy when stacking the new element
self.state = .modifying
switch self.policy {
case .unlimited:
buffer.append(.success(.init(element)))
case .bufferingNewest(let limit):
if buffer.count >= limit {
_ = buffer.popFirst()
}
buffer.append(.success(.init(element)))
case .bufferingOldest(let limit):
if buffer.count < limit {
buffer.append(.success(.init(element)))
}
}
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .none
case .buffering(let task, let buffer, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it with the element
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .resumeConsumer(
continuation: suspendedConsumer,
result: UnsafeTransfer(.success(element))
)
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
}
}
enum FinishAction {
case none
case resumeConsumer(continuation: SuspendedConsumer?)
}
mutating func finish(error: Error?) -> FinishAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(_, var buffer, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if we have an error we stack it in the buffer so it can be consumed later
if let error {
buffer.append(.failure(error))
}
self.state = .finished(buffer: buffer)
return .none
case .buffering(_, let buffer, let suspendedConsumer):
// we have an awaiting consumer, we can resume it with nil or the error
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .finished(buffer: [])
return .resumeConsumer(continuation: suspendedConsumer)
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
}
}
enum NextAction {
case startTask(base: Base)
case suspend
case returnResult(Result<Element, Error>?)
}
mutating func next() -> NextAction {
switch self.state {
case .initial(let base):
return .startTask(base: base)
case .buffering(_, let buffer, let suspendedConsumer) where buffer.isEmpty:
// we are idle, we have to suspend the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
return .suspend
case .buffering(let task, var buffer, let suspendedConsumer):
// the buffer is already in use, we can unstack a value and directly resume the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .returnResult(result.map { $0.wrapped })
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(result.map { $0.wrapped })
}
}
enum NextSuspendedAction {
case none
case resumeConsumer(Result<Element, Error>?)
}
mutating func nextSuspended(continuation: SuspendedConsumer) -> NextSuspendedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(let task, let buffer, let suspendedConsumer) where buffer.isEmpty:
// we are idle, we confirm the suspension of the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: continuation)
return .none
case .buffering(let task, var buffer, let suspendedConsumer):
// the buffer is already in use, we can unstack a value and directly resume the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .resumeConsumer(result.map { $0.wrapped })
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .resumeConsumer(nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .resumeConsumer(result.map { $0.wrapped })
}
}
enum InterruptedAction {
case none
case resumeConsumer(task: Task<Void, Never>, continuation: SuspendedConsumer?)
}
mutating func interrupted() -> InterruptedAction {
switch self.state {
case .initial:
state = .finished(buffer: [])
return .none
case .buffering(let task, _, let suspendedConsumer):
self.state = .finished(buffer: [])
return .resumeConsumer(task: task, continuation: suspendedConsumer)
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
self.state = .finished(buffer: [])
return .none
}
}
}
@available(AsyncAlgorithms 1.0, *)
extension UnboundedBufferStateMachine: Sendable where Base: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension UnboundedBufferStateMachine.State: Sendable where Base: Sendable {}
@@ -0,0 +1,126 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class UnboundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Sendable {
private let stateMachine: ManagedCriticalState<UnboundedBufferStateMachine<Base>>
init(base: Base, policy: UnboundedBufferStateMachine<Base>.Policy) {
self.stateMachine = ManagedCriticalState(UnboundedBufferStateMachine<Base>(base: base, policy: policy))
}
func next() async -> UnsafeTransfer<Result<Base.Element, Error>?> {
return await withTaskCancellationHandler {
let action: UnboundedBufferStateMachine<Base>.NextAction? = self.stateMachine.withCriticalRegion {
stateMachine in
let action = stateMachine.next()
switch action {
case .startTask(let base):
self.startTask(stateMachine: &stateMachine, base: base)
return nil
case .suspend:
return action
case .returnResult:
return action
}
}
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .suspend:
break
case .returnResult(let result):
return UnsafeTransfer(result)
case .none:
break
}
return await withUnsafeContinuation {
(continuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.nextSuspended(continuation: continuation)
}
switch action {
case .none:
break
case .resumeConsumer(let result):
continuation.resume(returning: UnsafeTransfer(result))
}
}
} onCancel: {
self.interrupted()
}
}
private func startTask(
stateMachine: inout UnboundedBufferStateMachine<Base>,
base: Base
) {
let task = Task {
do {
for try await element in base {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.elementProduced(element: element)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation, let result):
continuation.resume(returning: result)
}
}
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: nil)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(nil))
}
} catch {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: error)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(Result<Base.Element, Error>.failure(error)))
}
}
}
stateMachine.taskStarted(task: task)
}
func interrupted() {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.interrupted()
}
switch action {
case .none:
break
case .resumeConsumer(let task, let continuation):
task.cancel()
continuation?.resume(returning: UnsafeTransfer(nil))
}
}
deinit {
self.interrupted()
}
}
@@ -0,0 +1,64 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// A channel for sending elements from one task to another with back pressure.
///
/// The `AsyncChannel` class is intended to be used as a communication type between tasks,
/// particularly when one task produces values and another task consumes those values. The back
/// pressure applied by `send(_:)` via the suspension/resume ensures that
/// the production of values does not exceed the consumption of values from iteration. This method
/// suspends after enqueuing the event and is resumed when the next call to `next()`
/// on the `Iterator` is made, or when `finish()` is called from another Task.
/// As `finish()` induces a terminal state, there is no more need for a back pressure management.
/// This function does not suspend and will finish all the pending iterations.
@available(AsyncAlgorithms 1.0, *)
public final class AsyncChannel<Element: Sendable>: AsyncSequence, Sendable {
public typealias Element = Element
public typealias AsyncIterator = Iterator
let storage: ChannelStorage<Element, Never>
public init() {
self.storage = ChannelStorage()
}
/// Sends an element to an awaiting iteration. This function will resume when the next call to `next()` is made
/// or when a call to `finish()` is made from another task.
/// If the channel is already finished then this returns immediately.
/// If the task is cancelled, this function will resume without sending the element.
/// Other sending operations from other tasks will remain active.
public func send(_ element: Element) async {
await self.storage.send(element: element)
}
/// Immediately resumes all the suspended operations.
/// All subsequent calls to `next(_:)` will resume immediately.
public func finish() {
self.storage.finish()
}
public func makeAsyncIterator() -> Iterator {
Iterator(storage: self.storage)
}
public struct Iterator: AsyncIteratorProtocol {
let storage: ChannelStorage<Element, Never>
public mutating func next() async -> Element? {
// Although the storage can throw, its usage in the context of an `AsyncChannel` guarantees it cannot.
// There is no public way of sending a failure to it.
try! await self.storage.next()
}
}
}
@available(*, unavailable)
extension AsyncChannel.Iterator: Sendable {}
@@ -0,0 +1,67 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// An error-throwing channel for sending elements from on task to another with back pressure.
///
/// The `AsyncThrowingChannel` class is intended to be used as a communication types between tasks,
/// particularly when one task produces values and another task consumes those values. The back
/// pressure applied by `send(_:)` via suspension/resume ensures that the production of values does
/// not exceed the consumption of values from iteration. This method suspends after enqueuing the event
/// and is resumed when the next call to `next()` on the `Iterator` is made, or when `finish()`/`fail(_:)` is called
/// from another Task. As `finish()` and `fail(_:)` induce a terminal state, there is no more need for a back pressure management.
/// Those functions do not suspend and will finish all the pending iterations.
@available(AsyncAlgorithms 1.0, *)
public final class AsyncThrowingChannel<Element: Sendable, Failure: Error>: AsyncSequence, Sendable {
public typealias Element = Element
public typealias AsyncIterator = Iterator
let storage: ChannelStorage<Element, Failure>
public init() {
self.storage = ChannelStorage()
}
/// Sends an element to an awaiting iteration. This function will resume when the next call to `next()` is made
/// or when a call to `finish()` or `fail` is made from another task.
/// If the channel is already finished then this returns immediately.
/// If the task is cancelled, this function will resume without sending the element.
/// Other sending operations from other tasks will remain active.
public func send(_ element: Element) async {
await self.storage.send(element: element)
}
/// Sends an error to all awaiting iterations.
/// All subsequent calls to `next(_:)` will resume immediately.
public func fail(_ error: Error) where Failure == Error {
self.storage.finish(error: error)
}
/// Immediately resumes all the suspended operations.
/// All subsequent calls to `next(_:)` will resume immediately.
public func finish() {
self.storage.finish()
}
public func makeAsyncIterator() -> Iterator {
Iterator(storage: self.storage)
}
public struct Iterator: AsyncIteratorProtocol {
let storage: ChannelStorage<Element, Failure>
public mutating func next() async throws -> Element? {
try await self.storage.next()
}
}
}
@available(*, unavailable)
extension AsyncThrowingChannel.Iterator: Sendable {}
@@ -0,0 +1,348 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
import OrderedCollections
@available(AsyncAlgorithms 1.0, *)
struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
private struct SuspendedProducer: Hashable, Sendable {
let id: UInt64
let continuation: UnsafeContinuation<Void, Never>?
let element: Element?
func hash(into hasher: inout Hasher) {
hasher.combine(self.id)
}
static func == (_ lhs: SuspendedProducer, _ rhs: SuspendedProducer) -> Bool {
return lhs.id == rhs.id
}
static func placeHolder(id: UInt64) -> SuspendedProducer {
SuspendedProducer(id: id, continuation: nil, element: nil)
}
}
private struct SuspendedConsumer: Hashable, Sendable {
let id: UInt64
let continuation: UnsafeContinuation<Element?, any Error>?
func hash(into hasher: inout Hasher) {
hasher.combine(self.id)
}
static func == (_ lhs: SuspendedConsumer, _ rhs: SuspendedConsumer) -> Bool {
return lhs.id == rhs.id
}
static func placeHolder(id: UInt64) -> SuspendedConsumer {
SuspendedConsumer(id: id, continuation: nil)
}
}
private enum Termination {
case finished
case failed(Error)
}
private enum State: Sendable {
case channeling(
suspendedProducers: OrderedSet<SuspendedProducer>,
cancelledProducers: Set<SuspendedProducer>,
suspendedConsumers: OrderedSet<SuspendedConsumer>,
cancelledConsumers: Set<SuspendedConsumer>
)
case terminated(Termination)
}
private var state: State = .channeling(
suspendedProducers: [],
cancelledProducers: [],
suspendedConsumers: [],
cancelledConsumers: []
)
enum SendAction {
case resumeConsumer(continuation: UnsafeContinuation<Element?, any Error>?)
case suspend
}
mutating func send() -> SendAction {
switch self.state {
case .channeling(_, _, let suspendedConsumers, _) where suspendedConsumers.isEmpty:
// we are idle or waiting for consumers, we have to suspend the producer
return .suspend
case .channeling(let suspendedProducers, let cancelledProducers, var suspendedConsumers, let cancelledConsumers):
// we are waiting for producers, we can resume the first available consumer
let suspendedConsumer = suspendedConsumers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(continuation: suspendedConsumer.continuation)
case .terminated:
return .resumeConsumer(continuation: nil)
}
}
enum SendSuspendedAction {
case resumeProducer
case resumeProducerAndConsumer(continuation: UnsafeContinuation<Element?, any Error>?)
}
mutating func sendSuspended(
continuation: UnsafeContinuation<Void, Never>,
element: Element,
producerID: UInt64
) -> SendSuspendedAction? {
switch self.state {
case .channeling(var suspendedProducers, var cancelledProducers, var suspendedConsumers, let cancelledConsumers):
let suspendedProducer = SuspendedProducer(id: producerID, continuation: continuation, element: element)
if let _ = cancelledProducers.remove(suspendedProducer) {
// the producer was already cancelled, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer
}
guard suspendedConsumers.isEmpty else {
// we are waiting for producers
// we resume the first consumer
let suspendedConsumer = suspendedConsumers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducerAndConsumer(continuation: suspendedConsumer.continuation)
}
// we are idle or waiting for consumers
// we stack the incoming producer in a suspended state
suspendedProducers.append(suspendedProducer)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated:
return .resumeProducer
}
}
enum SendCancelledAction {
case none
case resumeProducer(continuation: UnsafeContinuation<Void, Never>?)
}
mutating func sendCancelled(producerID: UInt64) -> SendCancelledAction {
switch self.state {
case .channeling(var suspendedProducers, var cancelledProducers, let suspendedConsumers, let cancelledConsumers):
// the cancelled producer might be part of the waiting list
let placeHolder = SuspendedProducer.placeHolder(id: producerID)
if let removed = suspendedProducers.remove(placeHolder) {
// the producer was cancelled after being added to the suspended ones, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer(continuation: removed.continuation)
}
// the producer was cancelled before being added to the suspended ones
cancelledProducers.update(with: placeHolder)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated:
return .none
}
}
enum FinishAction {
case none
case resumeProducersAndConsumers(
producerSontinuations: [UnsafeContinuation<Void, Never>?],
consumerContinuations: [UnsafeContinuation<Element?, any Error>?]
)
}
mutating func finish(error: Failure?) -> FinishAction {
switch self.state {
case .channeling(let suspendedProducers, _, let suspendedConsumers, _):
// no matter if we are idle, waiting for producers or waiting for consumers, we resume every thing that is suspended
if let error {
if suspendedConsumers.isEmpty {
self.state = .terminated(.failed(error))
} else {
self.state = .terminated(.finished)
}
} else {
self.state = .terminated(.finished)
}
return .resumeProducersAndConsumers(
producerSontinuations: suspendedProducers.map { $0.continuation },
consumerContinuations: suspendedConsumers.map { $0.continuation }
)
case .terminated:
return .none
}
}
enum NextAction {
case resumeProducer(continuation: UnsafeContinuation<Void, Never>?, result: Result<Element?, Error>)
case suspend
}
mutating func next() -> NextAction {
switch self.state {
case .channeling(let suspendedProducers, _, _, _) where suspendedProducers.isEmpty:
// we are idle or waiting for producers, we must suspend
return .suspend
case .channeling(var suspendedProducers, let cancelledProducers, let suspendedConsumers, let cancelledConsumers):
// we are waiting for consumers, we can resume the first awaiting producer
let suspendedProducer = suspendedProducers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer(
continuation: suspendedProducer.continuation,
result: .success(suspendedProducer.element)
)
case .terminated(.failed(let error)):
self.state = .terminated(.finished)
return .resumeProducer(continuation: nil, result: .failure(error))
case .terminated:
return .resumeProducer(continuation: nil, result: .success(nil))
}
}
enum NextSuspendedAction {
case resumeConsumer(element: Element?)
case resumeConsumerWithError(error: Error)
case resumeProducerAndConsumer(continuation: UnsafeContinuation<Void, Never>?, element: Element?)
}
mutating func nextSuspended(
continuation: UnsafeContinuation<Element?, any Error>,
consumerID: UInt64
) -> NextSuspendedAction? {
switch self.state {
case .channeling(var suspendedProducers, let cancelledProducers, var suspendedConsumers, var cancelledConsumers):
let suspendedConsumer = SuspendedConsumer(id: consumerID, continuation: continuation)
if let _ = cancelledConsumers.remove(suspendedConsumer) {
// the consumer was already cancelled, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(element: nil)
}
guard suspendedProducers.isEmpty else {
// we are waiting for consumers
// we resume the first producer
let suspendedProducer = suspendedProducers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducerAndConsumer(
continuation: suspendedProducer.continuation,
element: suspendedProducer.element
)
}
// we are idle or waiting for producers
// we stack the incoming consumer in a suspended state
suspendedConsumers.append(suspendedConsumer)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated(.finished):
return .resumeConsumer(element: nil)
case .terminated(.failed(let error)):
self.state = .terminated(.finished)
return .resumeConsumerWithError(error: error)
}
}
enum NextCancelledAction {
case none
case resumeConsumer(continuation: UnsafeContinuation<Element?, any Error>?)
}
mutating func nextCancelled(consumerID: UInt64) -> NextCancelledAction {
switch self.state {
case .channeling(let suspendedProducers, let cancelledProducers, var suspendedConsumers, var cancelledConsumers):
// the cancelled consumer might be part of the suspended ones
let placeHolder = SuspendedConsumer.placeHolder(id: consumerID)
if let removed = suspendedConsumers.remove(placeHolder) {
// the consumer was cancelled after being added to the suspended ones, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(continuation: removed.continuation)
}
// the consumer was cancelled before being added to the suspended ones
cancelledConsumers.update(with: placeHolder)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated:
return .none
}
}
}
@@ -0,0 +1,143 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
private let stateMachine: ManagedCriticalState<ChannelStateMachine<Element, Failure>>
private let ids = ManagedCriticalState<UInt64>(0)
init() {
self.stateMachine = ManagedCriticalState(ChannelStateMachine())
}
func generateId() -> UInt64 {
self.ids.withCriticalRegion { ids in
defer { ids &+= 1 }
return ids
}
}
func send(element: Element) async {
// check if a suspension is needed
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.send()
}
switch action {
case .suspend:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: element)
return
}
let producerID = self.generateId()
await withTaskCancellationHandler {
// a suspension is needed
await withUnsafeContinuation { (continuation: UnsafeContinuation<Void, Never>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.sendSuspended(continuation: continuation, element: element, producerID: producerID)
}
switch action {
case .none:
break
case .resumeProducer:
continuation.resume()
case .resumeProducerAndConsumer(let consumerContinuation):
continuation.resume()
consumerContinuation?.resume(returning: element)
}
}
} onCancel: {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.sendCancelled(producerID: producerID)
}
switch action {
case .none:
break
case .resumeProducer(let continuation):
continuation?.resume()
}
}
}
func finish(error: Failure? = nil) {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: error)
}
switch action {
case .none:
break
case .resumeProducersAndConsumers(let producerContinuations, let consumerContinuations):
producerContinuations.forEach { $0?.resume() }
if let error {
consumerContinuations.forEach { $0?.resume(throwing: error) }
} else {
consumerContinuations.forEach { $0?.resume(returning: nil) }
}
}
}
func next() async throws -> Element? {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.next()
}
switch action {
case .suspend:
break
case .resumeProducer(let producerContinuation, let result):
producerContinuation?.resume()
return try result._rethrowGet()
}
let consumerID = self.generateId()
return try await withTaskCancellationHandler {
try await withUnsafeThrowingContinuation { (continuation: UnsafeContinuation<Element?, any Error>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.nextSuspended(
continuation: continuation,
consumerID: consumerID
)
}
switch action {
case .none:
break
case .resumeConsumer(let element):
continuation.resume(returning: element)
case .resumeConsumerWithError(let error):
continuation.resume(throwing: error)
case .resumeProducerAndConsumer(let producerContinuation, let element):
producerContinuation?.resume()
continuation.resume(returning: element)
}
}
} onCancel: {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.nextCancelled(consumerID: consumerID)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: nil)
}
}
}
}
@@ -0,0 +1,98 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// Creates an asynchronous sequence that combines the latest values from two `AsyncSequence` types
/// by emitting a tuple of the values. ``combineLatest(_:_:)`` only emits a value whenever any of the base `AsyncSequence`s
/// emit a value (so long as each of the bases have emitted at least one value).
///
/// Finishes:
/// ``combineLatest(_:_:)`` finishes when one of the bases finishes before emitting any value or
/// when all bases finished.
///
/// Throws:
/// ``combineLatest(_:_:)`` throws when one of the bases throws. If one of the bases threw any buffered and not yet consumed
/// values will be dropped.
@available(AsyncAlgorithms 1.0, *)
public func combineLatest<
Base1: AsyncSequence,
Base2: AsyncSequence
>(_ base1: Base1, _ base2: Base2) -> AsyncCombineLatest2Sequence<Base1, Base2>
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable
{
AsyncCombineLatest2Sequence(base1, base2)
}
/// An `AsyncSequence` that combines the latest values produced from two asynchronous sequences into an asynchronous sequence of tuples.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncCombineLatest2Sequence<
Base1: AsyncSequence,
Base2: AsyncSequence
>: AsyncSequence, Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable
{
public typealias Element = (Base1.Element, Base2.Element)
public typealias AsyncIterator = Iterator
let base1: Base1
let base2: Base2
init(_ base1: Base1, _ base2: Base2) {
self.base1 = base1
self.base2 = base2
}
public func makeAsyncIterator() -> AsyncIterator {
Iterator(storage: .init(self.base1, self.base2, nil))
}
public struct Iterator: AsyncIteratorProtocol {
final class InternalClass {
private let storage: CombineLatestStorage<Base1, Base2, Base2>
fileprivate init(storage: CombineLatestStorage<Base1, Base2, Base2>) {
self.storage = storage
}
deinit {
self.storage.iteratorDeinitialized()
}
func next() async rethrows -> Element? {
guard let element = try await self.storage.next() else {
return nil
}
return (element.0, element.1)
}
}
let internalClass: InternalClass
fileprivate init(storage: CombineLatestStorage<Base1, Base2, Base2>) {
self.internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await self.internalClass.next()
}
}
}
@available(*, unavailable)
extension AsyncCombineLatest2Sequence.Iterator: Sendable {}
@@ -0,0 +1,109 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// Creates an asynchronous sequence that combines the latest values from three `AsyncSequence` types
/// by emitting a tuple of the values. ``combineLatest(_:_:_:)`` only emits a value whenever any of the base `AsyncSequence`s
/// emit a value (so long as each of the bases have emitted at least one value).
///
/// Finishes:
/// ``combineLatest(_:_:_:)`` finishes when one of the bases finishes before emitting any value or
/// when all bases finished.
///
/// Throws:
/// ``combineLatest(_:_:_:)`` throws when one of the bases throws. If one of the bases threw any buffered and not yet consumed
/// values will be dropped.
@available(AsyncAlgorithms 1.0, *)
public func combineLatest<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncCombineLatest3Sequence<Base1, Base2, Base3>
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable
{
AsyncCombineLatest3Sequence(base1, base2, base3)
}
/// An `AsyncSequence` that combines the latest values produced from three asynchronous sequences into an asynchronous sequence of tuples.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncCombineLatest3Sequence<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: AsyncSequence, Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable
{
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public typealias AsyncIterator = Iterator
let base1: Base1
let base2: Base2
let base3: Base3
init(_ base1: Base1, _ base2: Base2, _ base3: Base3) {
self.base1 = base1
self.base2 = base2
self.base3 = base3
}
public func makeAsyncIterator() -> AsyncIterator {
Iterator(
storage: .init(self.base1, self.base2, self.base3)
)
}
public struct Iterator: AsyncIteratorProtocol {
final class InternalClass {
private let storage: CombineLatestStorage<Base1, Base2, Base3>
fileprivate init(storage: CombineLatestStorage<Base1, Base2, Base3>) {
self.storage = storage
}
deinit {
self.storage.iteratorDeinitialized()
}
func next() async rethrows -> Element? {
guard let element = try await self.storage.next() else {
return nil
}
// This force unwrap is safe since there must be a third element.
return (element.0, element.1, element.2!)
}
}
let internalClass: InternalClass
fileprivate init(storage: CombineLatestStorage<Base1, Base2, Base3>) {
self.internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await self.internalClass.next()
}
}
}
@available(*, unavailable)
extension AsyncCombineLatest3Sequence.Iterator: Sendable {}
@@ -0,0 +1,773 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
import DequeModule
/// State machine for combine latest
@available(AsyncAlgorithms 1.0, *)
struct CombineLatestStateMachine<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable
where
Base1: Sendable,
Base2: Sendable,
Base3: Sendable,
Base1.Element: Sendable,
Base2.Element: Sendable,
Base3.Element: Sendable
{
typealias DownstreamContinuation = UnsafeContinuation<
Result<
(
Base1.Element,
Base2.Element,
Base3.Element?
)?, Error
>, Never
>
private enum State: Sendable {
/// Small wrapper for the state of an upstream sequence.
struct Upstream<Element: Sendable>: Sendable {
/// The upstream continuation.
var continuation: UnsafeContinuation<Void, Error>?
/// The produced upstream element.
var element: Element?
/// Indicates wether the upstream finished/threw already
var isFinished: Bool
}
/// The initial state before a call to `next` happened.
case initial(base1: Base1, base2: Base2, base3: Base3?)
/// The state while we are waiting for downstream demand.
case waitingForDemand(
task: Task<Void, Never>,
upstreams: (Upstream<Base1.Element>, Upstream<Base2.Element>, Upstream<Base3.Element>),
buffer: Deque<(Base1.Element, Base2.Element, Base3.Element?)>
)
/// The state while we are consuming the upstream and waiting until we get a result from all upstreams.
case combining(
task: Task<Void, Never>,
upstreams: (Upstream<Base1.Element>, Upstream<Base2.Element>, Upstream<Base3.Element>),
downstreamContinuation: DownstreamContinuation,
buffer: Deque<(Base1.Element, Base2.Element, Base3.Element?)>
)
case upstreamsFinished(
buffer: Deque<(Base1.Element, Base2.Element, Base3.Element?)>
)
case upstreamThrew(
error: Error
)
/// The state once the downstream consumer stopped, i.e. by dropping all references
/// or by getting their `Task` cancelled.
case finished
/// Internal state to avoid CoW.
case modifying
}
private var state: State
private let numberOfUpstreamSequences: Int
/// Initializes a new `StateMachine`.
init(
base1: Base1,
base2: Base2,
base3: Base3?
) {
self.state = .initial(
base1: base1,
base2: base2,
base3: base3
)
if base3 == nil {
self.numberOfUpstreamSequences = 2
} else {
self.numberOfUpstreamSequences = 3
}
}
/// Actions returned by `iteratorDeinitialized()`.
enum IteratorDeinitializedAction {
/// Indicates that the `Task` needs to be cancelled and
/// the upstream continuations need to be resumed with a `CancellationError`.
case cancelTaskAndUpstreamContinuations(
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
}
mutating func iteratorDeinitialized() -> IteratorDeinitializedAction? {
switch self.state {
case .initial:
// Nothing to do here. No demand was signalled until now
return .none
case .combining:
// An iterator was deinitialized while we have a suspended continuation.
preconditionFailure(
"Internal inconsistency current state \(self.state) and received iteratorDeinitialized()"
)
case .waitingForDemand(let task, let upstreams, _):
// The iterator was dropped which signals that the consumer is finished.
// We can transition to finished now and need to clean everything up.
self.state = .finished
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .upstreamThrew, .upstreamsFinished:
// The iterator was dropped so we can transition to finished now.
self.state = .finished
return .none
case .finished:
// We are already finished so there is nothing left to clean up.
// This is just the references dropping afterwards.
return .none
case .modifying:
preconditionFailure("Invalid state")
}
}
mutating func taskIsStarted(
task: Task<Void, Never>,
downstreamContinuation: DownstreamContinuation
) {
switch self.state {
case .initial:
// The user called `next` and we are starting the `Task`
// to consume the upstream sequences
self.state = .combining(
task: task,
upstreams: (.init(isFinished: false), .init(isFinished: false), .init(isFinished: false)),
downstreamContinuation: downstreamContinuation,
buffer: .init()
)
case .combining, .waitingForDemand, .upstreamThrew, .upstreamsFinished, .finished:
// We only allow a single task to be created so this must never happen.
preconditionFailure("Internal inconsistency current state \(self.state) and received taskStarted()")
case .modifying:
preconditionFailure("Invalid state")
}
}
/// Actions returned by `childTaskSuspended()`.
enum ChildTaskSuspendedAction {
/// Indicates that the continuation should be resumed which will lead to calling `next` on the upstream.
case resumeContinuation(
upstreamContinuation: UnsafeContinuation<Void, Error>
)
/// Indicates that the continuation should be resumed with an Error because another upstream sequence threw.
case resumeContinuationWithError(
upstreamContinuation: UnsafeContinuation<Void, Error>,
error: Error
)
}
mutating func childTaskSuspended(
baseIndex: Int,
continuation: UnsafeContinuation<Void, Error>
) -> ChildTaskSuspendedAction? {
switch self.state {
case .initial:
// Child tasks are only created after we transitioned to `zipping`
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended()")
case .upstreamsFinished:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamThrew()")
case .waitingForDemand(let task, var upstreams, let buffer):
self.state = .modifying
switch baseIndex {
case 0:
upstreams.0.continuation = continuation
case 1:
upstreams.1.continuation = continuation
case 2:
upstreams.2.continuation = continuation
default:
preconditionFailure(
"Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)"
)
}
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .none
case .combining:
// We are currently combining and need to resume any upstream until we transition to waitingForDemand
return .resumeContinuation(upstreamContinuation: continuation)
case .upstreamThrew, .finished:
// Since cancellation is cooperative it might be that child tasks are still getting
// suspended even though we already cancelled them. We must tolerate this and just resume
// the continuation with an error.
return .resumeContinuationWithError(
upstreamContinuation: continuation,
error: CancellationError()
)
case .modifying:
preconditionFailure("Invalid state")
}
}
/// Actions returned by `elementProduced()`.
enum ElementProducedAction {
/// Indicates that the downstream continuation should be resumed with the element.
case resumeContinuation(
downstreamContinuation: DownstreamContinuation,
result: Result<(Base1.Element, Base2.Element, Base3.Element?)?, Error>
)
}
mutating func elementProduced(_ result: (Base1.Element?, Base2.Element?, Base3.Element?)) -> ElementProducedAction? {
switch self.state {
case .initial:
// Child tasks that are producing elements are only created after we transitioned to `zipping`
preconditionFailure("Internal inconsistency current state \(self.state) and received elementProduced()")
case .upstreamsFinished:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamThrew()")
case .waitingForDemand(let task, var upstreams, var buffer):
// We got an element in late. This can happen since we race the upstreams.
// We have to store the new tuple in our buffer and remember the upstream states.
self.state = .modifying
switch result {
case (.some(let first), .none, .none):
buffer.append((first, upstreams.1.element!, upstreams.2.element))
upstreams.0.element = first
case (.none, .some(let second), .none):
buffer.append((upstreams.0.element!, second, upstreams.2.element))
upstreams.1.element = second
case (.none, .none, .some(let third)):
buffer.append((upstreams.0.element!, upstreams.1.element!, third))
upstreams.2.element = third
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received elementProduced()")
}
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .none
case .combining(let task, var upstreams, let downstreamContinuation, let buffer):
precondition(
buffer.isEmpty,
"Internal inconsistency current state \(self.state) and the buffer is not empty"
)
self.state = .modifying
switch result {
case (.some(let first), .none, .none):
upstreams.0.element = first
case (.none, .some(let second), .none):
upstreams.1.element = second
case (.none, .none, .some(let third)):
upstreams.2.element = third
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received elementProduced()")
}
// Implementing this for the two arities without variadic generics is a bit awkward sadly.
if let first = upstreams.0.element,
let second = upstreams.1.element,
let third = upstreams.2.element
{
// We got an element from each upstream so we can resume the downstream now
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .resumeContinuation(
downstreamContinuation: downstreamContinuation,
result: .success((first, second, third))
)
} else if let first = upstreams.0.element,
let second = upstreams.1.element,
self.numberOfUpstreamSequences == 2
{
// We got an element from each upstream so we can resume the downstream now
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .resumeContinuation(
downstreamContinuation: downstreamContinuation,
result: .success((first, second, nil))
)
} else {
// We are still waiting for one of the upstreams to produce an element
self.state = .combining(
task: task,
upstreams: (
.init(
continuation: upstreams.0.continuation,
element: upstreams.0.element,
isFinished: upstreams.0.isFinished
),
.init(
continuation: upstreams.1.continuation,
element: upstreams.1.element,
isFinished: upstreams.1.isFinished
),
.init(
continuation: upstreams.2.continuation,
element: upstreams.2.element,
isFinished: upstreams.2.isFinished
)
),
downstreamContinuation: downstreamContinuation,
buffer: buffer
)
return .none
}
case .upstreamThrew, .finished:
// Since cancellation is cooperative it might be that child tasks
// are still producing elements after we finished.
// We are just going to drop them since there is nothing we can do
return .none
case .modifying:
preconditionFailure("Invalid state")
}
}
/// Actions returned by `upstreamFinished()`.
enum UpstreamFinishedAction {
/// Indicates the task and the upstream continuations should be cancelled.
case cancelTaskAndUpstreamContinuations(
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
/// Indicates that the downstream continuation should be resumed with `nil` and
/// the task and the upstream continuations should be cancelled.
case resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: DownstreamContinuation,
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
}
mutating func upstreamFinished(baseIndex: Int) -> UpstreamFinishedAction? {
switch self.state {
case .initial:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .upstreamsFinished:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .waitingForDemand(let task, var upstreams, let buffer):
// One of the upstreams finished.
self.state = .modifying
switch baseIndex {
case 0:
upstreams.0.isFinished = true
case 1:
upstreams.1.isFinished = true
case 2:
upstreams.2.isFinished = true
default:
preconditionFailure(
"Internal inconsistency current state \(self.state) and received upstreamFinished() with base index \(baseIndex)"
)
}
if upstreams.0.isFinished && upstreams.1.isFinished && upstreams.2.isFinished {
// All upstreams finished we can transition to either finished or upstreamsFinished now
if buffer.isEmpty {
self.state = .finished
} else {
self.state = .upstreamsFinished(buffer: buffer)
}
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else if upstreams.0.isFinished && upstreams.1.isFinished && self.numberOfUpstreamSequences == 2 {
// All upstreams finished we can transition to either finished or upstreamsFinished now
if buffer.isEmpty {
self.state = .finished
} else {
self.state = .upstreamsFinished(buffer: buffer)
}
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else {
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .none
}
case .combining(let task, var upstreams, let downstreamContinuation, let buffer):
// One of the upstreams finished.
self.state = .modifying
// We need to track if an empty upstream finished.
// If that happens we can transition to finish right away.
let emptyUpstreamFinished: Bool
switch baseIndex {
case 0:
upstreams.0.isFinished = true
emptyUpstreamFinished = upstreams.0.element == nil
case 1:
upstreams.1.isFinished = true
emptyUpstreamFinished = upstreams.1.element == nil
case 2:
upstreams.2.isFinished = true
emptyUpstreamFinished = upstreams.2.element == nil
default:
preconditionFailure(
"Internal inconsistency current state \(self.state) and received upstreamFinished() with base index \(baseIndex)"
)
}
// Implementing this for the two arities without variadic generics is a bit awkward sadly.
if emptyUpstreamFinished {
// All upstreams finished
self.state = .finished
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else if upstreams.0.isFinished && upstreams.1.isFinished && upstreams.2.isFinished {
// All upstreams finished
self.state = .finished
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else if upstreams.0.isFinished && upstreams.1.isFinished && self.numberOfUpstreamSequences == 2 {
// All upstreams finished
self.state = .finished
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else {
self.state = .combining(
task: task,
upstreams: upstreams,
downstreamContinuation: downstreamContinuation,
buffer: buffer
)
return .none
}
case .upstreamThrew, .finished:
// This is just everything finishing up, nothing to do here
return .none
case .modifying:
preconditionFailure("Invalid state")
}
}
/// Actions returned by `upstreamThrew()`.
enum UpstreamThrewAction {
/// Indicates the task and the upstream continuations should be cancelled.
case cancelTaskAndUpstreamContinuations(
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
/// Indicates that the downstream continuation should be resumed with the `error` and
/// the task and the upstream continuations should be cancelled.
case resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: DownstreamContinuation,
error: Error,
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
}
mutating func upstreamThrew(_ error: Error) -> UpstreamThrewAction? {
switch self.state {
case .initial:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamThrew()")
case .upstreamsFinished:
// We need to tolerate multiple upstreams failing
return .none
case .waitingForDemand(let task, let upstreams, _):
// An upstream threw. We can cancel everything now and transition to finished.
// We just need to store the error for the next downstream demand
self.state = .upstreamThrew(
error: error
)
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .combining(let task, let upstreams, let downstreamContinuation, _):
// One of our upstreams threw. We need to transition to finished ourselves now
// and resume the downstream continuation with the error. Furthermore, we need to cancel all of
// the upstream work.
self.state = .finished
return .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
error: error,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .upstreamThrew, .finished:
// This is just everything finishing up, nothing to do here
return .none
case .modifying:
preconditionFailure("Invalid state")
}
}
/// Actions returned by `cancelled()`.
enum CancelledAction {
/// Indicates that the downstream continuation needs to be resumed and
/// task and the upstream continuations should be cancelled.
case resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: DownstreamContinuation,
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
/// Indicates that the task and the upstream continuations should be cancelled.
case cancelTaskAndUpstreamContinuations(
task: Task<Void, Never>,
upstreamContinuations: [UnsafeContinuation<Void, Error>]
)
}
mutating func cancelled() -> CancelledAction? {
switch self.state {
case .initial:
state = .finished
return .none
case .waitingForDemand(let task, let upstreams, _):
// The downstream task got cancelled so we need to cancel our upstream Task
// and resume all continuations. We can also transition to finished.
self.state = .finished
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .combining(let task, let upstreams, let downstreamContinuation, _):
// The downstream Task got cancelled so we need to cancel our upstream Task
// and resume all continuations. We can also transition to finished.
self.state = .finished
return .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .upstreamsFinished:
// We can transition to finished now
self.state = .finished
return .none
case .upstreamThrew, .finished:
// We are already finished so nothing to do here:
return .none
case .modifying:
preconditionFailure("Invalid state")
}
}
/// Actions returned by `next()`.
enum NextAction {
/// Indicates that a new `Task` should be created that consumes the sequence.
case startTask(Base1, Base2, Base3?)
/// Indicates that all upstream continuations should be resumed.
case resumeUpstreamContinuations(
upstreamContinuation: [UnsafeContinuation<Void, Error>]
)
/// Indicates that the downstream continuation should be resumed with the result.
case resumeContinuation(
downstreamContinuation: DownstreamContinuation,
result: Result<(Base1.Element, Base2.Element, Base3.Element?)?, Error>
)
/// Indicates that the downstream continuation should be resumed with `nil`.
case resumeDownstreamContinuationWithNil(DownstreamContinuation)
}
mutating func next(for continuation: DownstreamContinuation) -> NextAction {
switch self.state {
case .initial(let base1, let base2, let base3):
// This is the first time we get demand singalled so we have to start the task
// The transition to the next state is done in the taskStarted method
return .startTask(base1, base2, base3)
case .combining:
// We already got demand signalled and have suspended the downstream task
// Getting a second next calls means the iterator was transferred across Tasks which is not allowed
preconditionFailure("Internal inconsistency current state \(self.state) and received next()")
case .waitingForDemand(let task, var upstreams, var buffer):
// We got demand signalled now we have to check if there is anything buffered.
// If not we have to transition to combining and need to resume all upstream continuations now
self.state = .modifying
guard let element = buffer.popFirst() else {
let upstreamContinuations = [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
upstreams.0.continuation = nil
upstreams.1.continuation = nil
upstreams.2.continuation = nil
self.state = .combining(
task: task,
upstreams: upstreams,
downstreamContinuation: continuation,
buffer: buffer
)
return .resumeUpstreamContinuations(
upstreamContinuation: upstreamContinuations
)
}
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .resumeContinuation(
downstreamContinuation: continuation,
result: .success(element)
)
case .upstreamsFinished(var buffer):
self.state = .modifying
guard let element = buffer.popFirst() else {
self.state = .finished
return .resumeDownstreamContinuationWithNil(continuation)
}
self.state = .upstreamsFinished(buffer: buffer)
return .resumeContinuation(
downstreamContinuation: continuation,
result: .success(element)
)
case .upstreamThrew(let error):
// One of the upstreams threw and we have to return this error now.
self.state = .finished
return .resumeContinuation(downstreamContinuation: continuation, result: .failure(error))
case .finished:
// We are already finished so we are just returning `nil`
return .resumeDownstreamContinuationWithNil(continuation)
case .modifying:
preconditionFailure("Invalid state")
}
}
}
@@ -0,0 +1,379 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class CombineLatestStorage<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable
{
typealias StateMachine = CombineLatestStateMachine<Base1, Base2, Base3>
private let stateMachine: ManagedCriticalState<StateMachine>
init(_ base1: Base1, _ base2: Base2, _ base3: Base3?) {
self.stateMachine = .init(.init(base1: base1, base2: base2, base3: base3))
}
func iteratorDeinitialized() {
let action = self.stateMachine.withCriticalRegion { $0.iteratorDeinitialized() }
switch action {
case .cancelTaskAndUpstreamContinuations(
let task,
let upstreamContinuation
):
upstreamContinuation.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
}
func next() async rethrows -> (Base1.Element, Base2.Element, Base3.Element?)? {
try await withTaskCancellationHandler {
let result = await withUnsafeContinuation { continuation in
let action: StateMachine.NextAction? = self.stateMachine.withCriticalRegion { stateMachine in
let action = stateMachine.next(for: continuation)
switch action {
case .startTask(let base1, let base2, let base3):
// first iteration, we start one child task per base to iterate over them
self.startTask(
stateMachine: &stateMachine,
base1: base1,
base2: base2,
base3: base3,
downstreamContinuation: continuation
)
return nil
case .resumeContinuation:
return action
case .resumeUpstreamContinuations:
return action
case .resumeDownstreamContinuationWithNil:
return action
}
}
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .resumeContinuation(let downstreamContinuation, let result):
downstreamContinuation.resume(returning: result)
case .resumeUpstreamContinuations(let upstreamContinuations):
// bases can be iterated over for 1 iteration so their next value can be retrieved
upstreamContinuations.forEach { $0.resume() }
case .resumeDownstreamContinuationWithNil(let continuation):
// the async sequence is already finished, immediately resuming
continuation.resume(returning: .success(nil))
case .none:
break
}
}
return try result._rethrowGet()
} onCancel: {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.cancelled()
}
switch action {
case .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
}
}
private func startTask(
stateMachine: inout StateMachine,
base1: Base1,
base2: Base2,
base3: Base3?,
downstreamContinuation: StateMachine.DownstreamContinuation
) {
// This creates a new `Task` that is iterating the upstream
// sequences. We must store it to cancel it at the right times.
let task = Task {
await withThrowingTaskGroup(of: Void.self) { group in
// For each upstream sequence we are adding a child task that
// is consuming the upstream sequence
group.addTask {
var base1Iterator = base1.makeAsyncIterator()
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand.
try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.childTaskSuspended(baseIndex: 0, continuation: continuation)
}
switch action {
case .resumeContinuation(let upstreamContinuation):
upstreamContinuation.resume()
case .resumeContinuationWithError(let upstreamContinuation, let error):
upstreamContinuation.resume(throwing: error)
case .none:
break
}
}
if let element1 = try await base1Iterator.next() {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.elementProduced((element1, nil, nil))
}
switch action {
case .resumeContinuation(let downstreamContinuation, let result):
downstreamContinuation.resume(returning: result)
case .none:
break
}
} else {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamFinished(baseIndex: 0)
}
switch action {
case .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
break loop
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
break loop
case .none:
break loop
}
}
}
}
group.addTask {
var base1Iterator = base2.makeAsyncIterator()
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand.
try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.childTaskSuspended(baseIndex: 1, continuation: continuation)
}
switch action {
case .resumeContinuation(let upstreamContinuation):
upstreamContinuation.resume()
case .resumeContinuationWithError(let upstreamContinuation, let error):
upstreamContinuation.resume(throwing: error)
case .none:
break
}
}
if let element2 = try await base1Iterator.next() {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.elementProduced((nil, element2, nil))
}
switch action {
case .resumeContinuation(let downstreamContinuation, let result):
downstreamContinuation.resume(returning: result)
case .none:
break
}
} else {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamFinished(baseIndex: 1)
}
switch action {
case .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
break loop
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
break loop
case .none:
break loop
}
}
}
}
if let base3 = base3 {
group.addTask {
var base1Iterator = base3.makeAsyncIterator()
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand.
try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.childTaskSuspended(baseIndex: 2, continuation: continuation)
}
switch action {
case .resumeContinuation(let upstreamContinuation):
upstreamContinuation.resume()
case .resumeContinuationWithError(let upstreamContinuation, let error):
upstreamContinuation.resume(throwing: error)
case .none:
break
}
}
if let element3 = try await base1Iterator.next() {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.elementProduced((nil, nil, element3))
}
switch action {
case .resumeContinuation(let downstreamContinuation, let result):
downstreamContinuation.resume(returning: result)
case .none:
break
}
} else {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamFinished(baseIndex: 2)
}
switch action {
case .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
break loop
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
break loop
case .none:
break loop
}
}
}
}
}
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let error,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .none:
break
}
group.cancelAll()
}
}
}
}
stateMachine.taskIsStarted(task: task, downstreamContinuation: downstreamContinuation)
}
}
@@ -0,0 +1,110 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that emits the latest element after a given quiescence period
/// has elapsed by using a specified Clock.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func debounce<C: Clock>(
for interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
) -> AsyncDebounceSequence<Self, C> where Self: Sendable, Self.Element: Sendable {
AsyncDebounceSequence(self, interval: interval, tolerance: tolerance, clock: clock)
}
/// Creates an asynchronous sequence that emits the latest element after a given quiescence period
/// has elapsed.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func debounce(
for interval: Duration,
tolerance: Duration? = nil
) -> AsyncDebounceSequence<Self, ContinuousClock> where Self: Sendable, Self.Element: Sendable {
self.debounce(for: interval, tolerance: tolerance, clock: .continuous)
}
}
/// An `AsyncSequence` that emits the latest element after a given quiescence period
/// has elapsed.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public struct AsyncDebounceSequence<Base: AsyncSequence & Sendable, C: Clock>: Sendable where Base.Element: Sendable {
private let base: Base
private let clock: C
private let interval: C.Instant.Duration
private let tolerance: C.Instant.Duration?
/// Initializes a new ``AsyncDebounceSequence``.
///
/// - Parameters:
/// - base: The base sequence.
/// - interval: The interval to debounce.
/// - tolerance: The tolerance of the clock.
/// - clock: The clock.
init(_ base: Base, interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.base = base
self.interval = interval
self.tolerance = tolerance
self.clock = clock
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncDebounceSequence: AsyncSequence {
public typealias Element = Base.Element
public func makeAsyncIterator() -> Iterator {
let storage = DebounceStorage(
base: self.base,
interval: self.interval,
tolerance: self.tolerance,
clock: self.clock
)
return Iterator(storage: storage)
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncDebounceSequence {
public struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: DebounceStorage<Base, C>
fileprivate init(storage: DebounceStorage<Base, C>) {
self.storage = storage
}
deinit {
self.storage.iteratorDeinitialized()
}
func next() async rethrows -> Element? {
try await self.storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: DebounceStorage<Base, C>) {
self.internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await self.internalClass.next()
}
}
}
@available(*, unavailable)
extension AsyncDebounceSequence.Iterator: Sendable {}
@@ -0,0 +1,709 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
struct DebounceStateMachine<Base: AsyncSequence & Sendable, C: Clock>: Sendable where Base.Element: Sendable {
typealias Element = Base.Element
private enum State: Sendable {
/// The initial state before a call to `next` happened.
case initial(base: Base)
/// The state while we are waiting for downstream demand.
case waitingForDemand(
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?,
bufferedElement: (element: Element, deadline: C.Instant)?
)
/// The state once the downstream signalled demand but before we received
/// the first element from the upstream.
case demandSignalled(
task: Task<Void, Never>,
clockContinuation: UnsafeContinuation<C.Instant, Error>?,
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>
)
/// The state while we are consuming the upstream and waiting for the Clock.sleep to finish.
case debouncing(
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>,
currentElement: (element: Element, deadline: C.Instant)
)
/// The state once any of the upstream sequences threw an `Error`.
case upstreamFailure(
error: Error
)
/// The state once all upstream sequences finished or the downstream consumer stopped, i.e. by dropping all references
/// or by getting their `Task` cancelled.
case finished
}
/// The state machine's current state.
private var state: State
/// The interval to debounce.
private let interval: C.Instant.Duration
/// The clock.
private let clock: C
init(base: Base, clock: C, interval: C.Instant.Duration) {
self.state = .initial(base: base)
self.clock = clock
self.interval = interval
}
/// Actions returned by `iteratorDeinitialized()`.
enum IteratorDeinitializedAction {
/// Indicates that the `Task` needs to be cancelled and
/// the upstream and clock continuation need to be resumed with a `CancellationError`.
case cancelTaskAndUpstreamAndClockContinuations(
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
}
mutating func iteratorDeinitialized() -> IteratorDeinitializedAction? {
switch self.state {
case .initial:
// Nothing to do here. No demand was signalled until now
return .none
case .debouncing, .demandSignalled:
// An iterator was deinitialized while we have a suspended continuation.
preconditionFailure(
"Internal inconsistency current state \(self.state) and received iteratorDeinitialized()"
)
case .waitingForDemand(let task, let upstreamContinuation, let clockContinuation, _):
// The iterator was dropped which signals that the consumer is finished.
// We can transition to finished now and need to clean everything up.
self.state = .finished
return .cancelTaskAndUpstreamAndClockContinuations(
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: clockContinuation
)
case .upstreamFailure:
// The iterator was dropped which signals that the consumer is finished.
// We can transition to finished now. The cleanup already happened when we
// transitioned to `upstreamFailure`.
self.state = .finished
return .none
case .finished:
// We are already finished so there is nothing left to clean up.
// This is just the references dropping afterwards.
return .none
}
}
mutating func taskStarted(
_ task: Task<Void, Never>,
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>
) {
switch self.state {
case .initial:
// The user called `next` and we are starting the `Task`
// to consume the upstream sequence
self.state = .demandSignalled(
task: task,
clockContinuation: nil,
downstreamContinuation: downstreamContinuation
)
case .debouncing, .demandSignalled, .waitingForDemand, .upstreamFailure, .finished:
// We only a single iterator to be created so this must never happen.
preconditionFailure("Internal inconsistency current state \(self.state) and received taskStarted()")
}
}
/// Actions returned by `upstreamTaskSuspended()`.
enum UpstreamTaskSuspendedAction {
/// Indicates that the continuation should be resumed which will lead to calling `next` on the upstream.
case resumeContinuation(
upstreamContinuation: UnsafeContinuation<Void, Error>
)
/// Indicates that the continuation should be resumed with an Error because another upstream sequence threw.
case resumeContinuationWithError(
upstreamContinuation: UnsafeContinuation<Void, Error>,
error: Error
)
}
mutating func upstreamTaskSuspended(_ continuation: UnsafeContinuation<Void, Error>) -> UpstreamTaskSuspendedAction? {
switch self.state {
case .initial:
// Child tasks are only created after we transitioned to `merging`
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended()")
case .waitingForDemand(_, .some, _, _), .debouncing(_, .some, _, _):
// We already have an upstream continuation so we can never get a second one
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended()")
case .upstreamFailure:
// The upstream already failed so it should never suspend again since the child task
// should have exited
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended()")
case .waitingForDemand(let task, .none, let clockContinuation, let bufferedElement):
// The upstream task is ready to consume the next element
// we are just waiting to get demand
self.state = .waitingForDemand(
task: task,
upstreamContinuation: continuation,
clockContinuation: clockContinuation,
bufferedElement: bufferedElement
)
return .none
case .demandSignalled:
// It can happen that the demand got signalled before our upstream suspended for the first time
// We need to resume it right away to demand the first element from the upstream
return .resumeContinuation(upstreamContinuation: continuation)
case .debouncing(_, .none, _, _):
// We are currently debouncing and the upstream task suspended again
// We need to resume the continuation right away so that it continues to
// consume new elements from the upstream
return .resumeContinuation(upstreamContinuation: continuation)
case .finished:
// Since cancellation is cooperative it might be that child tasks are still getting
// suspended even though we already cancelled them. We must tolerate this and just resume
// the continuation with an error.
return .resumeContinuationWithError(
upstreamContinuation: continuation,
error: CancellationError()
)
}
}
/// Actions returned by `elementProduced()`.
enum ElementProducedAction {
/// Indicates that the clock continuation should be resumed to start the `Clock.sleep`.
case resumeClockContinuation(
clockContinuation: UnsafeContinuation<C.Instant, Error>?,
deadline: C.Instant
)
}
mutating func elementProduced(_ element: Element, deadline: C.Instant) -> ElementProducedAction? {
switch self.state {
case .initial:
// Child tasks that are producing elements are only created after we transitioned to `merging`
preconditionFailure("Internal inconsistency current state \(self.state) and received elementProduced()")
case .waitingForDemand(_, _, _, .some):
// We can only ever buffer one element because of the race of both child tasks
// After that element got buffered we are not resuming the upstream continuation
// and should never get another element until we get downstream demand signalled
preconditionFailure("Internal inconsistency current state \(self.state) and received elementProduced()")
case .upstreamFailure:
// The upstream already failed so it should never have produced another element
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended()")
case .waitingForDemand(let task, let upstreamContinuation, let clockContinuation, .none):
// We got an element even though we don't have an outstanding demand
// this can happen because we race the upstream and Clock child tasks
// and the upstream might finish after the Clock. We just need
// to buffer the element for the next demand.
self.state = .waitingForDemand(
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: clockContinuation,
bufferedElement: (element, deadline)
)
return .none
case .demandSignalled(let task, let clockContinuation, let downstreamContinuation):
// This is the first element that got produced after we got demand signalled
// We can now transition to debouncing and start the Clock.sleep
self.state = .debouncing(
task: task,
upstreamContinuation: nil,
downstreamContinuation: downstreamContinuation,
currentElement: (element, deadline)
)
let deadline = self.clock.now.advanced(by: self.interval)
return .resumeClockContinuation(
clockContinuation: clockContinuation,
deadline: deadline
)
case .debouncing(let task, let upstreamContinuation, let downstreamContinuation, _):
// We just got another element and the Clock hasn't finished sleeping yet
// We just need to store the new element
self.state = .debouncing(
task: task,
upstreamContinuation: upstreamContinuation,
downstreamContinuation: downstreamContinuation,
currentElement: (element, deadline)
)
return .none
case .finished:
// Since cancellation is cooperative it might be that child tasks
// are still producing elements after we finished.
// We are just going to drop them since there is nothing we can do
return .none
}
}
/// Actions returned by `upstreamFinished()`.
enum UpstreamFinishedAction {
/// Indicates that the task and the clock continuation should be cancelled.
case cancelTaskAndClockContinuation(
task: Task<Void, Never>,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
/// Indicates that the downstream continuation should be resumed with `nil` and
/// the task and the upstream continuation should be cancelled.
case resumeContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>,
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
/// Indicates that the downstream continuation should be resumed with `nil` and
/// the task and the upstream continuation should be cancelled.
case resumeContinuationWithElementAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>,
element: Element,
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
}
mutating func upstreamFinished() -> UpstreamFinishedAction? {
switch self.state {
case .initial:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .waitingForDemand(_, .some, _, _):
// We will never receive an upstream finished and have an outstanding continuation
// since we only receive finish after resuming the upstream continuation
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .waitingForDemand(_, .none, _, .some):
// We will never receive an upstream finished while we have a buffered element
// To get there we would need to have received the buffered element and then
// received upstream finished all while waiting for demand; however, we should have
// never demanded the next element from upstream in the first place
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .upstreamFailure:
// The upstream already failed so it should never have finished again
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended()")
case .waitingForDemand(let task, .none, let clockContinuation, .none):
// We don't have any buffered element so we can just go ahead
// and transition to finished and cancel everything
self.state = .finished
return .cancelTaskAndClockContinuation(
task: task,
clockContinuation: clockContinuation
)
case .demandSignalled(let task, let clockContinuation, let downstreamContinuation):
// We demanded the next element from the upstream after we got signalled demand
// and the upstream finished. This means we need to resume the downstream with nil
self.state = .finished
return .resumeContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuation: nil,
clockContinuation: clockContinuation
)
case .debouncing(let task, let upstreamContinuation, let downstreamContinuation, let currentElement):
// We are debouncing and the upstream finished. At this point
// we can just resume the downstream continuation with element and cancel everything else
self.state = .finished
return .resumeContinuationWithElementAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: downstreamContinuation,
element: currentElement.element,
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: nil
)
case .finished:
// This is just everything finishing up, nothing to do here
return .none
}
}
/// Actions returned by `upstreamThrew()`.
enum UpstreamThrewAction {
/// Indicates that the task and the clock continuation should be cancelled.
case cancelTaskAndClockContinuation(
task: Task<Void, Never>,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
/// Indicates that the downstream continuation should be resumed with the `error` and
/// the task and the upstream continuation should be cancelled.
case resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuation(
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>,
error: Error,
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
}
mutating func upstreamThrew(_ error: Error) -> UpstreamThrewAction? {
switch self.state {
case .initial:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamThrew()")
case .waitingForDemand(_, .some, _, _):
// We will never receive an upstream threw and have an outstanding continuation
// since we only receive threw after resuming the upstream continuation
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .waitingForDemand(_, .none, _, .some):
// We will never receive an upstream threw while we have a buffered element
// To get there we would need to have received the buffered element and then
// received upstream threw all while waiting for demand; however, we should have
// never demanded the next element from upstream in the first place
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished()")
case .upstreamFailure:
// We need to tolerate multiple upstreams failing
return .none
case .waitingForDemand(let task, .none, let clockContinuation, .none):
// We don't have outstanding demand to deliver the error to, so store it
// until the downstream calls next and cancel everything.
self.state = .upstreamFailure(error: error)
return .cancelTaskAndClockContinuation(
task: task,
clockContinuation: clockContinuation
)
case .demandSignalled(let task, let clockContinuation, let downstreamContinuation):
// We demanded the next element from the upstream after we got signalled demand
// and the upstream threw. This means we need to resume the downstream with the error
self.state = .finished
return .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuation(
downstreamContinuation: downstreamContinuation,
error: error,
task: task,
upstreamContinuation: nil,
clockContinuation: clockContinuation
)
case .debouncing(let task, let upstreamContinuation, let downstreamContinuation, _):
// We are debouncing and the upstream threw. At this point
// we can just resume the downstream continuation with error and cancel everything else
self.state = .finished
return .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuation(
downstreamContinuation: downstreamContinuation,
error: error,
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: nil
)
case .finished:
// This is just everything finishing up, nothing to do here
return .none
}
}
/// Actions returned by `clockTaskSuspended()`.
enum ClockTaskSuspendedAction {
/// Indicates that the continuation should be resumed which will lead to calling `sleep` on the Clock.
case resumeContinuation(
clockContinuation: UnsafeContinuation<C.Instant, Error>,
deadline: C.Instant
)
/// Indicates that the continuation should be resumed with an Error because another upstream sequence threw.
case resumeContinuationWithError(
clockContinuation: UnsafeContinuation<C.Instant, Error>,
error: Error
)
}
mutating func clockTaskSuspended(_ continuation: UnsafeContinuation<C.Instant, Error>) -> ClockTaskSuspendedAction? {
switch self.state {
case .initial:
// Child tasks are only created after we transitioned to `merging`
preconditionFailure("Internal inconsistency current state \(self.state) and received clockTaskSuspended()")
case .waitingForDemand(_, _, .some, _):
// We already have a clock continuation so we can never get a second one
preconditionFailure("Internal inconsistency current state \(self.state) and received clockTaskSuspended()")
case .demandSignalled(_, .some, _):
// We already have a clock continuation so we can never get a second one
preconditionFailure("Internal inconsistency current state \(self.state) and received clockTaskSuspended()")
case .waitingForDemand(let task, let upstreamContinuation, .none, let bufferedElement):
// The clock child task suspended and we just need to store the continuation until
// demand is signalled
self.state = .waitingForDemand(
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: continuation,
bufferedElement: bufferedElement
)
return .none
case .demandSignalled(let task, .none, let downstreamContinuation):
// The demand was signalled but we haven't gotten the first element from the upstream yet
// so we need to stay in this state and do nothing
self.state = .demandSignalled(
task: task,
clockContinuation: continuation,
downstreamContinuation: downstreamContinuation
)
return .none
case .debouncing(_, _, _, let currentElement):
// We are currently debouncing and the Clock task suspended
// We need to resume the continuation right away.
return .resumeContinuation(
clockContinuation: continuation,
deadline: currentElement.deadline
)
case .upstreamFailure:
// The upstream failed while we were waiting to suspend the clock task again
// The task should have already been cancelled and we just need to cancel the continuation
return .resumeContinuationWithError(
clockContinuation: continuation,
error: CancellationError()
)
case .finished:
// Since cancellation is cooperative it might be that child tasks are still getting
// suspended even though we already cancelled them. We must tolerate this and just resume
// the continuation with an error.
return .resumeContinuationWithError(
clockContinuation: continuation,
error: CancellationError()
)
}
}
/// Actions returned by `clockSleepFinished()`.
enum ClockSleepFinishedAction {
/// Indicates that the downstream continuation should be resumed with the given element.
case resumeDownstreamContinuation(
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>,
element: Element
)
}
mutating func clockSleepFinished() -> ClockSleepFinishedAction? {
switch self.state {
case .initial:
// Child tasks are only created after we transitioned to `merging`
preconditionFailure("Internal inconsistency current state \(self.state) and received clockSleepFinished()")
case .waitingForDemand:
// This can never happen since we kicked-off the Clock.sleep because we got signalled demand.
preconditionFailure("Internal inconsistency current state \(self.state) and received clockSleepFinished()")
case .demandSignalled:
// This can never happen since we are still waiting for the first element until we resume the Clock sleep.
preconditionFailure("Internal inconsistency current state \(self.state) and received clockSleepFinished()")
case .debouncing(let task, let upstreamContinuation, let downstreamContinuation, let currentElement):
guard currentElement.deadline <= self.clock.now else {
// The deadline is still in the future so we need to sleep again
return .none
}
// The deadline for the last produced element expired and we can forward it to the downstream
self.state = .waitingForDemand(
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: nil,
bufferedElement: nil
)
return .resumeDownstreamContinuation(
downstreamContinuation: downstreamContinuation,
element: currentElement.element
)
case .upstreamFailure:
// The upstream failed before the Clock.sleep finished
// We already cleaned everything up so nothing left to do here.
return .none
case .finished:
// The upstream failed before the Clock.sleep finished
// We already cleaned everything up so nothing left to do here.
return .none
}
}
/// Actions returned by `cancelled()`.
enum CancelledAction {
/// Indicates that the downstream continuation needs to be resumed and
/// task and the upstream continuations should be cancelled.
case resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: UnsafeContinuation<Result<Element?, Error>, Never>,
task: Task<Void, Never>,
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?
)
}
mutating func cancelled() -> CancelledAction? {
switch self.state {
case .initial:
state = .finished
return .none
case .waitingForDemand:
// We got cancelled before we event got any demand. This can happen if a cancelled task
// calls next and the onCancel handler runs first. We can transition to finished right away.
self.state = .finished
return .none
case .demandSignalled(let task, let clockContinuation, let downstreamContinuation):
// We got cancelled while we were waiting for the first upstream element
// We can cancel everything at this point and return nil
self.state = .finished
return .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuation: nil,
clockContinuation: clockContinuation
)
case .debouncing(let task, let upstreamContinuation, let downstreamContinuation, _):
// We got cancelled while debouncing.
// We can cancel everything at this point and return nil
self.state = .finished
return .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuation: upstreamContinuation,
clockContinuation: nil
)
case .upstreamFailure:
// An upstream already threw and we cancelled everything already.
// We should stay in the upstream failure state until the error is consumed
return .none
case .finished:
// We are already finished so nothing to do here:
self.state = .finished
return .none
}
}
/// Actions returned by `next()`.
enum NextAction {
/// Indicates that a new `Task` should be created that consumes the sequence.
case startTask(Base)
case resumeUpstreamContinuation(
upstreamContinuation: UnsafeContinuation<Void, Error>?
)
case resumeUpstreamAndClockContinuation(
upstreamContinuation: UnsafeContinuation<Void, Error>?,
clockContinuation: UnsafeContinuation<C.Instant, Error>?,
deadline: C.Instant
)
/// Indicates that the downstream continuation should be resumed with `nil`.
case resumeDownstreamContinuationWithNil(UnsafeContinuation<Result<Element?, Error>, Never>)
/// Indicates that the downstream continuation should be resumed with the error.
case resumeDownstreamContinuationWithError(
UnsafeContinuation<Result<Element?, Error>, Never>,
Error
)
}
mutating func next(for continuation: UnsafeContinuation<Result<Element?, Error>, Never>) -> NextAction {
switch self.state {
case .initial(let base):
// This is the first time we get demand singalled so we have to start the task
// The transition to the next state is done in the taskStarted method
return .startTask(base)
case .demandSignalled, .debouncing:
// We already got demand signalled and have suspended the downstream task
// Getting a second next calls means the iterator was transferred across Tasks which is not allowed
preconditionFailure("Internal inconsistency current state \(self.state) and received next()")
case .waitingForDemand(let task, let upstreamContinuation, let clockContinuation, let bufferedElement):
guard let bufferedElement = bufferedElement else {
// We don't have a buffered element so have to resume the upstream continuation
// to get the first one and transition to demandSignalled
self.state = .demandSignalled(
task: task,
clockContinuation: clockContinuation,
downstreamContinuation: continuation
)
return .resumeUpstreamContinuation(upstreamContinuation: upstreamContinuation)
}
// We already got an element from the last buffered one
// We can kick of the clock and upstream consumption right away and transition to debouncing
self.state = .debouncing(
task: task,
upstreamContinuation: nil,
downstreamContinuation: continuation,
currentElement: bufferedElement
)
return .resumeUpstreamAndClockContinuation(
upstreamContinuation: upstreamContinuation,
clockContinuation: clockContinuation,
deadline: bufferedElement.deadline
)
case .upstreamFailure(let error):
// The upstream threw and haven't delivered the error yet
// Let's deliver it and transition to finished
self.state = .finished
return .resumeDownstreamContinuationWithError(continuation, error)
case .finished:
// We are already finished so we are just returning `nil`
return .resumeDownstreamContinuationWithNil(continuation)
}
}
}
@@ -0,0 +1,327 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
final class DebounceStorage<Base: AsyncSequence & Sendable, C: Clock>: Sendable where Base.Element: Sendable {
typealias Element = Base.Element
/// The state machine protected with a lock.
private let stateMachine: ManagedCriticalState<DebounceStateMachine<Base, C>>
/// The interval to debounce.
private let interval: C.Instant.Duration
/// The tolerance for the clock.
private let tolerance: C.Instant.Duration?
/// The clock.
private let clock: C
init(base: Base, interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.stateMachine = .init(.init(base: base, clock: clock, interval: interval))
self.interval = interval
self.tolerance = tolerance
self.clock = clock
}
func iteratorDeinitialized() {
let action = self.stateMachine.withCriticalRegion { $0.iteratorDeinitialized() }
switch action {
case .cancelTaskAndUpstreamAndClockContinuations(
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
case .none:
break
}
}
func next() async rethrows -> Element? {
// We need to handle cancellation here because we are creating a continuation
// and because we need to cancel the `Task` we created to consume the upstream
return try await withTaskCancellationHandler {
// We always suspend since we can never return an element right away
let result: Result<Element?, Error> = await withUnsafeContinuation { continuation in
let action: DebounceStateMachine<Base, C>.NextAction? = self.stateMachine.withCriticalRegion {
let action = $0.next(for: continuation)
switch action {
case .startTask(let base):
self.startTask(
stateMachine: &$0,
base: base,
downstreamContinuation: continuation
)
return nil
case .resumeUpstreamContinuation:
return action
case .resumeUpstreamAndClockContinuation:
return action
case .resumeDownstreamContinuationWithNil:
return action
case .resumeDownstreamContinuationWithError:
return action
}
}
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .resumeUpstreamContinuation(let upstreamContinuation):
// This is signalling the upstream task that is consuming the upstream
// sequence to signal demand.
upstreamContinuation?.resume(returning: ())
case .resumeUpstreamAndClockContinuation(let upstreamContinuation, let clockContinuation, let deadline):
// This is signalling the upstream task that is consuming the upstream
// sequence to signal demand and start the clock task.
upstreamContinuation?.resume(returning: ())
clockContinuation?.resume(returning: deadline)
case .resumeDownstreamContinuationWithNil(let continuation):
continuation.resume(returning: .success(nil))
case .resumeDownstreamContinuationWithError(let continuation, let error):
continuation.resume(returning: .failure(error))
case .none:
break
}
}
return try result._rethrowGet()
} onCancel: {
let action = self.stateMachine.withCriticalRegion { $0.cancelled() }
switch action {
case .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
case .none:
break
}
}
}
private func startTask(
stateMachine: inout DebounceStateMachine<Base, C>,
base: Base,
downstreamContinuation: UnsafeContinuation<Result<Base.Element?, Error>, Never>
) {
let task = Task {
await withThrowingTaskGroup(of: Void.self) { group in
// The task that consumes the upstream sequence
group.addTask {
var iterator = base.makeAsyncIterator()
// This is our upstream consumption loop
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand
// and until the Clock sleep finished.
try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { $0.upstreamTaskSuspended(continuation) }
switch action {
case .resumeContinuation(let continuation):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: ())
case .resumeContinuationWithError(let continuation, let error):
// This happens if the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
// We got signalled from the downstream that we have demand so let's
// request a new element from the upstream
if let element = try await iterator.next() {
let action = self.stateMachine.withCriticalRegion {
let deadline = self.clock.now.advanced(by: self.interval)
return $0.elementProduced(element, deadline: deadline)
}
switch action {
case .resumeClockContinuation(let clockContinuation, let deadline):
clockContinuation?.resume(returning: deadline)
case .none:
break
}
} else {
// The upstream returned `nil` which indicates that it finished
let action = self.stateMachine.withCriticalRegion { $0.upstreamFinished() }
// All of this is mostly cleanup around the Task and the outstanding
// continuations used for signalling.
switch action {
case .cancelTaskAndClockContinuation(let task, let clockContinuation):
task.cancel()
clockContinuation?.resume(throwing: CancellationError())
break loop
case .resumeContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
break loop
case .resumeContinuationWithElementAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let element,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(element))
break loop
case .none:
break loop
}
}
}
}
group.addTask {
// This is our clock scheduling loop
loop: while true {
do {
// We are creating a continuation sleeping on the Clock.
// This continuation is only resumed if the downstream consumer called `next`.
let deadline: C.Instant = try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion {
$0.clockTaskSuspended(continuation)
}
switch action {
case .resumeContinuation(let continuation, let deadline):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: deadline)
case .resumeContinuationWithError(let continuation, let error):
// This happens if the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
try await self.clock.sleep(until: deadline, tolerance: self.tolerance)
let action = self.stateMachine.withCriticalRegion { $0.clockSleepFinished() }
switch action {
case .resumeDownstreamContinuation(let downstreamContinuation, let element):
downstreamContinuation.resume(returning: .success(element))
case .none:
break
}
} catch {
// The only error that we expect is the `CancellationError`
// thrown from the Clock.sleep or from the withUnsafeContinuation.
// This happens if we are cleaning everything up. We can just drop that error and break our loop
precondition(
error is CancellationError,
"Received unexpected error \(error) in the Clock loop"
)
break loop
}
}
}
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuation(
let downstreamContinuation,
let error,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .cancelTaskAndClockContinuation(
let task,
let clockContinuation
):
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
case .none:
break
}
}
group.cancelAll()
}
}
}
stateMachine.taskStarted(task, downstreamContinuation: downstreamContinuation)
}
}
+88
View File
@@ -0,0 +1,88 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension Dictionary {
/// Creates a new dictionary from the key-value pairs in the given asynchronous sequence.
///
/// You use this initializer to create a dictionary when you have an asynchronous sequence
/// of key-value tuples with unique keys. Passing an asynchronous sequence with duplicate
/// keys to this initializer results in a runtime error. If your
/// asynchronous sequence might have duplicate keys, use the
/// `Dictionary(_:uniquingKeysWith:)` initializer instead.
///
/// - Parameter keysAndValues: An asynchronous sequence of key-value pairs to use for
/// the new dictionary. Every key in `keysAndValues` must be unique.
/// - Precondition: The sequence must not have duplicate keys.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public init<S: AsyncSequence>(uniqueKeysWithValues keysAndValues: S) async rethrows
where S.Element == (Key, Value) {
self.init(uniqueKeysWithValues: try await Array(keysAndValues))
}
/// Creates a new dictionary from the key-value pairs in the given asynchronous sequence,
/// using a combining closure to determine the value for any duplicate keys.
///
/// You use this initializer to create a dictionary when you have a sequence
/// of key-value tuples that might have duplicate keys. As the dictionary is
/// built, the initializer calls the `combine` closure with the current and
/// new values for any duplicate keys. Pass a closure as `combine` that
/// returns the value to use in the resulting dictionary: The closure can
/// choose between the two values, combine them to produce a new value, or
/// even throw an error.
///
/// - Parameters:
/// - keysAndValues: An asynchronous sequence of key-value pairs to use for the new
/// dictionary.
/// - combine: A closure that is called with the values for any duplicate
/// keys that are encountered. The closure returns the desired value for
/// the final dictionary, or throws an error if building the dictionary
/// can't proceed.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public init<S: AsyncSequence>(
_ keysAndValues: S,
uniquingKeysWith combine: (Value, Value) async throws -> Value
) async rethrows where S.Element == (Key, Value) {
self.init()
for try await (key, value) in keysAndValues {
if let existing = self[key] {
self[key] = try await combine(existing, value)
} else {
self[key] = value
}
}
}
/// Creates a new dictionary whose keys are the groupings returned by the
/// given closure and whose values are arrays of the elements that returned
/// each key.
///
/// The arrays in the "values" position of the new dictionary each contain at
/// least one element, with the elements in the same order as the source
/// asynchronous sequence.
///
/// - Parameters:
/// - values: An asynchronous sequence of values to group into a dictionary.
/// - keyForValue: A closure that returns a key for each element in
/// `values`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public init<S: AsyncSequence>(grouping values: S, by keyForValue: (S.Element) async throws -> Key) async rethrows
where Value == [S.Element] {
self.init()
for try await value in values {
let key = try await keyForValue(value)
self[key, default: []].append(value)
}
}
}
@@ -0,0 +1,89 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.3, *)
extension AsyncSequence where Self: Sendable {
/// Transforms elements into new asynchronous sequences, emitting elements
/// from the most recent inner sequence.
///
/// When a new element is emitted by this sequence, the `transform`
/// is called to produce a new inner sequence. Iteration on the
/// previous inner sequence is cancelled, and iteration begins
/// on the new one.
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable
where T.Failure == Failure, T.Element: Sendable, Element: Sendable {
return AsyncFlatMapLatestSequence(self, transform: transform)
}
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, Failure> & Sendable
where T.Failure == Never, T.Element: Sendable, Element: Sendable {
return AsyncFlatMapLatestSequence(self) {
transform($0).mapError { _ -> Failure in
fatalError()
}
}
}
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable
where Failure == Never, T.Element: Sendable, Element: Sendable {
return AsyncFlatMapLatestSequence(
self.mapError { _ -> T.Failure in
fatalError()
},
transform: transform
)
}
}
@available(AsyncAlgorithms 1.3, *)
struct AsyncFlatMapLatestSequence<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>: AsyncSequence,
Sendable
where Base.Element: Sendable, Inner.Element: Sendable, Base.Failure == Inner.Failure {
typealias Element = Inner.Element
typealias Failure = Inner.Failure
let base: Base
let transform: @Sendable (Base.Element) -> Inner
init(_ base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.base = base
self.transform = transform
}
func makeAsyncIterator() -> Iterator {
return Iterator(base: base, transform: transform)
}
struct Iterator: AsyncIteratorProtocol, Sendable {
let storage: FlatMapLatestStorage<Base, Inner>
init(base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.storage = FlatMapLatestStorage(base: base, transform: transform)
}
func next() async rethrows -> Element? {
return try await storage.next(isolation: nil)
}
func next(isolation: isolated (any Actor)? = #isolation) async throws(Failure) -> Element? {
return try await storage.next(isolation: isolation)
}
}
}
@@ -0,0 +1,531 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
import DequeModule
@available(AsyncAlgorithms 1.3, *)
struct FlatMapLatestStateMachine<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>
where Base.Element: Sendable, Inner.Element: Sendable {
typealias Element = Inner.Element
private enum State {
case initial(Base)
case running(
outerTask: Task<Void, Never>?,
outerContinuation: UnsafeContinuation<Void, Error>?,
innerTask: Task<Void, Never>?,
innerContinuation: UnsafeContinuation<Void, Error>?,
downstreamContinuation: UnsafeContinuation<Element?, Error>?,
buffer: Deque<Result<Element, Error>>,
generation: Int,
outerFinished: Bool
)
case finished
}
private var state: State
private let transform: @Sendable (Base.Element) -> Inner
init(base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.state = .initial(base)
self.transform = transform
}
enum NextAction {
case returnElement(Element)
case returnNil
case throwError(Error)
case startOuterTask(Base)
case suspend
}
enum Action {
case startInnerTask(
Inner,
generation: Int,
previousTask: Task<Void, Never>?,
previousContinuation: UnsafeContinuation<Void, Error>?
)
case cancelInnerTask(Task<Void, Never>, UnsafeContinuation<Void, Error>?)
case resumeDownstream(UnsafeContinuation<Element?, Error>, Result<Element?, Error>)
case resumeOuterContinuation(UnsafeContinuation<Void, Error>)
case cancelTasks(
Task<Void, Never>?,
Task<Void, Never>?,
UnsafeContinuation<Void, Error>?,
UnsafeContinuation<Void, Error>?
)
case none
}
enum SuspendAction {
case resumeOuterContinuation(UnsafeContinuation<Void, Error>)
case resumeInnerContinuation(UnsafeContinuation<Void, Error>)
case none
}
mutating func next() -> NextAction {
switch state {
case .initial(let base):
return .startOuterTask(base)
case .running(
let outerTask,
let outerCont,
let innerTask,
let innerCont,
let downstreamCont,
var buffer,
let generation,
let outerFinished
):
if let result = buffer.popFirst() {
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
switch result {
case .success(let element): return .returnElement(element)
case .failure(let error): return .throwError(error)
}
} else if outerFinished && innerTask == nil {
state = .finished
return .returnNil
} else {
return .suspend
}
case .finished:
return .returnNil
}
}
mutating func next(for continuation: UnsafeContinuation<Element?, Error>) -> SuspendAction {
switch state {
case .initial:
fatalError("Should be started")
case .running(
let outerTask,
let outerCont,
let innerTask,
let innerCont,
let downstreamCont,
let buffer,
let generation,
let outerFinished
):
precondition(downstreamCont == nil, "Already have downstream continuation")
precondition(buffer.isEmpty, "Buffer should be empty if suspending")
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: continuation,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
// Resume waiting tasks to produce elements for downstream demand
if let innerCont = innerCont {
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: nil,
downstreamContinuation: continuation,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .resumeInnerContinuation(innerCont)
}
if let outerCont = outerCont {
state = .running(
outerTask: outerTask,
outerContinuation: nil,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: continuation,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .resumeOuterContinuation(outerCont)
}
return .none
case .finished:
continuation.resume(returning: nil)
return .none
}
}
mutating func outerTaskStarted(_ task: Task<Void, Never>) {
switch state {
case .initial:
// Transition to running
state = .running(
outerTask: task,
outerContinuation: nil,
innerTask: nil,
innerContinuation: nil,
downstreamContinuation: nil,
buffer: Deque(),
generation: 0,
outerFinished: false
)
default:
fatalError("Invalid state transition")
}
}
mutating func outerTaskSuspended(_ continuation: UnsafeContinuation<Void, Error>) -> SuspendAction {
switch state {
case .running(
let outerTask,
_,
let innerTask,
let innerCont,
let downstreamCont,
let buffer,
let generation,
let outerFinished
):
// If we have downstream demand, resume immediately
if downstreamCont != nil {
return .resumeOuterContinuation(continuation)
}
state = .running(
outerTask: outerTask,
outerContinuation: continuation,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .none
case .finished:
continuation.resume(throwing: CancellationError())
return .none
default:
fatalError("Invalid state")
}
}
mutating func outerElementProduced(_ element: Base.Element) -> Action {
switch state {
case .running(
let outerTask,
_,
let innerTask,
let innerCont,
let downstreamCont,
let buffer,
var generation,
let outerFinished
):
// New element from outer -> Cancel previous inner, start new inner
let newInner = transform(element)
generation += 1
state = .running(
outerTask: outerTask,
outerContinuation: nil, // We just consumed the continuation by producing
innerTask: nil, // Will be set by innerTaskStarted
innerContinuation: nil,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .startInnerTask(newInner, generation: generation, previousTask: innerTask, previousContinuation: innerCont)
case .finished:
return .none
default:
fatalError("Invalid state")
}
}
mutating func innerTaskStarted(_ task: Task<Void, Never>, generation: Int) {
switch state {
case .running(
let outerTask,
let outerCont,
_,
let innerCont,
let downstreamCont,
let buffer,
let currentGen,
let outerFinished
):
if generation != currentGen {
// Stale task from previous generation, ignore it
return
}
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: task,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
default:
// If finished, we don't care
break
}
}
mutating func innerTaskSuspended(_ continuation: UnsafeContinuation<Void, Error>, generation: Int) -> SuspendAction {
switch state {
case .running(
let outerTask,
let outerCont,
let innerTask,
_,
let downstreamCont,
let buffer,
let currentGen,
let outerFinished
):
if generation != currentGen {
// Stale generation
continuation.resume(throwing: CancellationError())
return .none
}
if downstreamCont != nil {
return .resumeInnerContinuation(continuation)
}
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: continuation,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .none
case .finished:
continuation.resume(throwing: CancellationError())
return .none
default:
fatalError("Invalid state")
}
}
mutating func innerElementProduced(_ element: Element, generation: Int) -> Action {
switch state {
case .running(
let outerTask,
let outerCont,
let innerTask,
_,
let downstreamCont,
var buffer,
let currentGen,
let outerFinished
):
if generation != currentGen {
return .none
}
guard let downstreamCont = downstreamCont else {
buffer.append(.success(element))
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: nil,
downstreamContinuation: nil,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .none
}
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: nil, // Consumed
downstreamContinuation: nil,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .resumeDownstream(downstreamCont, .success(element))
case .finished:
return .none
default:
fatalError("Invalid state")
}
}
mutating func innerFinished(generation: Int) -> Action {
switch state {
case .running(let outerTask, let outerCont, _, _, let downstreamCont, let buffer, let currentGen, let outerFinished):
if generation != currentGen {
return .none
}
// Inner finished.
if outerFinished {
// Both finished
state = .finished
if let downstreamCont = downstreamCont {
return .resumeDownstream(downstreamCont, .success(nil))
}
} else {
// Just this inner finished. Wait for next outer.
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: nil,
innerContinuation: nil,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
// Resume outer sequence if downstream is waiting for more elements
if downstreamCont != nil, let outerCont = outerCont {
state = .running(
outerTask: outerTask,
outerContinuation: nil, // Consumed
innerTask: nil,
innerContinuation: nil,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .resumeOuterContinuation(outerCont)
}
}
return .none
default:
return .none
}
}
mutating func innerThrew(_ error: Error, generation: Int) -> Action {
switch state {
case .running(let outerTask, let outerCont, let innerTask, let innerCont, let downstreamCont, _, let currentGen, _):
if generation != currentGen {
return .none
}
state = .finished
let action: Action = .cancelTasks(outerTask, innerTask, outerCont, innerCont)
guard let downstreamCont = downstreamCont else {
return action
}
return .resumeDownstream(downstreamCont, .failure(error))
default:
return .none
}
}
mutating func outerFinished() -> Action {
switch state {
case .running(let outerTask, _, let innerTask, let innerCont, let downstreamCont, let buffer, let generation, _):
if innerTask == nil && buffer.isEmpty {
state = .finished
if let downstreamCont = downstreamCont {
return .resumeDownstream(downstreamCont, .success(nil))
}
} else {
state = .running(
outerTask: outerTask,
outerContinuation: nil,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: true
)
}
return .none
default:
return .none
}
}
mutating func outerThrew(_ error: Error) -> Action {
switch state {
case .running(let outerTask, let outerCont, let innerTask, let innerCont, let downstreamCont, _, _, _):
state = .finished
let action: Action = .cancelTasks(outerTask, innerTask, outerCont, innerCont)
guard let downstreamCont = downstreamCont else {
return action
}
return .resumeDownstream(downstreamCont, .failure(error))
default:
return .none
}
}
mutating func cancelled() -> Action {
switch state {
case .running(let outerTask, let outerCont, let innerTask, let innerCont, let downstreamCont, _, _, _):
state = .finished
let action: Action = .cancelTasks(outerTask, innerTask, outerCont, innerCont)
if let downstreamCont = downstreamCont {
return .resumeDownstream(downstreamCont, .success(nil))
}
return action
default:
state = .finished
return .none
}
}
}
@@ -0,0 +1,199 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.3, *)
final class FlatMapLatestStorage<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>: @unchecked Sendable
where Base.Element: Sendable, Inner.Element: Sendable {
typealias Element = Inner.Element
private let lock = Lock.allocate()
private var stateMachine: FlatMapLatestStateMachine<Base, Inner>
init(base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.stateMachine = FlatMapLatestStateMachine(base: base, transform: transform)
}
deinit {
lock.deinitialize()
}
func next(isolation: isolated (any Actor)? = #isolation) async throws(Inner.Failure) -> Inner.Element? {
do {
return try await withTaskCancellationHandler {
lock.lock()
let action = stateMachine.next()
switch action {
case .returnElement(let element):
lock.unlock()
return element
case .returnNil:
lock.unlock()
return nil
case .throwError(let error):
lock.unlock()
throw error
case .startOuterTask(let base):
// We need to start the outer task and then suspend
startOuterTask(base)
return try await suspend()
case .suspend:
lock.unlock()
return try await suspend()
}
} onCancel: {
let action = lock.withLock { stateMachine.cancelled() }
handleAction(action)
}
} catch {
throw error as! Inner.Failure
}
}
private func suspend() async throws -> Element? {
return try await withUnsafeThrowingContinuation { continuation in
let action = lock.withLock { stateMachine.next(for: continuation) }
switch action {
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .resumeInnerContinuation(let continuation):
continuation.resume()
case .none:
break
}
}
}
private func startOuterTask(_ base: Base) {
let task = Task {
var iterator = base.makeAsyncIterator()
loop: while true {
// Create a continuation to wait for demand
do {
try await withUnsafeThrowingContinuation { continuation in
let action = lock.withLock { stateMachine.outerTaskSuspended(continuation) }
switch action {
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .resumeInnerContinuation(let continuation):
continuation.resume()
case .none:
break
}
}
} catch {
// Cancellation or other error during suspension
let action = lock.withLock { stateMachine.outerThrew(error) }
handleAction(action)
break loop
}
do {
guard let element = try await iterator.next() else {
let action = lock.withLock { stateMachine.outerFinished() }
handleAction(action)
break loop
}
let action = lock.withLock { stateMachine.outerElementProduced(element) }
handleAction(action)
} catch {
let action = lock.withLock { stateMachine.outerThrew(error) }
handleAction(action)
break loop
}
}
}
stateMachine.outerTaskStarted(task)
lock.unlock()
}
private func startInnerTask(_ inner: Inner, generation: Int) {
let task = Task {
var iterator = inner.makeAsyncIterator()
loop: while true {
// Wait for demand
do {
try await withUnsafeThrowingContinuation { continuation in
let action = lock.withLock { stateMachine.innerTaskSuspended(continuation, generation: generation) }
switch action {
case .resumeInnerContinuation(let continuation):
continuation.resume()
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .none:
break
}
}
} catch {
// Cancellation
let action = lock.withLock { stateMachine.innerThrew(error, generation: generation) }
handleAction(action)
break loop
}
do {
guard let element = try await iterator.next() else {
let action = lock.withLock { stateMachine.innerFinished(generation: generation) }
handleAction(action)
break loop
}
let action = lock.withLock { stateMachine.innerElementProduced(element, generation: generation) }
handleAction(action)
} catch {
let action = lock.withLock { stateMachine.innerThrew(error, generation: generation) }
handleAction(action)
break loop
}
}
}
stateMachine.innerTaskStarted(task, generation: generation)
}
private func handleAction(_ action: FlatMapLatestStateMachine<Base, Inner>.Action) {
switch action {
case .startInnerTask(let inner, let generation, let previousTask, let previousCont):
if let previousTask = previousTask {
previousTask.cancel()
previousCont?.resume(throwing: CancellationError())
}
startInnerTask(inner, generation: generation)
case .cancelInnerTask(let task, let continuation):
task.cancel()
continuation?.resume(throwing: CancellationError())
case .resumeDownstream(let continuation, let result):
continuation.resume(with: result)
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .cancelTasks(let outer, let inner, let outerCont, let innerCont):
outer?.cancel()
inner?.cancel()
outerCont?.resume(throwing: CancellationError())
innerCont?.resume(throwing: CancellationError())
case .none:
break
}
}
}
@@ -0,0 +1,38 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2026 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.1)
// This is a helper type to move a non-Sendable value across isolation regions.
@usableFromInline
struct Disconnected<Value: ~Copyable>: ~Copyable, Sendable {
// This is safe since we take the value as sending and take consumes it
// and returns it as sending.
private nonisolated(unsafe) var value: Value?
@usableFromInline
init(value: consuming sending Value) {
self.value = .some(value)
}
@usableFromInline
consuming func take() -> sending Value {
nonisolated(unsafe) let value = self.value.take()!
return value
}
@usableFromInline
mutating func swap(newValue: consuming sending Value) -> sending Value {
nonisolated(unsafe) let value = self.value.take()!
self.value = consume newValue
return value
}
}
#endif
@@ -0,0 +1,329 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
//===----------------------------------------------------------------------===//
//
// This source file is part of the SwiftCertificates open source project
//
// Copyright (c) 2023 Apple Inc. and the SwiftCertificates project authors
// Licensed under Apache License v2.0
//
// See LICENSE.txt for license information
// See CONTRIBUTORS.txt for the list of SwiftCertificates project authors
//
// SPDX-License-Identifier: Apache-2.0
//
//===----------------------------------------------------------------------===//
/// ``_TinyArray`` is a ``RandomAccessCollection`` optimised to store zero or one ``Element``.
/// It supports arbitrary many elements but if only up to one ``Element`` is stored it does **not** allocate separate storage on the heap
/// and instead stores the ``Element`` inline.
@usableFromInline
struct _TinyArray<Element> {
@usableFromInline
enum Storage {
case one(Element)
case arbitrary([Element])
}
@usableFromInline
var storage: Storage
}
// MARK: - TinyArray "public" interface
extension _TinyArray: Equatable where Element: Equatable {}
extension _TinyArray: Hashable where Element: Hashable {}
extension _TinyArray: Sendable where Element: Sendable {}
extension _TinyArray: RandomAccessCollection {
@usableFromInline
typealias Element = Element
@usableFromInline
typealias Index = Int
@inlinable
subscript(position: Int) -> Element {
get {
self.storage[position]
}
set {
self.storage[position] = newValue
}
}
@inlinable
var startIndex: Int {
self.storage.startIndex
}
@inlinable
var endIndex: Int {
self.storage.endIndex
}
}
extension _TinyArray {
@inlinable
init(_ elements: some Sequence<Element>) {
self.storage = .init(elements)
}
@inlinable
init() {
self.storage = .init()
}
@inlinable
mutating func append(_ newElement: Element) {
self.storage.append(newElement)
}
@inlinable
mutating func append(contentsOf newElements: some Sequence<Element>) {
self.storage.append(contentsOf: newElements)
}
@discardableResult
@inlinable
mutating func remove(at index: Int) -> Element {
self.storage.remove(at: index)
}
@inlinable
mutating func removeAll(where shouldBeRemoved: (Element) throws -> Bool) rethrows {
try self.storage.removeAll(where: shouldBeRemoved)
}
@inlinable
mutating func sort(by areInIncreasingOrder: (Element, Element) throws -> Bool) rethrows {
try self.storage.sort(by: areInIncreasingOrder)
}
}
// MARK: - TinyArray.Storage "private" implementation
extension _TinyArray.Storage: Equatable where Element: Equatable {
@inlinable
static func == (lhs: Self, rhs: Self) -> Bool {
switch (lhs, rhs) {
case (.one(let lhs), .one(let rhs)):
return lhs == rhs
case (.arbitrary(let lhs), .arbitrary(let rhs)):
// we don't use lhs.elementsEqual(rhs) so we can hit the fast path from Array
// if both arrays share the same underlying storage: https://github.com/apple/swift/blob/b42019005988b2d13398025883e285a81d323efa/stdlib/public/core/Array.swift#L1775
return lhs == rhs
case (.one(let element), .arbitrary(let array)),
(.arbitrary(let array), .one(let element)):
guard array.count == 1 else {
return false
}
return element == array[0]
}
}
}
extension _TinyArray.Storage: Hashable where Element: Hashable {
@inlinable
func hash(into hasher: inout Hasher) {
// same strategy as Array: https://github.com/apple/swift/blob/b42019005988b2d13398025883e285a81d323efa/stdlib/public/core/Array.swift#L1801
hasher.combine(count)
for element in self {
hasher.combine(element)
}
}
}
extension _TinyArray.Storage: Sendable where Element: Sendable {}
extension _TinyArray.Storage: RandomAccessCollection {
@inlinable
subscript(position: Int) -> Element {
get {
switch self {
case .one(let element):
guard position == 0 else {
fatalError("index \(position) out of bounds")
}
return element
case .arbitrary(let elements):
return elements[position]
}
}
set {
switch self {
case .one:
guard position == 0 else {
fatalError("index \(position) out of bounds")
}
self = .one(newValue)
case .arbitrary(var elements):
elements[position] = newValue
self = .arbitrary(elements)
}
}
}
@inlinable
var startIndex: Int {
0
}
@inlinable
var endIndex: Int {
switch self {
case .one: return 1
case .arbitrary(let elements): return elements.endIndex
}
}
}
extension _TinyArray.Storage {
@inlinable
init(_ elements: some Sequence<Element>) {
var iterator = elements.makeIterator()
guard let firstElement = iterator.next() else {
self = .arbitrary([])
return
}
guard let secondElement = iterator.next() else {
// newElements just contains a single element
// and we hit the fast path
self = .one(firstElement)
return
}
var elements: [Element] = []
elements.reserveCapacity(elements.underestimatedCount)
elements.append(firstElement)
elements.append(secondElement)
while let nextElement = iterator.next() {
elements.append(nextElement)
}
self = .arbitrary(elements)
}
@inlinable
init() {
self = .arbitrary([])
}
@inlinable
mutating func append(_ newElement: Element) {
self.append(contentsOf: CollectionOfOne(newElement))
}
@inlinable
mutating func append(contentsOf newElements: some Sequence<Element>) {
switch self {
case .one(let firstElement):
var iterator = newElements.makeIterator()
guard let secondElement = iterator.next() else {
// newElements is empty, nothing to do
return
}
var elements: [Element] = []
elements.reserveCapacity(1 + newElements.underestimatedCount)
elements.append(firstElement)
elements.append(secondElement)
elements.appendRemainingElements(from: &iterator)
self = .arbitrary(elements)
case .arbitrary(var elements):
if elements.isEmpty {
// if `self` is currently empty and `newElements` just contains a single
// element, we skip allocating an array and set `self` to `.one(firstElement)`
var iterator = newElements.makeIterator()
guard let firstElement = iterator.next() else {
// newElements is empty, nothing to do
return
}
guard let secondElement = iterator.next() else {
// newElements just contains a single element
// and we hit the fast path
self = .one(firstElement)
return
}
elements.reserveCapacity(elements.count + newElements.underestimatedCount)
elements.append(firstElement)
elements.append(secondElement)
elements.appendRemainingElements(from: &iterator)
self = .arbitrary(elements)
} else {
elements.append(contentsOf: newElements)
self = .arbitrary(elements)
}
}
}
@discardableResult
@inlinable
mutating func remove(at index: Int) -> Element {
switch self {
case .one(let oldElement):
guard index == 0 else {
fatalError("index \(index) out of bounds")
}
self = .arbitrary([])
return oldElement
case .arbitrary(var elements):
defer {
self = .arbitrary(elements)
}
return elements.remove(at: index)
}
}
@inlinable
mutating func removeAll(where shouldBeRemoved: (Element) throws -> Bool) rethrows {
switch self {
case .one(let oldElement):
if try shouldBeRemoved(oldElement) {
self = .arbitrary([])
}
case .arbitrary(var elements):
defer {
self = .arbitrary(elements)
}
return try elements.removeAll(where: shouldBeRemoved)
}
}
@inlinable
mutating func sort(by areInIncreasingOrder: (Element, Element) throws -> Bool) rethrows {
switch self {
case .one:
// a collection of just one element is always sorted, nothing to do
break
case .arbitrary(var elements):
defer {
self = .arbitrary(elements)
}
try elements.sort(by: areInIncreasingOrder)
}
}
}
extension Array {
@inlinable
mutating func appendRemainingElements(from iterator: inout some IteratorProtocol<Element>) {
while let nextElement = iterator.next() {
append(nextElement)
}
}
}
Loaded 100 of 208 files, more files were not shown because too many files have changed in this diff. Show more