Compare commits

...
44 Commits
Author SHA1 Message Date
Franz Busch d0b4a06d0f Rework AsyncStreaming protocols to use generic RangeReplaceableContainer buffers (#417)
Replace `InputSpan`/`OutputSpan` with generic `Buffer` associated types constrained to `RangeReplaceableContainer` across all four streaming protocols. This sidesteps the limitation that `OutputSpan` (and other `~Escapable` types) cannot be used in async contexts today, while still allowing conforming types to choose a buffer representation optimized for their use case (e.g. `UniqueArray` for heap-backed storage, or a future stack-allocated container for embedded).

Switch callee-owned protocols from `consuming` to `inout` closure semantics, rename `forEachChunk` to `forEachBuffer`, and move `forEach`/`collect` to future directions in the proposal.

Adds test coverage for `AsyncWriter`, `CallerAsyncReader`, and `EitherError`. Updates all documentation to reflect the new API surface.
2026-05-14 23:42:25 +02:00
Franz Busch 6a92272e13 Fix a few new region isolation errors in nightly-main (#419)
* Rework `AsyncStreaming` protocols to use generic `RangeReplaceableContainer` buffers

Replace `InputSpan`/`OutputSpan` with generic `Buffer` associated types constrained to `RangeReplaceableContainer` across all four streaming protocols. This sidesteps the limitation that `OutputSpan` (and other `~Escapable` types) cannot be used in async contexts today, while still allowing conforming types to choose a buffer representation optimized for their use case (e.g. `UniqueArray` for heap-backed storage, or a future stack-allocated container for embedded).

Switch callee-owned protocols from `consuming` to `inout` closure semantics, rename `forEachChunk` to `forEachBuffer`, and move `forEach`/`collect` to future directions in the proposal.

Adds test coverage for `AsyncWriter`, `CallerAsyncReader`, and `EitherError`. Updates all documentation to reflect the new API surface.

* Fix a few new region isolation errors in `nightly-main`

Region isolation got smarter and started diagnosing more potential data races. This PR addresses the new errors. Importantly I had to introduce one `@unchecked Sendable` conformance since we can't yet use `UniqueDeque`
2026-05-14 23:22:27 +02:00
Franz Busch 6783e0f2d2 Adds a new AsyncReader.forEach method (#416)
It is often needed to iterate over all elements of a reader. This PR provides two convenience `forEach` methods to make iteration easier.
2026-05-13 13:20:41 +02:00
Franz Busch d14a83b31f Adds a new AsyncReader.collect method (#415)
Often developers want to collect elements from a reader up to a certain limit. This PR provides a few convenience methods for this.
2026-05-12 23:07:29 +02:00
Franz Busch 0a5f92d159 Expose the AsyncStreaming target as a product (#414) 2026-05-12 22:20:26 +02:00
Franz Busch be2467e0c0 Add new experimental _AsyncStreaming module (#400)
This PR adds an experimental module for asynchronous streaming interfaces as a potential evolution to async sequence. These APIs are considered highly unstable and are mostly for experimentation purposes; hence, they are part of an underscored module and all APIs are hidden behind a trait. I expect significant changes to those APIs to offer more functionality such as gather scatter and more.
2026-05-11 14:58:21 +02:00
Philippe Hausler 9d349bcc32 Cleanup pass for flatMapLatest and housekeeping tasks around proposals (#403)
* rename the map error proposal to have a specific number

* rename the flatMapLatest proposal to have a specific numbering

* Update availability markers for flatmapLatest

* Cleanup formatting pass for FlatMapLatest

* Remove stray extra single quote

* Add a fallback for older versions of swift for the next method on flatMapLatest
2026-03-04 14:27:48 -08:00
5043186441 feat: Add AsyncSequence.flatMapLatest operator (#382)
* feat: Add initial naive implementation of flatMapLatest

This introduces a simplified version of the flatMapLatest operator for AsyncSequence, along with a basic unit test. Note that this implementation uses unstructured concurrency and may have race conditions regarding task cancellation.

* refactor: Implement robust flatMapLatest with generation tracking

- Replaces naive implementation with a thread-safe approach using ManagedCriticalState.
- Introduces generation tracking to prevent race conditions where cancelled inner sequences could yield stale values.
- Adds test_interleaving_race_condition to verify correctness under concurrent load.
- Ensures Swift 6 Sendable compliance.

* refactor: Implement high-performance State Machine for flatMapLatest

- Replaces AsyncThrowingStream implementation with a custom AsyncSequence, Storage, and StateMachine.
- Implements explicit state management using Lock for thread safety.
- Handles concurrency between outer and inner sequences robustly.
- Ensures correct cancellation propagation and error handling.
- Verified with test_interleaving_race_condition.

* refactor: Split flatMapLatest into separate files

- Moves AsyncFlatMapLatestSequence.swift to Sources/AsyncAlgorithms/FlatMapLatest/
- Extracts FlatMapLatestStateMachine and FlatMapLatestStorage into their own files.
- Aligns project structure with other complex operators like CombineLatest and Debounce.

* test: Add comprehensive tests for flatMapLatest

- Added test_outer_throwing to verify outer sequence error propagation
- Added test_inner_throwing to verify inner sequence error propagation
- Added test_cancellation to verify proper cancellation handling
- Added test_empty_outer to verify empty outer sequence handling
- Added test_empty_inner to verify empty inner sequence handling
- Fixed test_simple_sequence to be more robust against timing issues

* fix: Make test_outer_throwing stable

- Changed from synchronous map with throwIf to AsyncThrowingStream
- Added delay to ensure proper error propagation timing
- Fixes intermittent test failures

* refactor: Clean up verbose comments in FlatMapLatestStateMachine

- Removed LLM-style thinking comments
- Kept only essential, professional explanations
- No functional changes, all tests pass

* docs: Add comprehensive guide for flatMapLatest

- Added FlatMapLatest.md documentation guide
- Authored by Peter Friese
- Includes introduction, code samples, and use cases
- Covers search-as-you-type, location-based data, and dynamic config examples
- Compares with similar operators in ReactiveX and Combine

* docs: Add evolution proposal for flatMapLatest

- Created SAA-00nn proposal for flatMapLatest operator
- Includes motivation, detailed design, and examples
- Covers implementation strategy with state machine
- Compares with ReactiveX switchMap and Combine switchToLatest

* mapError - transforming failures (#366)

* Implement AsyncMapErrorSequence

* Update the proposal, implementation and tests with formatting, additional details and modifications for being able to push it to a review

* Foratting and preambles

* Update Copyright date for error sequence

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update copyright year for mapError tests

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update availablity and modify the mapError algorithm to be opaque

* Correct pre 6.0 build avail parameters

* Make the formatting of the algorithm accesor a bit more consistent

* Update the 5.8 package builder to define 1.2

* re-run formatter

---------

Co-authored-by: Clive <clive819@gmail.com>
Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update the flatMapLatest proposal and implementation to a pitchable state

* Fix race condition in flatMapLatest and add regression test

* chore: Remove test_output.log and add it to .gitignore

---------

Co-authored-by: Philippe Hausler <phausler@apple.com>
Co-authored-by: Clive <clive819@gmail.com>
Co-authored-by: Clive <clive819@users.noreply.github.com>
2026-03-04 13:58:48 -08:00
Franz Busch 2971dd5d9f Fix a data race error with the internal Optional.takeSending (#399)
We added a little helper to move a non-Sendable type into an optional and take it out as sending so that it is disconnected. This was actually a data race hole which got fixed by the compiler so we need to use a proper disconnected abstractions.
2026-02-11 11:02:07 -08:00
Oleksii M. 103f5e5bea Fix swift tool version (#374) 2026-01-26 09:34:26 -08:00
Kenta Kubo b8006a9de0 Add support for FreeBSD (#375)
* Add partial support for FreeBSD

This change will allow `swift build` to succeed. However, SwiftPM support is still incomplete and does not yet allow adding `.freebsd` to `.when(platforms:)`, so `swift test` cannot be executed.

* Make `swift test` work on FreeBSD
2026-01-26 09:33:54 -08:00
Alessio NossaandPhilippe Hausler 2de50036f6 Fix unbuffered share deadlock (#387)
* Introduce test_share_with_no_buffering

* Update test_share_with_no_buffering implementation

* Check limit bigger than 0 in AsyncShareSequence iterate

* Revert "Check limit bigger than 0 in AsyncShareSequence iterate"

This reverts commit 3c9e1ab0ff.

* Check limit bigger than 0 in iterate in AsyncShareSequence

* Hand correct the lint failures in TestShare.swift

* An additional hand-edit lint fix for TestShare.swift

---------

Co-authored-by: Philippe Hausler <phausler@apple.com>
2026-01-26 09:29:41 -08:00
scottmarchant ba540240d3 docs: Update comments in Locking.swift to better clarify threading scenario for WASILibc. (#393)
### Motivation:

The previous comments could mislead readers into thinking WASILibc doesn't have pthreads or locks, which isn't correct. The updated comment adds needed clarification to help readers know there multiple variations and configurations of Swift for WebAssembly, some of which provide pthreads and locks.

### Modifications:

- Updated code comments in Locking.swift

### Result:

- Better explanation of WASILibc capabilities.
2026-01-13 22:12:46 +00:00
65ed06871b mapError - transforming failures (#366)
* Implement AsyncMapErrorSequence

* Update the proposal, implementation and tests with formatting, additional details and modifications for being able to push it to a review

* Foratting and preambles

* Update Copyright date for error sequence

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update copyright year for mapError tests

Co-authored-by: Clive <clive819@users.noreply.github.com>

* Update availablity and modify the mapError algorithm to be opaque

* Correct pre 6.0 build avail parameters

* Make the formatting of the algorithm accesor a bit more consistent

* Update the 5.8 package builder to define 1.2

* re-run formatter

---------

Co-authored-by: Clive <clive819@gmail.com>
Co-authored-by: Clive <clive819@users.noreply.github.com>
2026-01-12 12:21:06 -08:00
scottmarchant d98a48b8c2 feat: Add support for compiling to wasm when pthreads aren't available (#389) 2026-01-12 10:37:45 -08:00
Franz Busch 6c050d5ef8 Make MPSCAsyncChannel source methods nonisolated(nonsending) (#380)
`nonisolated(nonsending)` is a 6.2 language feature that allows the inheritance of the callers isolation. The new `MPSCAsyncChannel.Source` send methods should adopt this to avoid unnecessary isolation hops.
2025-12-01 11:18:00 -08:00
Joseph Heck 2773d41253 This removes the build constraint at Swift Package Index from Swift 5.8 so that (#377)
symbols that are only available in later versions (for example, MultiProducerSingleConsumerAsyncChannel)
are reflected in the hosted documentation.
2025-11-19 22:22:29 +01:00
Franz BuschandJoe Heck 4686d2edd3 Add MultiProducerSingleConsumerChannel (#305)
* Add `AsyncBackpressuredStream` proposal and implementation

# Motivation

The pitch to add external backpressure support to the standard libraries `AsyncStream` got returned for revision since there are larger open questions around `AsyncSequence`. However, having external backpressure in a source asynchronous sequence is becoming more and more important.

# Modification

This PR adds a modified proposal and implementation that brings the Swift Evolution proposal over to Swift Async Algorithms.

* Update proposal and implementation

* Update proposal

* Add example project

* Formatting

* Fix Swift 6.0 build

* Future direction for ~Copyable elements

* Apply formatting

* Fix CI

* Move to 6.1 and update proposal

* Guard tests

* Minor edits to the proposal

* Fix revision order

* FIxup setOnTerminationCallback

* Address review feedback

* Rename to `MultiProducerSingleConsumerAsyncChannel`

* Allow one termination callback per source.

* Fix all sendable warnings

* Remove unbounded strategy, rename copy -> makeAdditionalSource, rename asyncSequence() -> elements(), and move enqueue and cancel to the `CallbackHandle` previously called `CallbackToken`

* Remove Example and fix docs and fix format

* fixes grammar and some word order issues

* Remove unnecessary consume

---------

Co-authored-by: Joe Heck <j_heck@apple.com>
2025-11-18 15:56:41 -08:00
Melissa Kilby 24bd5acc81 chore: restrict GitHub workflow permissions - future-proof (#373)
Signed-off-by: Melissa Kilby <mkilby@apple.com>
2025-10-27 15:09:46 -07:00
Changmin Lee 26111a6fb7 Fix broken documentation links in Interspersed guide (#371) 2025-10-06 09:28:20 -07:00
Janosch Hildebrand daf3c0d723 Fix typo in doc comment for share (#370) 2025-10-01 08:43:45 -07:00
Philippe HauslerandJamie 70c36ce9a1 Share algorithm (#357)
This is an implementation of share; an algorithm for allowing AsyncSequence types to be iterated by multiple consumers. In addition this cleans up the project to be a starting point for the 1.1 release - it fixes the continuous integration support and configurations but has the cost of only supporting 3 active releases (6.0, 6.1, and 6.2). Previous releases are only going to have legacy support via older released versions.

* Fix the remaining todo on hard cancellation vs soft cancellation

* Update Sources/AsyncAlgorithms/AsyncShareSequence.swift

Co-authored-by: Jamie <2119834+jamieQ@users.noreply.github.com>

* Update the implementation of share to handle sendability requirements, address some edge cases and update to the latest version of the discussion around buffering behaviors as well as adding some documentation and commentary

* Add some first drafts at unit tests for verifying share behaviors

* Cleanup some of the documentation and address feedback for implementation details

* Fix merge damage by restoring the changes from @jamieQ

* Fix up some availability issues w.r.t. sendable metatypes

* Add a preamble to SendableMetatypes.swift

* Add a fallback for next (for pre 6.2 swift's)

* Slight adjustments for older compiler builds and expose the sendable metatypes since those are part of documentation

* Update formatting and gate availability for older compilers

* Speculative fix for pre 6.1 compiler crashes by removing the sending keyword

* Revert "Speculative fix for pre 6.1 compiler crashes by removing the sending keyword"

This reverts commit 875fa81bc6.

* Speculative fix for pre 6.1 compiler crashes by removing the sending keyword, and correct sendable metatype shim availability

* Remove block comment for formatting check

* Take a more conservative approach and roll back to only supporting share on 6.2

* Only test share on 6.2 or newer

* roll back to older sendable metatype constraints

* yet another stab at 5.10 visionOS guards

* Formatting pass

* Disable 5.10 builds and leave those to older releases

* Ignore swift-version files to avoid local swiftly toolchain selection from being required by others

* Renumber the proposals and copy share into a guide

* Add a link from the primary doc interface to share

---------

Co-authored-by: Jamie <2119834+jamieQ@users.noreply.github.com>
2025-09-10 10:00:31 -07:00
Philippe Hausler ca38935fc0 Clear up CI issues and testing failures (#365)
* Conditionalize the testing targets for non-windows targets to avoid the TaskDriver requirements
* Some driveby formatting fixes and remove the freebsd target since that is not supported in swiftpm yet
* Rework validation import conditions to dependency conditionals
* Exclude 5.9 workflows and enable macOS testing scheme for CI
* Additional ifdefing out non windows targets for sequence diagram testing and more formatting fixes
2025-09-08 16:23:41 -07:00
Philippe Hausler 18e2398481 Update the project to address concurrency failures for 6.2 build modes. (#362)
* Correct build failures while updating to 6.2 swift build modes

* Add a package definition for 5.8 builds

* Add some brief explanation for the alteration of the Element in the buffer to an UnsafeTransfer

* Fix broken merge for wasi threads
2025-09-08 09:38:39 -07:00
Max Desiatov c537393339 Add enable_wasm_sdk_build: true to pull_request.yml (#363)
`swift-async-algorithms` should be built for Wasm as one of the officially supported platforms on CI to prevent possible future regressions.
2025-09-02 16:10:59 +01:00
Philippe Hausler 6dcb3acddd Simplify the version definitions and add a new definition for 1.1 (#359) 2025-08-11 14:51:05 -07:00
Franz Busch 3997ce3255 Remove the DocC plugin as a dependency (#354)
When DocC was introduced the DocC plugin was useful for building documentation locally. Nowadays both Xcode and VSCode have built-in support to generate this without the need for the plugin.

This PR removes the direct dependency on the plugin.
2025-07-02 08:09:32 -07:00
George Barnett 042e1c4d9d Move platform requirements to availability annotations (#348)
* Move platform requirements to availability annotations

Adding or raising the deployment platforms in the package manifest is a
SemVer major breaking change as consumers must also add or raise their
deployment platforms. This is a known limitation of SwiftPM.

Unforunately this means that it's very difficult for non-leaf packages
to adopt packages which declare their platforms in the manifest. Doing
so puts the brakes on adoption and ecosystem growth. For 'core' packages
like this one availability constraints should be expressed on
declarations rather than in the manifest.

This patch adds equivalent availability annotations to declarations
across the package and removes platforms from the package manifest.

* Address naming feedback
2025-04-22 08:07:37 +01:00
Mishal Shah 57b0814edf [CI] Add support for GitHub Actions (#344) 2025-03-29 09:49:13 +01:00
Franz Busch 9fcf11df15 Fix doc paste mistake 2025-03-28 21:59:41 +01:00
Franz Busch 08bb1bb014 Fix yaml check and remove docker compose 2025-03-28 21:59:41 +01:00
Franz Busch 5878737018 Language check 2025-03-28 21:59:41 +01:00
Franz Busch 552856a85c License headers 2025-03-28 21:59:41 +01:00
Franz Busch ba30f2051b Documentation 2025-03-28 21:59:41 +01:00
Franz Busch d5b49aab17 Apply formatting 2025-03-28 21:59:41 +01:00
Franz Busch ba33a225e9 Format rules 2025-03-28 21:59:41 +01:00
orobio 4c3ea81f81 Fix memory leak in Lock (#331) 2024-10-09 10:47:15 +02:00
Mason Kim 5c8bd186f4 Fix a few typos (#329)
* [DOCS] fix typo: precicely → precisely

* [DOCS] fix typo: ever → every

* [DOCS] fix typo: AsyncBufferSequence, relative
2024-09-02 14:15:02 +02:00
finagolfin 2503842d68 Use Bionic module from new Android overlay in Swift 6 instead (#326)
The new module and overlay were merged into Swift 6 in swiftlang/swift#74758.
2024-08-07 11:18:51 +01:00
Adam Fowler e83857ca55 Add Musl import, error if unrecognised platform (#325)
* Add Musl import, error if unrecognised platform

* Add #else #error for all platform checks
2024-08-06 12:26:55 +02:00
Yuta Saito 6ae9a051f7 Add support for SWIFTCI_USE_LOCAL_DEPS convention (#311)
To use this package in utils/build-script pipeline
2024-04-10 14:41:45 -07:00
Franz Busch 46b4464735 Fix some strict concurrency warnings (#310)
# Motivation

There were a few new strict concurrency warnings that this PR fixes.
2024-04-04 20:46:16 +01:00
Eric Horacek d162617838 Depend on specific products from swift-collections (#307)
* Only depend on OrderedCollections from swift-collections

Before this change, once consumers update to SwiftCollections 1.1.0 they'll pull in all of the new collection dependencies (HashTree, BitCollections, etc.). In our case, this increased our binary size by ~1MB. To fix this, we switch to only depending on what we need.

* Add DequeModule dependency
2024-02-28 12:47:12 +00:00
leogdion 14729ea7f9 Update README.md (#306)
Missing Version Number
2024-01-24 11:28:26 +00:00
172 changed files with 16224 additions and 4662 deletions

No files matched your search

+8
View File
@@ -0,0 +1,8 @@
root = true
[*]
indent_style = space
indent_size = 2
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
+24
View File
@@ -0,0 +1,24 @@
name: Pull request
permissions:
contents: read
on:
pull_request:
types: [opened, reopened, synchronize]
jobs:
tests:
name: Test
uses: swiftlang/github-workflows/.github/workflows/swift_package_test.yml@main
with:
linux_exclude_swift_versions: "[{\"swift_version\": \"5.9\"}, {\"swift_version\": \"5.10\"}]]"
windows_exclude_swift_versions: "[{\"swift_version\": \"5.9\"}]"
enable_wasm_sdk_build: true
soundness:
name: Soundness
uses: swiftlang/github-workflows/.github/workflows/soundness.yml@main
with:
license_header_check_project_name: "Swift Async Algorithms"
format_check_container_image: "swiftlang/swift:nightly-main-noble" # Needed due to https://github.com/swiftlang/swift-format/issues/1081
+3 -1
View File
@@ -8,4 +8,6 @@ DerivedData/
.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata
.netrc
.swiftpm
Package.resolved
Package.resolved
.swift-version
test_output.log
+10
View File
@@ -0,0 +1,10 @@
@@===----------------------------------------------------------------------===@@
@@
@@ This source file is part of the Swift Async Algorithms open source project
@@
@@ Copyright (c) YEARS Apple Inc. and the Swift project authors
@@ Licensed under Apache License v2.0 with Runtime Library Exception
@@
@@ See https://swift.org/LICENSE.txt for license information
@@
@@===----------------------------------------------------------------------===@@
+40
View File
@@ -0,0 +1,40 @@
.gitignore
**/.gitignore
.licenseignore
.gitattributes
.mailfilter
.mailmap
.spi.yml
.swift-format
.editorconfig
.github/*
*.md
*.txt
*.yml
*.yaml
*.json
Package.swift
**/Package.swift
Package@-*.swift
Package@swift-*.swift
**/Package@-*.swift
Package.resolved
**/Package.resolved
Makefile
*.modulemap
**/*.modulemap
**/*.docc/*
*.xcprivacy
**/*.xcprivacy
*.symlink
**/*.symlink
Dockerfile
**/Dockerfile
Snippets/*
dev/git.commit.template
*.crt
**/*.crt
*.pem
**/*.pem
*.der
**/*.der
-1
View File
@@ -2,4 +2,3 @@ version: 1
builder:
configs:
- documentation_targets: [AsyncAlgorithms]
swift_version: 5.8
+58
View File
@@ -0,0 +1,58 @@
{
"fileScopedDeclarationPrivacy" : {
"accessLevel" : "private"
},
"indentation" : {
"spaces" : 2
},
"indentConditionalCompilationBlocks" : false,
"indentSwitchCaseLabels" : false,
"lineBreakAroundMultilineExpressionChainComponents" : false,
"lineBreakBeforeControlFlowKeywords" : false,
"lineBreakBeforeEachArgument" : true,
"lineBreakBeforeEachGenericRequirement" : true,
"lineLength" : 120,
"maximumBlankLines" : 1,
"prioritizeKeepingFunctionOutputTogether" : true,
"respectsExistingLineBreaks" : true,
"rules" : {
"AllPublicDeclarationsHaveDocumentation" : false,
"AlwaysUseLowerCamelCase" : false,
"AmbiguousTrailingClosureOverload" : true,
"BeginDocumentationCommentWithOneLineSummary" : false,
"DoNotUseSemicolons" : true,
"DontRepeatTypeInStaticProperties" : true,
"FileScopedDeclarationPrivacy" : true,
"FullyIndirectEnum" : true,
"GroupNumericLiterals" : true,
"IdentifiersMustBeASCII" : true,
"NeverForceUnwrap" : false,
"NeverUseForceTry" : false,
"NeverUseImplicitlyUnwrappedOptionals" : false,
"NoAccessLevelOnExtensionDeclaration" : true,
"NoAssignmentInExpressions" : true,
"NoBlockComments" : true,
"NoCasesWithOnlyFallthrough" : true,
"NoEmptyTrailingClosureParentheses" : true,
"NoLabelsInCasePatterns" : false,
"NoLeadingUnderscores" : false,
"NoParensAroundConditions" : true,
"NoVoidReturnOnFunctionSignature" : true,
"OneCasePerLine" : true,
"OneVariableDeclarationPerLine" : true,
"OnlyOneTrailingClosureArgument" : true,
"OrderedImports" : false,
"ReturnVoidInsteadOfEmptyTuple" : true,
"UseEarlyExits" : true,
"UseLetInEveryBoundCaseVariable" : false,
"UseShorthandTypeNames" : true,
"UseSingleLinePropertyGetter" : false,
"UseSynthesizedInitializer" : false,
"UseTripleSlashForDocumentationComments" : true,
"UseWhereClausesInForLoops" : false,
"ValidateDocumentationComments" : false
},
"spacesAroundRangeFormationOperators" : false,
"tabWidth" : 4,
"version" : 1
}
-25
View File
@@ -1,25 +0,0 @@
# file options
--swiftversion 5.7
--exclude .build
# format options
--self insert
--patternlet inline
--ranges nospace
--stripunusedargs unnamed-only
--ifdef no-indent
--extensionacl on-declarations
--disable typeSugar # https://github.com/nicklockwood/SwiftFormat/issues/636
--disable andOperator
--disable wrapMultilineStatementBraces
--disable enumNamespaces
--disable redundantExtensionACL
--disable redundantReturn
--disable preferKeyPath
--disable sortedSwitchCases
--disable hoistAwait
--disable hoistTry
# rules
+7
View File
@@ -0,0 +1,7 @@
extends: default
rules:
line-length: false
document-start: false
truthy:
check-keys: false # Otherwise we get a false positive on GitHub action's `on` key
@@ -51,7 +51,7 @@ would become part of a public API? If so, what kinds of changes can be
made without breaking ABI? Can this feature be added/removed without
breaking ABI? For more information about the resilience model, see the
[library evolution
document](https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst)
document](https://github.com/apple/swift/blob/main/docs/LibraryEvolution.rst)
in the Swift repository.
## Alternatives considered
+1 -1
View File
@@ -13,7 +13,7 @@
## Introduction
Similar to the `zip` algorithm there is a need to combine the latest values from multiple input asynchronous sequences. Since `AsyncSequence` augments the concept of sequence with the characteristic of time it means that the composition of elements may not just be pairwise emissions but instead be temporal composition. This means that it is useful to emit a new tuple _when_ a value is produced. The `combineLatest` algorithm provides precicely that.
Similar to the `zip` algorithm there is a need to combine the latest values from multiple input asynchronous sequences. Since `AsyncSequence` augments the concept of sequence with the characteristic of time it means that the composition of elements may not just be pairwise emissions but instead be temporal composition. This means that it is useful to emit a new tuple _when_ a value is produced. The `combineLatest` algorithm provides precisely that.
## Detailed Design
+2 -2
View File
@@ -28,7 +28,7 @@ By applying the buffer operator to the previous example, the file can be read as
## Proposed Solution
We propose to extend `AsyncSequence` with a `buffer()` operator. This operator will return an `AsyncBuffereSequence` that wraps the source `AsyncSequence` and handle the buffering mechanism.
We propose to extend `AsyncSequence` with a `buffer()` operator. This operator will return an `AsyncBufferSequence` that wraps the source `AsyncSequence` and handle the buffering mechanism.
This operator will accept an `AsyncBufferSequencePolicy`. The policy will dictate the behaviour in case of a buffer overflow.
@@ -43,7 +43,7 @@ public struct AsyncBufferSequencePolicy: Sendable {
}
```
And the public API of `AsyncBuffereSequence` will be:
And the public API of `AsyncBufferSequence` will be:
```swift
extension AsyncSequence where Self: Sendable {
+3 -3
View File
@@ -178,7 +178,7 @@ public struct AsyncInterspersedSequence<Base: AsyncSequence> {
@usableFromInline
internal init(_ base: Base, every: Int, separator: Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .element(separator)
self.every = every
@@ -186,7 +186,7 @@ public struct AsyncInterspersedSequence<Base: AsyncSequence> {
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () -> Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .syncClosure(separator)
self.every = every
@@ -194,7 +194,7 @@ public struct AsyncInterspersedSequence<Base: AsyncSequence> {
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () async -> Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .asyncClosure(separator)
self.every = every
File renamed without changes.
@@ -1,7 +1,7 @@
# Chunked & Timer
* Proposal: [SAA-NNNN](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/NNNN-chunk.md)
* Proposal: [SAA-0013](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0013-chunk.md)
* Author(s): [Kevin Perry](https://github.com/kperryua), [Philippe Hausler](https://github.com/phausler)
* Status: **Implemented**
* Status: **Accepted**
* Implementation: [
[By group](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedByGroupSequence.swift),
[On projection](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChunkedOnProjectionSequence.swift),
@@ -1,8 +1,8 @@
# Rate Limiting
* Proposal: [SAA-NNNN]()
* Proposal: [SAA-0014](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0014-rate-limits.md)
* Authors: [Philippe Hausler](https://github.com/phausler)
* Status: **Implemented**
* Status: **Accepted**
* Implementation:
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncDebounceSequence.swift) |
@@ -18,7 +18,7 @@
## Introduction
When events can potentially happen faster than the desired consumption rate, there are multiple ways to handle the situation. One approach is to only emit values after a given period of time of inactivity, or "quiescence", has elapsed. This algorithm is commonly referred to as debouncing. A very close reelativee is an apporach to emit values after a given period has elapsed. These emitted values can be reduced from the values encountered during the waiting period. This algorithm is commonly referred to as throttling.
When events can potentially happen faster than the desired consumption rate, there are multiple ways to handle the situation. One approach is to only emit values after a given period of time of inactivity, or "quiescence", has elapsed. This algorithm is commonly referred to as debouncing. A very close relative is an approach to emit values after a given period has elapsed. These emitted values can be reduced from the values encountered during the waiting period. This algorithm is commonly referred to as throttling.
## Proposed Solution
@@ -1,7 +1,7 @@
# Reductions
* Proposal: [SAA-NNNN](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/NNNN-reductions.md)
* Proposal: [SAA-0015](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0015-reductions.md)
* Author(s): [Philippe Hausler](https://github.com/phausler)
* Status: **Implemented**
* Status: **Accepted**
* Implementation: [
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncExclusiveReductionsSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestReductions.swift)
@@ -0,0 +1,838 @@
# MultiProducerSingleConsumerAsyncChannel
* Proposal: [SAA-0016](0016-multi-producer-single-consumer-channel.md)
* Authors: [Franz Busch](https://github.com/FranzBusch)
* Status: **Implemented**
## Revision
- 2023/12/18: Migrate proposal from Swift Evolution to Swift Async Algorithms.
- 2023/12/19: Add element size dependent strategy
- 2024/05/19: Rename to multi producer single consumer channel
- 2024/05/28: Add unbounded strategy
- 2025/03/24: Adopt `~Copyable` for correct semantics and better performance.
## Introduction
[SE-0314](https://github.com/apple/swift-evolution/blob/main/proposals/0314-async-stream.md)
introduced new `Async[Throwing]Stream` types which act as root asynchronous
sequences. These two types allow bridging from synchronous callbacks such as
delegates to an asynchronous sequence. This proposal adds a new root primitive
with the goal of modeling asynchronous multi-producer-single-consumer systems.
## Motivation
After using the `AsyncSequence` protocol, the `Async[Throwing]Stream` types, and
the `Async[Throwing]Channel` types extensively over the past years, we learned
that there is a gap in the ecosystem for a type that provides strict
multi-producer-single-consumer guarantees with external backpressure support.
Additionally, any stream/channel like type needs to have a clear definition
about the following behaviors:
1. Backpressure
2. Multi/single consumer support
3. Downstream consumer termination
4. Upstream producer termination
The below sections are providing a detailed explanation of each of those.
### Backpressure
In general, backpressure is the mechanism that prevents a fast producer from
overwhelming a slow consumer. It helps the stability of the overall system by
regulating the flow of data between different components. Additionally, it
allows us to put an upper bound on the resource consumption of a system. In reality,
backpressure is used in almost all networked applications.
In Swift, asynchronous sequences also have the concept of internal backpressure.
This is modeled by the pull-based implementation where a consumer has to call
`next` on the `AsyncIterator`. In this model, there is no way for a consumer to
overwhelm a producer since the producer controls the rate of pulling elements.
However, the internal backpressure of an asynchronous sequence isn't the only
backpressure in play. There is also the source backpressure that is producing
the actual elements. For a backpressured system, it is important that every
component of such a system is aware of the backpressure of its consumer and its
producer.
Let's take a quick look at how our current root asynchronous sequences are handling
this.
`Async[Throwing]Stream` aims to support backpressure by providing a configurable
buffer and returning `Async[Throwing]Stream.Continuation.YieldResult` which
contains the current buffer depth from the `yield()` method. However, only
providing the current buffer depth on `yield()` is not enough to bridge a
backpressured system into an asynchronous sequence since this can only be used
as a "stop" signal, but we are missing a signal to indicate resuming the
production. The only viable backpressure strategy that can be implemented with
the current API is a timed backoff where we stop producing for some period of
time and then speculatively produce again. This is a very inefficient pattern
that produces high latencies and inefficient use of resources.
`Async[Throwing]Channel` is a multi-producer-multi-consumer channel that only
supports asynchronous producers. Additionally, the backpressure strategy is
fixed by a buffer size of 1 element per producer.
We are currently lacking a type that supports a configurable backpressure
strategy and both asynchronous and synchronous producers.
### Multi/single consumer support
The `AsyncSequence` protocol itself makes no assumptions about whether the
implementation supports multiple consumers or not. This allows the creation of
unicast and multicast asynchronous sequences. The difference between a unicast
and multicast asynchronous sequence is if they allow multiple iterators to be
created. `AsyncStream` does support the creation of multiple iterators and it
does handle multiple consumers correctly. On the other hand,
`AsyncThrowingStream` also supports multiple iterators but does `fatalError`
when more than one iterator has to suspend. The original proposal states:
> As with any sequence, iterating over an AsyncStream multiple times, or
creating multiple iterators and iterating over them separately, may produce an
unexpected series of values.
While that statement leaves room for any behavior, we learned that a clear distinction
of behavior for root asynchronous sequences is beneficial; especially when it comes to
how transformation algorithms are applied on top.
### Downstream consumer termination
Downstream consumer termination allows the producer to notify the consumer that
no more values are going to be produced. `Async[Throwing]Stream` does support
this by calling the `finish()` or `finish(throwing:)` methods of the
`Async[Throwing]Stream.Continuation`. However, `Async[Throwing]Stream` does not
handle the case that the `Continuation` may be `deinit`ed before one of the
finish methods is called. This currently leads to async streams that never
terminate.
### Upstream producer termination
Upstream producer termination is the inverse of downstream consumer termination,
where the producer is notified once the consumption has terminated. Currently,
`Async[Throwing]Stream` does expose the `onTermination` property on the
`Continuation`. The `onTermination` closure is invoked once the consumer has
terminated. The consumer can terminate in four separate cases:
1. The asynchronous sequence was `deinit`ed and no iterator was created.
2. The iterator was `deinit`ed and the asynchronous sequence is unicast.
3. The consuming task is canceled.
4. The asynchronous sequence returned `nil` or threw.
`Async[Throwing]Stream` currently invokes `onTermination` in all cases; however,
since `Async[Throwing]Stream` supports multiple consumers (as discussed in the
`Multi/single consumer support` section), a single consumer task being canceled
leads to the termination of all consumers. This is not expected from multicast
asynchronous sequences in general.
## Proposed solution
The above motivation lays out the expected behaviors for any consumer/producer
system and compares them to the behaviors of `Async[Throwing]Stream` and
`Async[Throwing]Channel`.
This section proposes a new type called `MultiProducerSingleConsumerAsyncChannel`
that implements all of the above-mentioned behaviors. Importantly, this proposed
solution is taking advantage of `~Copyable` types to model the
multi-producer-single-consumer behavior. While the current `AsyncSequence`
protocols are not supporting `~Copyable` types, we provide a way to convert the
proposed channel to an asynchronous sequence. This leaves us room to support any
potential future asynchronous streaming protocol that supports `~Copyable`.
### Creating a MultiProducerSingleConsumerAsyncChannel
You can create an `MultiProducerSingleConsumerAsyncChannel` instance using the
`makeChannel(of:backpressureStrategy:)` method. This method returns you the
channel and the source. The source can be used to send new values to the
asynchronous channel. The new API specifically provides a
multi-producer/single-consumer pattern.
```swift
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
// The channel and source can be extracted from the returned type
let channel = consume channelAndSource.channel
let source = consume channelAndSource.source
```
The new proposed APIs offer two different backpressure strategies:
- Watermark: Using a low and high watermark.
- Unbounded: Unbounded buffering of the channel. **Only** use this if the
production is limited through some other means.
The source is used to send values to the channel. It provides different APIs for
synchronous and asynchronous producers. All of the APIs are relaying the
backpressure of the channel. The synchronous multi-step APIs are the foundation
for all other APIs. Below is an example of how it can be used:
```swift
do {
let sendResult = try source.send(contentsOf: sequence)
switch sendResult {
case .produceMore:
// Trigger more production in the underlying system
case .enqueueCallback(let callbackHandle):
// There are enough values in the channel already. We need to enqueue
// a callback to get notified when we should produce more.
callbackHandle.enqueueCallback(onProduceMore: { result in
switch result {
case .success:
// Trigger more production in the underlying system
case .failure(let error):
// Terminate the underlying producer
}
})
}
} catch {
// `send(contentsOf:)` throws if the channel already terminated
}
```
The above API offers the most control and highest performance when bridging a
synchronous producer to a `MultiProducerSingleConsumerAsyncChannel`. First, you have
to send values using the `send(contentsOf:)` which returns a `SendResult`. The
result either indicates that more values should be produced or that a callback
should be enqueued by calling the `enqueueCallback(onProduceMore:)` method.
This callback is invoked once the backpressure strategy
decides that more values should be produced. This API aims to offer the most
flexibility with the greatest performance. The callback only has to be allocated
in the case where the producer needs to pause production.
Additionally, the above API is the building block for some higher-level and
easier-to-use APIs to send values to the channel. Below is an
example of the two higher-level APIs.
```swift
// Writing new values and providing a callback when to produce more
try source.send(contentsOf: sequence, onProduceMore: { result in
switch result {
case .success:
// Trigger more production
case .failure(let error):
// Terminate the underlying producer
}
})
// This method suspends until more values should be produced
try await source.send(contentsOf: sequence)
```
With the above APIs, we should be able to effectively bridge any system into a
`MultiProducerSingleConsumerAsyncChannel` regardless of whether the system is callback-based,
blocking, or asynchronous.
### Multi producer
To support multiple producers, the source offers a `copy` method to produce a new
source. The source is returned `sending`, so it is in a disconnected isolation
region from the original source, allowing it to be passed into a different isolation
region to concurrently produce elements.
```swift
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 5)
)
var channel = consume channelAndSource.channel
var source1 = consume channelAndSource.source
var source2 = source1.makeAdditionalSource()
group.addTask {
try await source1.send(1)
}
group.addTask() {
try await source2.send(2)
}
print(await channel.next()) // Prints either 1 or 2 depending on which child task runs first
print(await channel.next()) // Prints either 1 or 2 depending on which child task runs first
```
### Downstream consumer termination
> When reading the next two examples of termination behavior, keep in mind
that the newly proposed APIs are providing a strict single consumer channel.
Calling `finish()` terminates the downstream consumer. Below is an example of
this:
```swift
// Termination through calling finish
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
try await source.send(1)
source.finish()
print(await channel.next()) // Prints Optional(1)
print(await channel.next()) // Prints nil
```
If the channel has a failure type it can also be finished with an error.
```swift
// Termination through calling finish
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
throwing: SomeError.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
try await source.send(1)
source.finish(throwing: SomeError)
print(try await channel.next()) // Prints Optional(1)
print(try await channel.next()) // Throws SomeError
```
The other way to terminate the consumer is by deiniting the source. This has the
same effect as calling `finish()`. Since the source is a `~Copyable` type, this
will happen automatically when the source is last used or explicitly consumed.
```swift
// Termination through deiniting the source
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
try await source.send(1)
_ = consume source // Explicitly consume the source
print(await channel.next()) // Prints Optional(1)
print(await channel.next()) // Prints nil
```
### Upstream producer termination
The producer will get notified about termination through the `onTerminate`
callback. Termination of the producer happens in the following scenarios:
```swift
// Termination through task cancellation
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
source.setOnTerminationCallback { print("Terminated") }
let task = Task {
await channel.next()
}
task.cancel() // Prints Terminated
```
```swift
// Termination through deiniting the channel
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
source.setOnTerminationCallback { print("Terminated") }
_ = consume channel // Prints Terminated
```
```swift
// Termination through finishing the source and consuming the last element
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source = consume channelAndSource.source
source.setOnTerminationCallback { print("Terminated") }
_ = try await source.send(1)
source.finish()
print(await channel.next()) // Prints Optional(1)
await channel.next() // Prints Terminated
```
```swift
// Termination through deiniting the last source and consuming the last element
let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
of: Int.self,
backpressureStrategy: .watermark(low: 2, high: 4)
)
var channel = consume channelAndSource.channel
var source1 = consume channelAndSource.source
var source2 = source1.makeAdditionalSource()
source1.setOnTerminationCallback { print("Terminated") }
_ = try await source1.send(1)
_ = consume source1
_ = try await source2.send(2)
print(await channel.next()) // Prints Optional(1)
print(await channel.next()) // Prints Optional(2)
_ = consume source2
await channel.next() // Prints Terminated
```
Similar to the downstream consumer termination, trying to send more elements after the
producer has been terminated will result in an error thrown from the send methods.
## Detailed design
```swift
#if compiler(>=6.1)
/// An error that is thrown from the various `send` methods of the
/// ``MultiProducerSingleConsumerAsyncChannel/Source``.
///
/// This error is thrown when the channel is already finished when
/// trying to send new elements to the source.
public struct MultiProducerSingleConsumerAsyncChannelAlreadyFinishedError: Error { }
/// A multi-producer single-consumer channel.
///
/// The ``MultiProducerSingleConsumerAsyncChannel`` provides a ``MultiProducerSingleConsumerAsyncChannel/Source`` to
/// send values to the channel. The channel supports different back pressure strategies to control the
/// buffering and demand. The channel will buffer values until its backpressure strategy decides that the
/// producer have to wait.
///
/// This channel is also suitable for the single-producer single-consumer use-case
///
/// ## Using a MultiProducerSingleConsumerAsyncChannel
///
/// To use a ``MultiProducerSingleConsumerAsyncChannel`` you have to create a new channel with its source first by calling
/// the ``MultiProducerSingleConsumerAsyncChannel/makeChannel(of:throwing:BackpressureStrategy:)`` method.
/// Afterwards, you can pass the source to the producer and the channel to the consumer.
///
/// ```
/// let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
/// of: Int.self,
/// backpressureStrategy: .watermark(low: 2, high: 4)
/// )
///
/// // The channel and source can be extracted from the returned type
/// let channel = consume channelAndSource.channel
/// let source = consume channelAndSource.source
/// ```
///
/// ### Asynchronous producing
///
/// Values can be send to the source from asynchronous contexts using ``MultiProducerSingleConsumerAsyncChannel/Source/send(_:)-8eo96``
/// and ``MultiProducerSingleConsumerAsyncChannel/Source/send(contentsOf:)``. Backpressure results in calls
/// to the `send` methods to be suspended. Once more elements should be produced the `send` methods will be resumed.
///
/// ```
/// try await withThrowingTaskGroup(of: Void.self) { group in
/// group.addTask {
/// try await source.send(1)
/// try await source.send(2)
/// try await source.send(3)
/// }
///
/// for await element in channel {
/// print(element)
/// }
/// }
/// ```
///
/// ### Synchronous produceing
///
/// Values can also be send to the source from synchronous context. Backpressure is also exposed on the synchronous contexts; however,
/// it is up to the caller to decide how to properly translate the backpressure to underlying producer e.g. by blocking the thread.
///
/// ```swift
/// do {
/// let sendResult = try source.send(contentsOf: sequence)
///
/// switch sendResult {
/// case .produceMore:
/// // Trigger more production in the underlying system
///
/// case .enqueueCallback(let callbackHandle):
/// // There are enough values in the channel already. We need to enqueue
/// // a callback to get notified when we should produce more.
/// callbackHandle.enqueueCallback(onProduceMore: { result in
/// switch result {
/// case .success:
/// // Trigger more production in the underlying system
/// case .failure(let error):
/// // Terminate the underlying producer
/// }
/// })
/// }
/// } catch {
/// // `send(contentsOf:)` throws if the channel already terminated
/// }
/// ```
///
/// ### Multiple producers
///
/// To support multiple producers the source offers a ``Source/makeAdditionalSource()`` method to produce a new source.
///
/// ### Terminating the production of values
///
/// The consumer can be terminated through multiple ways:
/// - Calling ``Source/finish(throwing:)``.
/// - Deiniting all sources.
///
/// In both cases, if there are still elements buffered by the channel, then the consumer will receive
/// all buffered elements. Afterwards it will be terminated.
///
/// ### Observing termination of the consumer
///
/// When the consumer stops consumption by either deiniting the channel or the task calling ``next(isolation:)``
/// getting cancelled, the source will get notified about the termination if a termination callback has been set
/// before by calling ``Source/setOnTerminationCallback(_:)``.
@available(macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, *)
public struct MultiProducerSingleConsumerAsyncChannel<Element, Failure: Error>: ~Copyable {
/// A struct containing the initialized channel and source.
///
/// This struct can be deconstructed by consuming the individual
/// components from it.
///
/// ```swift
/// let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
/// of: Int.self,
/// backpressureStrategy: .watermark(low: 5, high: 10)
/// )
/// var channel = consume channelAndSource.channel
/// var source = consume channelAndSource.source
/// ```
@frozen
public struct ChannelAndStream : ~Copyable {
/// The channel.
public var channel: MultiProducerSingleConsumerAsyncChannel
/// The source.
public var source: Source
}
/// Initializes a new ``MultiProducerSingleConsumerAsyncChannel`` and an ``MultiProducerSingleConsumerAsyncChannel/Source``.
///
/// - Parameters:
/// - elementType: The element type of the channel.
/// - failureType: The failure type of the channel.
/// - backpressureStrategy: The backpressure strategy that the channel should use.
/// - Returns: A tuple containing the channel and its source. The source should be passed to the
/// producer while the channel should be passed to the consumer.
public static func makeChannel(
of elementType: Element.Type = Element.self,
throwing failureType: Failure.Type = Never.self,
backpressureStrategy: Source.BackpressureStrategy
) -> ChannelAndStream
/// Returns the next element.
///
/// If this method returns `nil` it indicates that no further values can ever
/// be returned. The channel automatically closes when all sources have been deinited.
///
/// If there are no elements and the channel has not been finished yet, this method will
/// suspend until an element is send to the channel.
///
/// If the task calling this method is cancelled this method will return `nil`.
///
/// - Parameter isolation: The callers isolation.
/// - Returns: The next buffered element.
public func next(isolation: isolated (any Actor)? = #isolation) async throws(Failure) -> Element?
}
@available(macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, *)
extension MultiProducerSingleConsumerAsyncChannel {
/// A struct to send values to the channel.
///
/// Use this source to provide elements to the channel by calling one of the `send` methods.
public struct Source: ~Copyable, Sendable {
/// A struct representing the backpressure of the channel.
public struct BackpressureStrategy: Sendable {
/// A backpressure strategy using a high and low watermark to suspend and resume production respectively.
///
/// - Parameters:
/// - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
/// - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
public static func watermark(low: Int, high: Int) -> BackpressureStrategy
/// A backpressure strategy using a high and low watermark to suspend and resume production respectively.
///
/// - Parameters:
/// - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
/// - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
/// - waterLevelForElement: A closure used to compute the contribution of each buffered element to the current water level.
///
/// - Note, `waterLevelForElement` will be called on each element when it is written into the source and when
/// it is consumed from the channel, so it is recommended to provide a function that runs in constant time.
public static func watermark(low: Int, high: Int, waterLevelForElement: @escaping @Sendable (borrowing Element) -> Int) -> BackpressureStrategy
/// An unbounded backpressure strategy.
///
/// - Important: Only use this strategy if the production of elements is limited through some other mean. Otherwise
/// an unbounded backpressure strategy can result in infinite memory usage and cause
/// your process to run out of memory.
public static func unbounded() -> BackpressureStrategy
}
/// A type that indicates the result of sending elements to the source.
public enum SendResult: ~Copyable, Sendable {
/// A handle that is returned when the channel's backpressure strategy indicated that production should
/// be suspended. Use this handle to enqueue a callback by calling the ``CallbackHandle/enqueueCallback(onProduceMore:)`` method.
///
/// - Important: ``CallbackHandle/enqueueCallback(onProduceMore:)`` and ``CallbackHandle/cancelCallback()`` must
/// only be called once.
public struct CallbackHandle: Sendable, Hashable {
/// Enqueues a callback that will be invoked once more elements should be produced.
///
/// - Important: Calling enqueue more than once is **not allowed**.
///
/// - Parameters:
/// - onProduceMore: The callback which gets invoked once more elements should be produced.
@inlinable
public mutating func enqueueCallback(
onProduceMore: sending @escaping (Result<Void, Error>) -> Void
)
/// Cancel an enqueued callback.
///
/// - Note: This methods supports being called before ``enqueueCallback(onProduceMore:)`` is called.
///
/// - Important: Calling enqueue more than once is **not allowed**.
@inlinable
public mutating func cancelCallback()
}
/// Indicates that more elements should be produced and send to the source.
case produceMore
/// Indicates that a callback should be enqueued.
case enqueueCallback(CallbackHandle)
}
/// A callback to invoke when the channel finished.
///
/// This is called after the last element has been consumed by the channel.
public func setOnTerminationCallback(_ callback: @escaping @Sendable () -> Void)
/// Creates a new source which can be used to send elements to the channel concurrently.
///
/// The channel will only automatically be finished if all existing sources have been deinited.
///
/// - Returns: A new source for sending elements to the channel.
public mutating func makeAdditionalSource() -> Source
/// Sends new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// - Parameter sequence: The elements to send to the channel.
/// - Returns: The result that indicates if more elements should be produced at this time.
public mutating func send<S>(
contentsOf sequence: consuming sending S
) throws -> SendResult where Element == S.Element, S: Sequence
/// Send the element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// - Parameter element: The element to send to the channel.
/// - Returns: The result that indicates if more elements should be produced at this time.
public mutating func send(_ element: sending consuming Element) throws -> SendResult
/// Send new elements to the channel and provide a callback which will be invoked once more elements should be produced.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then `onProduceMore` will be invoked with
/// a `Result.failure`.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
/// - onProduceMore: The callback which gets invoked once more elements should be produced. This callback might be
/// invoked during the call to ``send(contentsOf:onProduceMore:)``.
public mutating func send<S>(
contentsOf sequence: consuming sending S,
onProduceMore: @escaping @Sendable (Result<Void, Error>) -> Void
) where Element == S.Element, S: Sequence
/// Sends the element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then `onProduceMore` will be invoked with
/// a `Result.failure`.
///
/// - Parameters:
/// - element: The element to send to the channel.
/// - onProduceMore: The callback which gets invoked once more elements should be produced. This callback might be
/// invoked during the call to ``send(_:onProduceMore:)``.
public mutating func send(
_ element: consuming sending Element,
onProduceMore: @escaping @Sendable (Result<Void, Error>
) -> Void)
/// Send new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
public mutating func send<S>(
contentsOf sequence: consuming sending S
) async throws where Element == S.Element, S: Sequence
/// Send new element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - element: The element to send to the channel.
public mutating func send(_ element: consuming sending Element) async throws
/// Send the elements of the asynchronous sequence to the channel.
///
/// This method returns once the provided asynchronous sequence or the channel finished.
///
/// - Important: This method does not finish the source if consuming the upstream sequence terminated.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
public mutating func send<S>(
contentsOf sequence: consuming sending S
) async throws where Element: Sendable, Element == S.Element, S: Sendable, S: AsyncSequence
/// Indicates that the production terminated.
///
/// After all buffered elements are consumed the subsequent call to ``MultiProducerSingleConsumerAsyncChannel/next(isolation:)`` will return
/// `nil` or throw an error.
///
/// Calling this function more than once has no effect. After calling finish, the channel enters a terminal state and doesn't accept
/// new elements.
///
/// - Parameters:
/// - error: The error to throw, or `nil`, to finish normally.
public consuming func finish(throwing error: Failure? = nil)
}
}
@available(macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0, *)
extension MultiProducerSingleConsumerAsyncChannel {
/// Converts the channel to an asynchronous sequence for consumption.
///
/// - Important: The returned asynchronous sequence only supports a single iterator to be created and
/// will fatal error at runtime on subsequent calls to `makeAsyncIterator`.
public consuming func asyncSequence() -> some (AsyncSequence<Element, Failure> & Sendable)
}
```
## Comparison to other root asynchronous primitives
### swift-async-algorithm: AsyncChannel
The `AsyncChannel` is a multi-consumer/multi-producer root asynchronous sequence
which can be used to communicate between two tasks. It only offers asynchronous
production APIs and has an effective buffer of one per producer. This means that
any producer will be suspended until its value has been consumed. `AsyncChannel`
can handle multiple consumers and resumes them in FIFO order.
### swift-nio: NIOAsyncSequenceProducer
The NIO team has created their own root asynchronous sequence with the goal to
provide a high-performance sequence that can be used to bridge a NIO `Channel`
inbound stream into Concurrency. The `NIOAsyncSequenceProducer` is a highly
generic and fully inlinable type and quite unwieldy to use. This proposal is
heavily inspired by the learnings from this type but tries to create a more
flexible and easier-to-use API that fits into the standard library.
## Future directions
### Adaptive backpressure strategy
The high/low watermark strategy is common in networking code; however, there are
other strategies such as an adaptive strategy that we could offer in the future.
An adaptive strategy regulates the backpressure based on the rate of
consumption and production. With the proposed new APIs, we can easily add further
strategies.
### Support `~Copyable` elements
In the future, we can extend the channel to support `~Copyable` elements. We
only need an underlying buffer primitive that can hold `~Copyable` types, and the
continuations need to support `~Copyable` elements as well. By making the
channel not directly conform to `AsyncSequence`, we can support this down the
road.
## Alternatives considered
### Provide an `onTermination` callback to the factory method
During development of the new APIs, I first tried to provide the `onTermination`
callback in the `makeChannel` method. However, that showed significant usability
problems in scenarios where one wants to store the source in a type and
reference `self` in the `onTermination` closure at the same time; hence, I kept
the current pattern of setting the `onTermination` closure on the source.
### Provide a `onConsumerCancellation` callback
During the pitch phase, it was raised that we should provide a
`onConsumerCancellation` callback which gets invoked once the asynchronous
channel notices that the consuming task got cancelled. This callback could be
used to customize how cancellation is handled by the channel, e.g. one could
imagine writing a few more elements to the channel before finishing it. Right now,
the channel immediately returns `nil` or throws a `CancellationError` when it
notices cancellation. This proposal decided not to provide this customization
because it opens up the possibility that asynchronous channels are not terminating
when implemented incorrectly. Additionally, asynchronous sequences are not the
only place where task cancellation leads to an immediate error being thrown i.e.
`Task.sleep()` does the same. Hence, the value of the asynchronous not
terminating immediately brings little value when the next call in the iterating
task might throw. However, the implementation is flexible enough to add this in
the future, and we can just default it to the current behaviour.
### Create a custom type for the `Result` of the `onProduceMore` callback
The `onProducerMore` callback takes a `Result<Void, Error>` which is used to
indicate if the producer should produce more or if the asynchronous channel
finished. We could introduce a new type for this, but the proposal decided
against it since it effectively is a result type.
### Use an initializer instead of factory methods
Instead of providing a `makeChannel` factory method, we could use an initializer
approach that takes a closure which gets the `Source` passed into. A similar API
has been offered with the `Continuation` based approach and
[SE-0388](https://github.com/apple/swift-evolution/blob/main/proposals/0388-async-stream-factory.md)
introduced new factory methods to solve some of the usability ergonomics with
the initializer based APIs.
### Provide the type on older compilers
To achieve maximum performance, the implementation is using `~Copyable` extensively.
On Swift versions before 6.1, there is a https://github.com/swiftlang/swift/issues/78048 when using; hence, this type
is only usable with Swift 6.1 and later compilers.
## Acknowledgements
- [Johannes Weiss](https://github.com/weissi) - For making me aware how
important this problem is and providing great ideas on how to shape the API.
- [Philippe Hausler](https://github.com/phausler) - For helping me designing the
APIs and continuously providing feedback
- [George Barnett](https://github.com/glbrntt) - For providing extensive code
reviews and testing the implementation.
- [Si Beaumont](https://github.com/simonjbeaumont) - For implementing the element size dependent strategy
+197
View File
@@ -0,0 +1,197 @@
# Share
* Proposal: [SAA-0015](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/0016-share.md)
* Author(s): [Philippe Hausler](https://github.com/phausler)
* Status: **Accepted**
* Implementation: [
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncShareSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestShare.swift)
]
* Decision Notes:
* Bugs:
## Introduction
Many of the AsyncSequence adopting types only permit a one singular consumption. However there are many times that the same produced values are useful in more than one place. Out of that mechanism there are a few approaches to share, distribute, and broadcast those values. This proposal will focus on one concept; sharing. Sharing is where each consumption independently can make forward progress and get the same values but do not replay from the beginning of time.
## Motivation
There are many potential usages for the sharing concept of AsyncSequences.
One such example is the case where a source of data as an asynchronous sequence needs to be consumed by updating UI, logging, and additionally a network connection. This particular case does not matter on which uses but instead that those uses are independent of each other. It would not be expected for networking to block or delay the updates to UI, nor should logging. This example case also illustrates that the isolation of each side might be different and that some of the sides may not tolerate coalescing or dropping values.
There are many other use cases that have been requested for this family of algorithms. Since the release of AsyncAlgorithms it has perhaps been the most popularly requested set of behaviors as additions to the package.
## Proposed solution
AsyncAlgorithms will introduce a new extension function on AsyncSequence that will provide a shareable asynchronous sequence that will produce the same values upon iteration from multiple instances of it's AsyncIterator. Those iterations can take place in multiple isolations.
When values from a differing isolation cannot be coalesced, the two options available are either awaiting (an exertion of back-pressure across the sequences) or buffering (an internal back-pressure to a buffer). Replaying the values from the beginning of the creation of the sequence is a distinctly different behavior that should be considered a different use case. This then leaves the behavioral characteristic of this particular operation of share as; sharing a buffer of values started from the initialization of a new iteration of the sequence. Control over that buffer should then have options to determine the behavior, similar to how AsyncStream allows that control. It should have options to be unbounded, buffering the oldest count of elements, or buffering the newest count of elements.
It is critical to identify that this is one algorithm in the family of algorithms for sharing values. It should not attempt to solve all behavioral requirements but instead serve a common set of them that make cohesive sense together. This proposal is not mutually exclusive to the other algorithms in the sharing family.
## Detailed design
A new extension will be added to return a `Sendable` `AsyncSequence`. This extension will take a buffering policy to identify how the buffer will be handled when iterations do not consume at the same rate.
The `Sendable` annotation identifies to the developer that this sequence can be shared and stored in an existental `any`.
```swift
extension AsyncSequence where Element: Sendable {
public func share(
bufferingPolicy: AsyncBufferSequencePolicy = .bounded(1)
) -> some AsyncSequence<Element, Failure> & Sendable
}
```
The buffer internally to the share algorithm will only extend back to the furthest element available but there will only be a singular buffer shared across all iterators. This ensures that with the application of the buffering policy the storage size is as minimal as possible while still allowing all iterations to avoid dropping values and keeping the memory usage in check. The signature reuses the existing `AsyncBufferSequencePolicy` type to specify the behavior around buffering either responding to how it should limit emitting to the buffer or what should happen when the buffer is exceeded.
## Runtime Behavior
The runtime behaviors fall into a few categories; ordering, iteration isolation, cancellation, and lifetimes. To understand the beahviors there are a terms useful to define. Each creation of the AsyncIterator of the sequence and invocation of next will be referred to a side of the share iteration. The back pressure to the system to fetch a new element or termination is refered to as demand. The limit which is the pending gate for awaiting until the buffer has been serviced used for the `AsyncBufferSequencePolicy.bounded(_ : Int)` policy. The last special definition is that of the extent which is specifically in this case the lifetime of the asynchronous sequence itself.
When the underlying type backing the share algorithm is constructed a new extent is created; this is used for tracking the reference lifetime under the hood and is used to both house the iteration but also to identify the point at which no more sides can be constructed. When no more sides can be constructed and no sides are left to iterate then the backing iteration is canceled. This prevents any un-referenced task backing the iteration to not be leaked by the algorith itself.
That construction then creates an initial shared state and buffer. No task is started initially; it is only upon the first demand that the task backing the iteration is started; this means on the first call to next a task is spun up servicing all potential sides. The order of which the sides are serviced is not specified and cannot be relied upon, however the order of delivery within a side is always guarenteed to be ordered. The singular task servicing the iteration will be the only place holding any sort of iterator from the base `AsyncSequence`; so that iterator is isolated and not sent from one isolation to another. That iteration first awaits any limit availability and then awaits for a demand given by a side. After-which it then awaits an element or terminal event from the iterator and enqueues the elements to the buffer.
The buffer itself is only held in one location, each side however has a cursor index into that buffer and when values are consumed it adjusts the indexes accordingly; leaving the buffer usage only as big as the largest deficit. This means that new sides that are started post initial start up will not have a "replay" effect; that is a similar but distinct algorithm and is not addressed by this proposal. Any buffer size sensitive systems that wish to adjust behavior should be aware that specifying a policy is a suggested step. However in common usage similar to other such systems servicing desktop and mobile applications the common behavior is often unbounded. Alternatively desktop or mobile applications will often want `.bounded(1)` since that enforces the slowest consumption to drive the forward progress at most 1 buffered element. All of the use cases have a reasonable default of `.bounded(1)`; mobile, deskop, and server side uses. Leaving this as the default parameter keeps the progressive disclosure of the beahviors - such that the easiest thing to write is correct for all uses, and then more advanced control can be adjusted by passing in a specific policy. This default argument diverges slightly from AsyncStream, but follows a similar behavior to that of Combine's `share`.
As previously stated, the isolation of the iteration of the upstream/base AsyncSequence is to a detached task, this ensures that individual sides can have independent cancellation. Those cancellations will have the effect of remvoing that side from the shared iteration and cleaning up accordingly (including adjusting the trimming of the internal buffer).
Representing concurrent access is difficult to express all potential examples but there are a few cases included with this proposal to illustrate some of the behaviors. If a more comprehensive behavioral analysis is needed, it is strongly suggested to try out the pending pull request to identify how specific behaviors work. Please keep in mind that the odering between tasks is not specified, only the order within one side of iteration.
Practically this all means that a given iteration may be "behind" another and can eventually catch up (provided it is within the buffer limit).
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .unbounded)
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
This example will print a possible ordering of the following:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 3
Task 2 4
Task 1 1
Task 1 2
Task 1 3
Task 1 4
```
The order of the interleaving of the prints are not guaranteed; however the order of the elements per iteration is. Likewise in this buffering case it is guaranteed that all values are represented in the output.
If the creation were instead altered to the following:
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingLatest(2))
```
The output would print the possible ordering of:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 3
Task 1 4
```
Some values are dropped due to the buffering policy, but eventually they reach consistency. Which similarly works for the following:
```
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingOldest(2))
```
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 1
Task 1 2
```
However in this particular case the newest values are the dropped elements.
The `.bounded(N)` policy enforces consumption to prevent any side from being beyond a given amount away from other sides' consumption.
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bounded(1))
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
Will have a potential ordering output of:
```
Task 2 0
Task 2 1
Task 1 0
Task 1 1
Task 2 2
Task 1 2
Task 1 3
Task 1 4
Task 2 3
Task 2 4
```
In that example output Task 2 can get element 0 and 1 but must await until task 1 has caught up to the specified buffering. This limit means that no additional iteration (and no values are then dropped) is made until the buffer count is below the specified value.
## Effect on API resilience
This is an additive API and no existing systems are changed, however it will introduce a few new types that will need to be maintained as ABI interfaces. Since the intent of this is to provide a mechanism to store AsyncSequences to a shared context the type must be exposed as ABI (for type sizing).
## Alternatives considered
It has been considered that the buffering policy would be nested inside the `AsyncShareSequence` type. However since this seems to be something that will be useful for other types it makes sense to use an existing type from a top level type. However if it is determined that a general form of a buffering policy would require additional behaviors this might be a debatable placement to move back to an interior type similar to AsyncStream.
+49
View File
@@ -0,0 +1,49 @@
# Map Error
* Proposal: [SAA-NNNN](NNNN-map-error.md)
* Authors: [Clive Liu](https://github.com/clive819) [Philippe Hausler](https://github.com/phausler)
* Review Manager: TBD
* Status: **Awaiting review**
## Introduction
Asynchronous sequences often particpate in carefully crafted systems of algorithms spliced together. Some of those algorithms require that both the element type and the failure type are the same as each-other. In order to line those types up for the element we have the map algorithm today, however there are other times at which when using more specific APIs that transforming the Failure type is needed.
## Motivation
The `mapError` function empowers developers to elegantly transform errors within asynchronous sequences, enhancing code readability and maintainability.
Building failure-type-safe versions of zip or other algorithms will need to require that the associated Failure types are the same. Having an effecient and easy to use transformation routine to adjust the failure-types is then key to delivering or interfacing with those failure-type-safe algorithms.
## Proposed solution
A new extension and type will be added to transform the failure-types of AsyncSequences.
## Detailed design
The method will be applied to all AsyncSequences via an extension with the function name of `mapError`. This is spiritually related to the `mapError` method on `Result` and similar in functionality to other frameworks' methods of the similar naming. This will not return an opaque result since the type needs to be refined for `Sendable`; in that the `AsyncMapERrorSequence` is only `Sendable` when the base `AsyncSequence` is `Sendable`.
```swift
extension AsyncSequence {
public func mapError<MappedFailure: Error>(_ transform: @Sendable @escaping (Failure) async -> MappedFailure) -> AsyncMapErrorSequence<Element, MappedFailure>
}
public struct AsyncMapErrorSequence<Base: AsyncSequence, TransformedFailure: Error>: AsyncSequence { }
extension AsyncMapErrorSequence: Sendable where Base: Sendable { }
@available(*, unavailable)
extension AsyncMapErrorSequence.Iterator: Sendable {}
```
## Effect on API resilience
This cannot be back-deployed to 1.0 since it has a base requirement for the associated `Failure` and requires typed throws.
## Naming
The naming follows to current method naming of the Combine [mapError](https://developer.apple.com/documentation/combine/publisher/maperror(_:)) method and similarly the name of the method on `Result`
## Alternatives considered
It was initially considered that the return type would be opaque, however the only way to refine that as Sendable would be to have a disfavored overload; this ended up creating more ambiguity than it seemed worth.
+200
View File
@@ -0,0 +1,200 @@
# FlatMapLatest
* Proposal: [SAA-00nn](https://github.com/apple/swift-async-algorithms/blob/main/Evolution/00nn-flatMapLatest.md)
* Author(s): [Peter Friese](https://github.com/peterfriese)
* Status: **Proposed**
* Implementation:
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/FlatMapLatest/AsyncFlatMapLatestSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestFlatMapLatest.swift)
]
* Decision Notes:
* Bugs:
## Introduction
When transforming elements of an asynchronous sequence into new asynchronous sequences, there are cases where only the results from the most recent transformation are relevant, and previous transformations should be abandoned. This is particularly common in reactive user interfaces where rapid user input triggers asynchronous operations, but only the result of the latest operation matters.
## Motivation
Consider a search-as-you-type interface where each keystroke triggers a network request:
```swift
let searchQueries = userInputField.textChanges
// Without flatMapLatest - all requests complete, wasting resources
for await query in searchQueries {
let results = try await searchAPI(query)
displayResults(results) // May display stale results
}
```
Without automatic cancellation, earlier requests continue to completion even though their results are no longer relevant. This wastes network bandwidth, server resources, and may display stale results to the user if a slower request completes after a faster one.
The `flatMapLatest` operator solves this by automatically cancelling iteration on the previous inner sequence whenever a new element arrives from the outer sequence:
```swift
let searchResults = searchQueries.flatMapLatest { query in
searchAPI(query)
}
for await result in searchResults {
displayResults(result) // Only latest results displayed
}
```
This pattern is broadly applicable:
- **Location-based queries**: Cancel previous location lookups when user moves
- **Dynamic configuration**: Restart data loading when settings change
- **Auto-save**: Only save the most recent changes when user types rapidly
- **Real-time data**: Switch to new data streams based on user selections
## Proposed Solution
The `flatMapLatest` algorithm transforms each element from the base `AsyncSequence` into a new inner `AsyncSequence` using a transform closure. When a new element is produced by the base sequence, iteration on the current inner sequence is cancelled, and iteration begins on the newly created sequence.
The interface is available on all `AsyncSequence` types where both the base and inner sequences are `Sendable` along with two disfavored refinements to account for variations of typed throws signatures:
```swift
extension AsyncSequence where Self: Sendable {
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable where T.Failure == Failure, T.Element: Sendable, Element: Sendable
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, Failure> & Sendable where T.Failure == Never, T.Element: Sendable, Element: Sendable
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable where Failure == Never, T.Element: Sendable, Element: Sendable
}
```
This provides a clean API for expressing switching behavior:
```swift
userActions.flatMapLatest { action in
performAction(action)
}
```
## Detailed Design
The type that implements the algorithm emits elements from the inner sequences. It throws when either the base type or any inner sequence throws.
Since both the base sequence and inner sequences must be `Sendable` (to support concurrent iteration and cancellation), `AsyncFlatMapLatestSequence` is unconditionally `Sendable`.
### Implementation Strategy
The implementation uses a state machine pattern to ensure thread-safe operation:
1. **Generation Tracking**: Each new inner sequence is assigned a generation number. Elements from stale generations are discarded.
2. **Explicit Cancellation**: When a new outer element arrives, the previous inner sequence's task is explicitly cancelled and its continuation is resumed with a cancellation error.
3. **Lock-Based Coordination**: A `Lock` protects the state machine from concurrent access.
4. **Continuation Management**: The storage manages continuations for both upstream (outer/inner sequences) and downstream (consumer) demand.
This approach eliminates race conditions where cancelled sequences could emit stale values.
### Behavioral Characteristics
**Switching**: When the base sequence produces a new element, the current inner sequence iteration is immediately cancelled. Any elements it would have produced are lost.
**Completion**: The sequence completes when:
1. The base sequence finishes producing elements, AND
2. The final inner sequence finishes producing elements
**Error Handling**: If either the base sequence or any inner sequence throws, the error is immediately propagated and all tasks are cancelled.
**Cancellation**: Cancelling the downstream iteration cancels both the base sequence iteration and the current inner sequence iteration.
## Example
```swift
let requests = AsyncStream<String> { continuation in
continuation.yield("query1")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("query2")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("query3")
continuation.finish()
}
let responses = requests.flatMapLatest { query in
AsyncStream<String> { continuation in
continuation.yield("\(query): loading")
try? await Task.sleep(for: .milliseconds(50))
continuation.yield("\(query): complete")
continuation.finish()
}
}
for await response in responses {
print(response)
}
// Output (may vary due to timing):
// query1: loading
// query2: loading
// query3: loading
// query3: complete
```
In this example, the earlier queries (query1 and query2) are cancelled before they complete, so only query3 produces its complete response.
## Effect on API Resilience
This is an additive API. No existing systems are changed. The new types introduced are:
- `AsyncFlatMapLatestSequence<Base, Inner>`: The sequence type
- Associated private types for the state machine implementation
These types will be part of the ABI surface area.
## Alternatives Considered
### Alternative Names
**`switchMap`**: Used in ReactiveX, but "switch" in Swift has strong association with control flow statements.
**`switchToLatest`**: Combine's terminology, but `flatMapLatest` is more discoverable alongside other `map` variants.
**`flatMap(...).latest()`**: Requires a hypothetical `flatMap` first, adding complexity.
### Delivering All Elements
An alternative behavior would buffer elements from cancelled sequences and deliver them later. However, this contradicts the core purpose of "latest" semantics and would be better served by a different operator.
### No Automatic Cancellation
Requiring manual cancellation would place significant burden on developers and be error-prone. The automatic cancellation is the key value proposition.
### Shorthand version for async sequences of async sequences
Mimicing combine a shorthand extension of `switchToLatest` could be added as an extension when the base AsyncSequence's element conforms to AsyncSequence. The implementation of which is trivial:
```swift
extension AsyncSequence where Self: Sendable {
func switchToLatest() -> some AsyncSequence<Element.Element, Failure> where Element: AsyncSequence, Element.Failure == Failure, Element.Element: Sendable, Element: Sendable {
flatMapLatest { $0 }
}
@_disfavoredOverload
func switchToLatest() -> some AsyncSequence<Element.Element, Failure> where Element: AsyncSequence, Element.Failure == Never, Element.Element: Sendable, Element: Sendable {
flatMapLatest { $0 }
}
@_disfavoredOverload
func switchToLatest() -> some AsyncSequence<Element.Element, Failure> where Element: AsyncSequence, Failure == Never, Element.Element: Sendable, Element: Sendable {
flatMapLatest { $0 }
}
}
```
## Comparison with Other Libraries
**ReactiveX**: ReactiveX has an [API definition of switchMap](https://reactivex.io/documentation/operators/flatmap.html) that performs the same operation for Observables, switching to the latest inner Observable.
**Combine**: Combine has an [API definition of switchToLatest()](https://developer.apple.com/documentation/combine/publisher/switchtolatest()) which subscribes to the most recent Publisher produced by an upstream Publisher of Publishers. `flatMapLatest` combines the map and switch operations into a single convenient operator.
**RxSwift**: RxSwift calls this operator `flatMapLatest`, which is the naming this proposal adopts.
+141 -18
View File
@@ -1,51 +1,174 @@
// swift-tools-version: 5.8
// swift-tools-version: 6.2
import PackageDescription
import CompilerPluginSupport
let AsyncAlgorithms_v1_0 = "AvailabilityMacro=AsyncAlgorithms 1.0:macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 6.0"
#if compiler(>=6.0) && swift(>=6.0) // 5.10 doesnt support visionOS availability
let AsyncAlgorithms_v1_1 =
"AvailabilityMacro=AsyncAlgorithms 1.1:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
let AsyncAlgorithms_v1_2 =
"AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
let AsyncAlgorithms_v1_3 =
"AvailabilityMacro=AsyncAlgorithms 1.3:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
#else
let AsyncAlgorithms_v1_1 = "AvailabilityMacro=AsyncAlgorithms 1.1:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0"
let AsyncAlgorithms_v1_2 = "AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0"
let AsyncAlgorithms_v1_3 = "AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0"
#endif
let availabilityMacros: [SwiftSetting] = [
.enableExperimentalFeature(
AsyncAlgorithms_v1_0
),
.enableExperimentalFeature(
AsyncAlgorithms_v1_1
),
.enableExperimentalFeature(
AsyncAlgorithms_v1_2
),
.enableExperimentalFeature(
AsyncAlgorithms_v1_3
),
]
let package = Package(
name: "swift-async-algorithms",
platforms: [
.macOS("10.15"),
.iOS("13.0"),
.tvOS("13.0"),
.watchOS("6.0")
],
products: [
.library(name: "AsyncAlgorithms", targets: ["AsyncAlgorithms"]),
.library(name: "AsyncStreaming", targets: ["AsyncStreaming"]),
],
dependencies: [
.package(url: "https://github.com/apple/swift-collections.git", from: "1.0.4"),
.package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0"),
traits: [
.default(
enabledTraits: [
// "UnstableAsyncStreaming"
]
),
.trait(
name: "UnstableAsyncStreaming",
description: """
Enables source unstable async streaming components in the _AsyncStreaming
module. Do not rely on this module in API stable packages.
"""
),
],
targets: [
.target(
name: "AsyncAlgorithms",
dependencies: [.product(name: "Collections", package: "swift-collections")],
dependencies: [
.product(name: "OrderedCollections", package: "swift-collections"),
.product(name: "DequeModule", package: "swift-collections"),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.target(
name: "AsyncStreaming",
dependencies: [
.product(name: "BasicContainers", package: "swift-collections"),
.product(name: "ContainersPreview", package: "swift-collections"),
],
swiftSettings: [
.enableExperimentalFeature("StrictConcurrency=complete"),
.enableExperimentalFeature("SuppressedAssociatedTypesWithDefaults"),
.enableExperimentalFeature("LifetimeDependence"),
.enableExperimentalFeature("Lifetimes"),
.enableUpcomingFeature("LifetimeDependence"),
.enableUpcomingFeature("NonisolatedNonsendingByDefault"),
.enableUpcomingFeature("InferIsolatedConformances"),
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("MemberImportVisibility"),
.enableUpcomingFeature("InternalImportsByDefault"),
]
),
.target(
name: "AsyncSequenceValidation",
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"],
swiftSettings: [
.enableExperimentalFeature("StrictConcurrency=complete"),
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.systemLibrary(name: "_CAsyncSequenceValidationSupport"),
.target(
name: "AsyncAlgorithms_XCTest",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"],
swiftSettings: [
.enableExperimentalFeature("StrictConcurrency=complete"),
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.testTarget(
name: "AsyncAlgorithmsTests",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation", "AsyncAlgorithms_XCTest"],
dependencies: [
.target(name: "AsyncAlgorithms"),
.target(
name: "AsyncSequenceValidation",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.custom("freebsd"),
.openbsd,
.wasi,
])
),
.target(
name: "AsyncAlgorithms_XCTest",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.custom("freebsd"),
.openbsd,
.wasi,
])
),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.testTarget(
name: "AsyncStreamingTests",
dependencies: [
"AsyncStreaming",
.product(name: "BasicContainers", package: "swift-collections"),
.product(name: "ContainersPreview", package: "swift-collections"),
],
swiftSettings: [
.enableExperimentalFeature("StrictConcurrency=complete"),
.enableExperimentalFeature("SuppressedAssociatedTypesWithDefaults"),
.enableExperimentalFeature("LifetimeDependence"),
.enableExperimentalFeature("Lifetimes"),
.enableUpcomingFeature("LifetimeDependence"),
.enableUpcomingFeature("NonisolatedNonsendingByDefault"),
.enableUpcomingFeature("InferIsolatedConformances"),
.enableUpcomingFeature("ExistentialAny"),
.enableUpcomingFeature("MemberImportVisibility"),
.enableUpcomingFeature("InternalImportsByDefault"),
]
),
]
)
if Context.environment["SWIFTCI_USE_LOCAL_DEPS"] == nil {
package.dependencies += [
.package(
url: "https://github.com/apple/swift-collections.git",
from: "1.5.0",
traits: [.trait(name: "UnstableContainersPreview", condition: .when(traits: ["UnstableAsyncStreaming"]))]
)
]
} else {
package.dependencies += [
.package(path: "../swift-collections")
]
}
+8 -5
View File
@@ -1,4 +1,4 @@
// swift-tools-version: 5.6
// swift-tools-version: 5.7
import PackageDescription
@@ -8,7 +8,7 @@ let package = Package(
.macOS("10.15"),
.iOS("13.0"),
.tvOS("13.0"),
.watchOS("6.0")
.watchOS("6.0"),
],
products: [
.library(name: "AsyncAlgorithms", targets: ["AsyncAlgorithms"]),
@@ -27,13 +27,16 @@ let package = Package(
),
.target(
name: "AsyncSequenceValidation",
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"]),
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"]
),
.systemLibrary(name: "_CAsyncSequenceValidationSupport"),
.target(
name: "AsyncAlgorithms_XCTest",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"]),
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"]
),
.testTarget(
name: "AsyncAlgorithmsTests",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation", "AsyncAlgorithms_XCTest"]),
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation", "AsyncAlgorithms_XCTest"]
),
]
)
+104
View File
@@ -0,0 +1,104 @@
// swift-tools-version: 5.8
import PackageDescription
import CompilerPluginSupport
// Availability Macros
let availabilityMacros: [SwiftSetting] = [
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.0:macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 6.0"
),
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.1:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
),
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.2:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
),
.enableExperimentalFeature(
"AvailabilityMacro=AsyncAlgorithms 1.3:macOS 15.0, iOS 18.0, tvOS 18.0, watchOS 11.0, visionOS 2.0"
),
]
let package = Package(
name: "swift-async-algorithms",
products: [
.library(name: "AsyncAlgorithms", targets: ["AsyncAlgorithms"])
],
targets: [
.target(
name: "AsyncAlgorithms",
dependencies: [
.product(name: "OrderedCollections", package: "swift-collections"),
.product(name: "DequeModule", package: "swift-collections"),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.target(
name: "AsyncSequenceValidation",
dependencies: ["_CAsyncSequenceValidationSupport", "AsyncAlgorithms"],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.systemLibrary(name: "_CAsyncSequenceValidationSupport"),
.target(
name: "AsyncAlgorithms_XCTest",
dependencies: ["AsyncAlgorithms", "AsyncSequenceValidation"],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
.testTarget(
name: "AsyncAlgorithmsTests",
dependencies: [
.target(name: "AsyncAlgorithms"),
.target(
name: "AsyncSequenceValidation",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.openbsd,
.wasi,
])
),
.target(
name: "AsyncAlgorithms_XCTest",
condition: .when(platforms: [
.macOS,
.iOS,
.tvOS,
.watchOS,
.visionOS,
.macCatalyst,
.android,
.linux,
.openbsd,
.wasi,
])
),
],
swiftSettings: availabilityMacros + [
.enableExperimentalFeature("StrictConcurrency=complete")
]
),
]
)
if Context.environment["SWIFTCI_USE_LOCAL_DEPS"] == nil {
package.dependencies += [
.package(url: "https://github.com/apple/swift-collections.git", from: "1.1.0")
]
} else {
package.dependencies += [
.package(path: "../swift-collections")
]
}
+1 -1
View File
@@ -67,7 +67,7 @@ To use the `AsyncAlgorithms` library in a SwiftPM project,
add the following line to the dependencies in your `Package.swift` file:
```swift
.package(url: "https://github.com/apple/swift-async-algorithms"),
.package(url: "https://github.com/apple/swift-async-algorithms", from: "1.0.0"),
```
Include `"AsyncAlgorithms"` as a dependency for your executable target:
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// An `AsyncSequence` that iterates over the adjacent pairs of the original
/// original `AsyncSequence`.
@@ -26,6 +27,7 @@ extension AsyncSequence {
///
/// - Returns: An `AsyncSequence` where the element is a tuple of two adjacent elements
/// or the original `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func adjacentPairs() -> AsyncAdjacentPairsSequence<Self> {
AsyncAdjacentPairsSequence(self)
@@ -34,6 +36,7 @@ extension AsyncSequence {
/// An `AsyncSequence` that iterates over the adjacent pairs of the original
/// `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncAdjacentPairsSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = (Base.Element, Base.Element)
@@ -83,7 +86,8 @@ public struct AsyncAdjacentPairsSequence<Base: AsyncSequence>: AsyncSequence {
}
}
extension AsyncAdjacentPairsSequence: Sendable where Base: Sendable, Base.Element: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncAdjacentPairsSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncAdjacentPairsSequence.Iterator: Sendable { }
extension AsyncAdjacentPairsSequence.Iterator: Sendable {}
@@ -30,6 +30,7 @@ This package has three main goals:
- <doc:Merge>
- <doc:Reductions>
- <doc:RemoveDuplicates>
- <doc:Share>
- <doc:Throttle>
- <doc:Timer>
- <doc:Zip>
@@ -1,10 +1,10 @@
# AsyncBufferedByteIterator
Provides a highly efficient iterator useful for iterating byte sequences derived from asynchronous read functions.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncBufferedByteIterator.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestBufferedByteIterator.swift)]
Provides a highly efficient iterator useful for iterating byte sequences derived from asynchronous read functions.
This type provides infrastructure for creating `AsyncSequence` types with an `Element` of `UInt8` backed by file descriptors or similar read sources.
```swift
@@ -1,5 +1,7 @@
# Chain
Chains two or more asynchronous sequences together sequentially.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChain2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncChain3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestChain.swift)]
@@ -1,10 +1,10 @@
# Combine Latest
Combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/CombineLatest/AsyncCombineLatest2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/CombineLatest/AsyncCombineLatest3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestCombineLatest.swift)]
Combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
```swift
let appleFeed = URL("http://www.example.com/ticker?symbol=AAPL").lines
let nasdaqFeed = URL("http://www.example.com/ticker?symbol=^IXIC").lines
@@ -1,3 +1,7 @@
# Effects
Lists the effects of all async algorithms.
| Type | Throws | Sendability |
|-----------------------------------------------------|--------------|-------------|
| `AsyncAdjacentPairsSequence` | rethrows | Conditional |
@@ -0,0 +1,139 @@
# FlatMapLatest
* Author(s): [Peter Friese](https://github.com/peterfriese)
Transforms elements into asynchronous sequences, emitting elements from only the most recent inner sequence.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/FlatMapLatest/AsyncFlatMapLatestSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestFlatMapLatest.swift)]
```swift
let searchQuery = AsyncStream<String> { continuation in
// User types into search field
continuation.yield("swi")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("swift")
try? await Task.sleep(for: .milliseconds(100))
continuation.yield("swift async")
continuation.finish()
}
let searchResults = searchQuery.flatMapLatest { query in
performSearch(query) // Returns AsyncSequence<SearchResult>
}
for try await result in searchResults {
print(result) // Only shows results from "swift async"
}
```
## Introduction
When transforming elements of an asynchronous sequence into new asynchronous sequences, you often want to abandon previous sequences when new data arrives. This is particularly useful for scenarios like search-as-you-type, where each keystroke triggers a new search request and you only care about results from the most recent query.
The `flatMapLatest` operator solves this by cancelling iteration on the previous inner sequence whenever a new element arrives from the outer sequence.
## Proposed Solution
The `flatMapLatest` algorithm transforms each element from the base `AsyncSequence` into a new inner `AsyncSequence` using the provided `transform` closure. When a new element is produced by the base sequence, iteration on the current inner sequence is cancelled, and iteration begins on the newly created sequence.
```swift
extension AsyncSequence where Self: Sendable {
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> AsyncFlatMapLatestSequence<Self, T>
}
```
This creates a concise way to express switching behavior:
```swift
userInput.flatMapLatest { input in
fetchDataFromNetwork(input)
}
```
In this case, each new user input cancels any ongoing network request and starts a fresh one, ensuring only the latest data is delivered.
## Detailed Design
The type that implements the algorithm emits elements from the inner sequences. It throws when either the base type or any inner sequence throws.
```swift
public struct AsyncFlatMapLatestSequence<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>: AsyncSequence, Sendable
where Base.Element: Sendable, Inner.Element: Sendable {
public typealias Element = Inner.Element
public struct Iterator: AsyncIteratorProtocol, Sendable {
public func next() async throws -> Element?
}
public func makeAsyncIterator() -> Iterator
}
```
The implementation uses a state machine to ensure thread-safe operation and generation tracking to prevent stale values from cancelled sequences.
### Behavior
- **Switching**: When a new element arrives from the base sequence, the current inner sequence is cancelled immediately
- **Completion**: The sequence completes when the base sequence finishes and the final inner sequence completes
- **Error Handling**: Errors from either the base or inner sequences are propagated immediately
- **Cancellation**: Cancelling iteration cancels both the base and current inner sequence
## Use Cases
### Search as You Type
```swift
let searchField = AsyncStream<String> { continuation in
// Emit search queries as user types
continuation.yield("s")
continuation.yield("sw")
continuation.yield("swift")
}
let results = searchField.flatMapLatest { query in
searchAPI(for: query)
}
```
Only the results for "swift" will be emitted, as earlier queries are cancelled.
### Location-Based Data
```swift
let locationUpdates = CLLocationManager.shared.locationUpdates
let nearbyPlaces = locationUpdates.flatMapLatest { location in
fetchNearbyPlaces(at: location)
}
```
Each location update triggers a new search, cancelling any ongoing requests for previous locations.
### Dynamic Configuration
```swift
let settings = userSettingsStream
let data = settings.flatMapLatest { config in
loadData(with: config)
}
```
When settings change, data loading is restarted with the new configuration.
## Comparison with Other Operators
**`map`**: Transforms elements synchronously without producing sequences.
**`flatMap`** (if it existed): Would emit elements from all inner sequences concurrently, not cancelling previous ones.
**`switchToLatest`** (Combine): Equivalent operator in Combine framework - `flatMapLatest` is the `AsyncSequence` equivalent.
## Comparison with Other Libraries
**ReactiveX** ReactiveX has an [API definition of switchMap](https://reactivex.io/documentation/operators/flatmap.html) which performs the same operation for Observables.
**Combine** Combine has an [API definition of switchToLatest()](https://developer.apple.com/documentation/combine/publisher/switchtolatest()) which flattens a publisher of publishers by subscribing to the most recent one.
@@ -1,10 +1,10 @@
# Intersperse
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncInterspersedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestInterspersed.swift)]
Places a given value in between each element of the asynchronous sequence.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Interspersed/AsyncInterspersedSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/Interspersed/TestInterspersed.swift)]
```swift
let numbers = [1, 2, 3].async.interspersed(with: 0)
for await number in numbers {
@@ -1,12 +1,10 @@
# AsyncSyncSequence
This operation is available for all `Sequence` types.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncSyncSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestLazy.swift)]
Converts a non-asynchronous sequence into an asynchronous one.
This operation is available for all `Sequence` types.
```swift
let numbers = [1, 2, 3, 4].async
let characters = "abcde".async
@@ -1,10 +1,10 @@
# Merge
Merges two or more asynchronous sequences sharing the same element type into one singular asynchronous sequence.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Merge/AsyncMerge2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Merge/AsyncMerge3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestMerge.swift)]
Merges two or more asynchronous sequences sharing the same element type into one singular asynchronous sequence.
```swift
let appleFeed = URL(string: "http://www.example.com/ticker?symbol=AAPL")!.lines.map { "AAPL: " + $0 }
let nasdaqFeed = URL(string:"http://www.example.com/ticker?symbol=^IXIC")!.lines.map { "^IXIC: " + $0 }
@@ -0,0 +1,194 @@
# Share
* Author(s): [Philippe Hausler](https://github.com/phausler)
[
[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/AsyncShareSequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestShare.swift)
]
## Introduction
Many of the AsyncSequence adopting types only permit a one singular consumption. However there are many times that the same produced values are useful in more than one place. Out of that mechanism there are a few approaches to share, distribute, and broadcast those values. This proposal will focus on one concept; sharing. Sharing is where each consumption independently can make forward progress and get the same values but do not replay from the beginning of time.
## Motivation
There are many potential usages for the sharing concept of AsyncSequences.
One such example is the case where a source of data as an asynchronous sequence needs to be consumed by updating UI, logging, and additionally a network connection. This particular case does not matter on which uses but instead that those uses are independent of each other. It would not be expected for networking to block or delay the updates to UI, nor should logging. This example case also illustrates that the isolation of each side might be different and that some of the sides may not tolerate coalescing or dropping values.
There are many other use cases that have been requested for this family of algorithms. Since the release of AsyncAlgorithms it has perhaps been the most popularly requested set of behaviors as additions to the package.
## Proposed solution
AsyncAlgorithms will introduce a new extension function on AsyncSequence that will provide a shareable asynchronous sequence that will produce the same values upon iteration from multiple instances of it's AsyncIterator. Those iterations can take place in multiple isolations.
When values from a differing isolation cannot be coalesced, the two options available are either awaiting (an exertion of back-pressure across the sequences) or buffering (an internal back-pressure to a buffer). Replaying the values from the beginning of the creation of the sequence is a distinctly different behavior that should be considered a different use case. This then leaves the behavioral characteristic of this particular operation of share as; sharing a buffer of values started from the initialization of a new iteration of the sequence. Control over that buffer should then have options to determine the behavior, similar to how AsyncStream allows that control. It should have options to be unbounded, buffering the oldest count of elements, or buffering the newest count of elements.
It is critical to identify that this is one algorithm in the family of algorithms for sharing values. It should not attempt to solve all behavioral requirements but instead serve a common set of them that make cohesive sense together. This proposal is not mutually exclusive to the other algorithms in the sharing family.
## Detailed design
A new extension will be added to return a `Sendable` `AsyncSequence`. This extension will take a buffering policy to identify how the buffer will be handled when iterations do not consume at the same rate.
The `Sendable` annotation identifies to the developer that this sequence can be shared and stored in an existental `any`.
```swift
extension AsyncSequence where Element: Sendable {
public func share(
bufferingPolicy: AsyncBufferSequencePolicy = .bounded(1)
) -> some AsyncSequence<Element, Failure> & Sendable
}
```
The buffer internally to the share algorithm will only extend back to the furthest element available but there will only be a singular buffer shared across all iterators. This ensures that with the application of the buffering policy the storage size is as minimal as possible while still allowing all iterations to avoid dropping values and keeping the memory usage in check. The signature reuses the existing `AsyncBufferSequencePolicy` type to specify the behavior around buffering either responding to how it should limit emitting to the buffer or what should happen when the buffer is exceeded.
## Runtime Behavior
The runtime behaviors fall into a few categories; ordering, iteration isolation, cancellation, and lifetimes. To understand the beahviors there are a terms useful to define. Each creation of the AsyncIterator of the sequence and invocation of next will be referred to a side of the share iteration. The back pressure to the system to fetch a new element or termination is refered to as demand. The limit which is the pending gate for awaiting until the buffer has been serviced used for the `AsyncBufferSequencePolicy.bounded(_ : Int)` policy. The last special definition is that of the extent which is specifically in this case the lifetime of the asynchronous sequence itself.
When the underlying type backing the share algorithm is constructed a new extent is created; this is used for tracking the reference lifetime under the hood and is used to both house the iteration but also to identify the point at which no more sides can be constructed. When no more sides can be constructed and no sides are left to iterate then the backing iteration is canceled. This prevents any un-referenced task backing the iteration to not be leaked by the algorith itself.
That construction then creates an initial shared state and buffer. No task is started initially; it is only upon the first demand that the task backing the iteration is started; this means on the first call to next a task is spun up servicing all potential sides. The order of which the sides are serviced is not specified and cannot be relied upon, however the order of delivery within a side is always guarenteed to be ordered. The singular task servicing the iteration will be the only place holding any sort of iterator from the base `AsyncSequence`; so that iterator is isolated and not sent from one isolation to another. That iteration first awaits any limit availability and then awaits for a demand given by a side. After-which it then awaits an element or terminal event from the iterator and enqueues the elements to the buffer.
The buffer itself is only held in one location, each side however has a cursor index into that buffer and when values are consumed it adjusts the indexes accordingly; leaving the buffer usage only as big as the largest deficit. This means that new sides that are started post initial start up will not have a "replay" effect; that is a similar but distinct algorithm and is not addressed by this proposal. Any buffer size sensitive systems that wish to adjust behavior should be aware that specifying a policy is a suggested step. However in common usage similar to other such systems servicing desktop and mobile applications the common behavior is often unbounded. Alternatively desktop or mobile applications will often want `.bounded(1)` since that enforces the slowest consumption to drive the forward progress at most 1 buffered element. All of the use cases have a reasonable default of `.bounded(1)`; mobile, deskop, and server side uses. Leaving this as the default parameter keeps the progressive disclosure of the beahviors - such that the easiest thing to write is correct for all uses, and then more advanced control can be adjusted by passing in a specific policy. This default argument diverges slightly from AsyncStream, but follows a similar behavior to that of Combine's `share`.
As previously stated, the isolation of the iteration of the upstream/base AsyncSequence is to a detached task, this ensures that individual sides can have independent cancellation. Those cancellations will have the effect of remvoing that side from the shared iteration and cleaning up accordingly (including adjusting the trimming of the internal buffer).
Representing concurrent access is difficult to express all potential examples but there are a few cases included with this proposal to illustrate some of the behaviors. If a more comprehensive behavioral analysis is needed, it is strongly suggested to try out the pending pull request to identify how specific behaviors work. Please keep in mind that the odering between tasks is not specified, only the order within one side of iteration.
Practically this all means that a given iteration may be "behind" another and can eventually catch up (provided it is within the buffer limit).
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .unbounded)
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
This example will print a possible ordering of the following:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 3
Task 2 4
Task 1 1
Task 1 2
Task 1 3
Task 1 4
```
The order of the interleaving of the prints are not guaranteed; however the order of the elements per iteration is. Likewise in this buffering case it is guaranteed that all values are represented in the output.
If the creation were instead altered to the following:
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingLatest(2))
```
The output would print the possible ordering of:
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 3
Task 1 4
```
Some values are dropped due to the buffering policy, but eventually they reach consistency. Which similarly works for the following:
```
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bufferingOldest(2))
```
```
Task 2 0
Task 2 1
Task 2 2
Task 1 0
Task 2 4
Task 1 1
Task 1 2
```
However in this particular case the newest values are the dropped elements.
The `.bounded(N)` policy enforces consumption to prevent any side from being beyond a given amount away from other sides' consumption.
```swift
let exampleSource = [0, 1, 2, 3, 4].async.share(bufferingPolicy: .bounded(1))
let t1 = Task {
for await element in exampleSource {
if element == 0 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 1", element)
}
}
let t2 = Task {
for await element in exampleSource {
if element == 3 {
try? await Task.sleep(for: .seconds(1))
}
print("Task 2", element)
}
}
await t1.value
await t2.value
```
Will have a potential ordering output of:
```
Task 2 0
Task 2 1
Task 1 0
Task 1 1
Task 2 2
Task 1 2
Task 1 3
Task 1 4
Task 2 3
Task 2 4
```
In that example output Task 2 can get element 0 and 1 but must await until task 1 has caught up to the specified buffering. This limit means that no additional iteration (and no values are then dropped) is made until the buffer count is below the specified value.
## Effect on API resilience
This is an additive API and no existing systems are changed, however it will introduce a few new types that will need to be maintained as ABI interfaces. Since the intent of this is to provide a mechanism to store AsyncSequences to a shared context the type must be exposed as ABI (for type sizing).
## Alternatives considered
It has been considered that the buffering policy would be nested inside the `AsyncShareSequence` type. However since this seems to be something that will be useful for other types it makes sense to use an existing type from a top level type. However if it is determined that a general form of a buffering policy would require additional behaviors this might be a debatable placement to move back to an interior type similar to AsyncStream.
@@ -1,10 +1,10 @@
# Zip
Combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
[[Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Zip/AsyncZip2Sequence.swift), [Source](https://github.com/apple/swift-async-algorithms/blob/main/Sources/AsyncAlgorithms/Zip/AsyncZip3Sequence.swift) |
[Tests](https://github.com/apple/swift-async-algorithms/blob/main/Tests/AsyncAlgorithmsTests/TestZip.swift)]
Combines the latest values produced from two or more asynchronous sequences into an asynchronous sequence of tuples.
```swift
let appleFeed = URL(string: "http://www.example.com/ticker?symbol=AAPL")!.lines
let nasdaqFeed = URL(string: "http://www.example.com/ticker?symbol=^IXIC")!.lines
@@ -39,10 +39,11 @@
/// }
///
///
@available(AsyncAlgorithms 1.0, *)
public struct AsyncBufferedByteIterator: AsyncIteratorProtocol {
public typealias Element = UInt8
@usableFromInline var buffer: _AsyncBytesBuffer
/// Creates an asynchronous buffered byte iterator with a specified capacity and read function.
///
/// - Parameters:
@@ -55,7 +56,7 @@ public struct AsyncBufferedByteIterator: AsyncIteratorProtocol {
) {
buffer = _AsyncBytesBuffer(capacity: capacity, readFunction: readFunction)
}
/// Reads a byte out of the buffer if available. When no bytes are available, this will trigger
/// the read function to reload the buffer and then return the next byte from that buffer.
@inlinable @inline(__always)
@@ -65,14 +66,15 @@ public struct AsyncBufferedByteIterator: AsyncIteratorProtocol {
}
@available(*, unavailable)
extension AsyncBufferedByteIterator: Sendable { }
extension AsyncBufferedByteIterator: Sendable {}
@available(AsyncAlgorithms 1.0, *)
@frozen @usableFromInline
internal struct _AsyncBytesBuffer {
@usableFromInline
final class Storage {
fileprivate let buffer: UnsafeMutableRawBufferPointer
init(
capacity: Int
) {
@@ -82,19 +84,19 @@ internal struct _AsyncBytesBuffer {
alignment: MemoryLayout<AnyObject>.alignment
)
}
deinit {
buffer.deallocate()
}
}
@usableFromInline internal let storage: Storage
@usableFromInline internal var nextPointer: UnsafeRawPointer
@usableFromInline internal var endPointer: UnsafeRawPointer
internal let readFunction: @Sendable (UnsafeMutableRawBufferPointer) async throws -> Int
internal var finished = false
@usableFromInline init(
capacity: Int,
readFunction: @Sendable @escaping (UnsafeMutableRawBufferPointer) async throws -> Int
@@ -105,7 +107,7 @@ internal struct _AsyncBytesBuffer {
nextPointer = UnsafeRawPointer(s.buffer.baseAddress!)
endPointer = nextPointer
}
@inline(never) @usableFromInline
internal mutating func reloadBufferAndNext() async throws -> UInt8? {
if finished {
@@ -128,7 +130,7 @@ internal struct _AsyncBytesBuffer {
}
return try await next()
}
@inlinable @inline(__always)
internal mutating func next() async throws -> UInt8? {
if _fastPath(nextPointer != endPointer) {
@@ -17,20 +17,25 @@
/// - s2: The second asynchronous sequence.
/// - Returns: An asynchronous sequence that iterates first over the elements of `s1`, and
/// then over the elements of `s2`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chain<Base1: AsyncSequence, Base2: AsyncSequence>(_ s1: Base1, _ s2: Base2) -> AsyncChain2Sequence<Base1, Base2> where Base1.Element == Base2.Element {
public func chain<Base1: AsyncSequence, Base2: AsyncSequence>(
_ s1: Base1,
_ s2: Base2
) -> AsyncChain2Sequence<Base1, Base2> where Base1.Element == Base2.Element {
AsyncChain2Sequence(s1, s2)
}
/// A concatenation of two asynchronous sequences with the same element type.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncChain2Sequence<Base1: AsyncSequence, Base2: AsyncSequence> where Base1.Element == Base2.Element {
@usableFromInline
let base1: Base1
@usableFromInline
let base2: Base2
@usableFromInline
init(_ base1: Base1, _ base2: Base2) {
self.base1 = base1
@@ -38,24 +43,26 @@ public struct AsyncChain2Sequence<Base1: AsyncSequence, Base2: AsyncSequence> wh
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain2Sequence: AsyncSequence {
public typealias Element = Base1.Element
/// The iterator for a `AsyncChain2Sequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base1: Base1.AsyncIterator?
@usableFromInline
var base2: Base2.AsyncIterator?
@usableFromInline
init(_ base1: Base1.AsyncIterator, _ base2: Base2.AsyncIterator) {
self.base1 = base1
self.base2 = base2
}
@inlinable
public mutating func next() async rethrows -> Element? {
do {
@@ -72,14 +79,16 @@ extension AsyncChain2Sequence: AsyncSequence {
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base1.makeAsyncIterator(), base2.makeAsyncIterator())
}
}
extension AsyncChain2Sequence: Sendable where Base1: Sendable, Base2: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain2Sequence: Sendable where Base1: Sendable, Base2: Sendable {}
@available(*, unavailable)
extension AsyncChain2Sequence.Iterator: Sendable { }
extension AsyncChain2Sequence.Iterator: Sendable {}
@@ -18,23 +18,30 @@
/// - s3: The third asynchronous sequence.
/// - Returns: An asynchronous sequence that iterates first over the elements of `s1`, and
/// then over the elements of `s2`, and then over the elements of `s3`
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chain<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(_ s1: Base1, _ s2: Base2, _ s3: Base3) -> AsyncChain3Sequence<Base1, Base2, Base3> {
public func chain<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(
_ s1: Base1,
_ s2: Base2,
_ s3: Base3
) -> AsyncChain3Sequence<Base1, Base2, Base3> {
AsyncChain3Sequence(s1, s2, s3)
}
/// A concatenation of three asynchronous sequences with the same element type.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncChain3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence> where Base1.Element == Base2.Element, Base1.Element == Base3.Element {
public struct AsyncChain3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>
where Base1.Element == Base2.Element, Base1.Element == Base3.Element {
@usableFromInline
let base1: Base1
@usableFromInline
let base2: Base2
@usableFromInline
let base3: Base3
@usableFromInline
init(_ base1: Base1, _ base2: Base2, _ base3: Base3) {
self.base1 = base1
@@ -43,28 +50,30 @@ public struct AsyncChain3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Ba
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain3Sequence: AsyncSequence {
public typealias Element = Base1.Element
/// The iterator for a `AsyncChain3Sequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base1: Base1.AsyncIterator?
@usableFromInline
var base2: Base2.AsyncIterator?
@usableFromInline
var base3: Base3.AsyncIterator?
@usableFromInline
init(_ base1: Base1.AsyncIterator, _ base2: Base2.AsyncIterator, _ base3: Base3.AsyncIterator) {
self.base1 = base1
self.base2 = base2
self.base3 = base3
}
@inlinable
public mutating func next() async rethrows -> Element? {
do {
@@ -87,14 +96,16 @@ extension AsyncChain3Sequence: AsyncSequence {
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base1.makeAsyncIterator(), base2.makeAsyncIterator(), base3.makeAsyncIterator())
}
}
extension AsyncChain3Sequence: Sendable where Base1: Sendable, Base2: Sendable, Base3: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncChain3Sequence: Sendable where Base1: Sendable, Base2: Sendable, Base3: Sendable {}
@available(*, unavailable)
extension AsyncChain3Sequence.Iterator: Sendable { }
extension AsyncChain3Sequence.Iterator: Sendable {}
@@ -9,17 +9,25 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection`
/// type by testing if elements belong in the same group.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked<Collected: RangeReplaceableCollection>(into: Collected.Type, by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool) -> AsyncChunkedByGroupSequence<Self, Collected> where Collected.Element == Element {
public func chunked<Collected: RangeReplaceableCollection>(
into: Collected.Type,
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, Collected> where Collected.Element == Element {
AsyncChunkedByGroupSequence(self, grouping: belongInSameGroup)
}
/// Creates an asynchronous sequence that creates chunks by testing if elements belong in the same group.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked(by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool) -> AsyncChunkedByGroupSequence<Self, [Element]> {
public func chunked(
by belongInSameGroup: @escaping @Sendable (Element, Element) -> Bool
) -> AsyncChunkedByGroupSequence<Self, [Element]> {
chunked(into: [Element].self, by: belongInSameGroup)
}
}
@@ -46,7 +54,9 @@ extension AsyncSequence {
/// // [10, 20, 30]
/// // [10, 40, 40]
/// // [10, 20]
public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence where Collected.Element == Base.Element {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
/// The iterator for a `AsyncChunkedByGroupSequence` instance.
@@ -76,7 +86,7 @@ public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeR
} else {
hangingNext = nil
}
guard let first = firstOpt else {
return nil
}
@@ -86,23 +96,22 @@ public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeR
var prev = first
while let next = try await base.next() {
if grouping(prev, next) {
result.append(next)
prev = next
} else {
guard grouping(prev, next) else {
hangingNext = next
break
}
result.append(next)
prev = next
}
return result
}
}
@usableFromInline
let base : Base
let base: Base
@usableFromInline
let grouping : @Sendable (Base.Element, Base.Element) -> Bool
let grouping: @Sendable (Base.Element, Base.Element) -> Bool
@usableFromInline
init(_ base: Base, grouping: @escaping @Sendable (Base.Element, Base.Element) -> Bool) {
@@ -116,7 +125,8 @@ public struct AsyncChunkedByGroupSequence<Base: AsyncSequence, Collected: RangeR
}
}
extension AsyncChunkedByGroupSequence : Sendable where Base : Sendable, Base.Element : Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunkedByGroupSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncChunkedByGroupSequence.Iterator: Sendable { }
extension AsyncChunkedByGroupSequence.Iterator: Sendable {}
@@ -9,22 +9,35 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type on the uniqueness of a given subject.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked<Subject : Equatable, Collected: RangeReplaceableCollection>(into: Collected.Type, on projection: @escaping @Sendable (Element) -> Subject) -> AsyncChunkedOnProjectionSequence<Self, Subject, Collected> {
public func chunked<Subject: Equatable, Collected: RangeReplaceableCollection>(
into: Collected.Type,
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, Collected> {
AsyncChunkedOnProjectionSequence(self, projection: projection)
}
/// Creates an asynchronous sequence that creates chunks on the uniqueness of a given subject.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunked<Subject : Equatable>(on projection: @escaping @Sendable (Element) -> Subject) -> AsyncChunkedOnProjectionSequence<Self, Subject, [Element]> {
public func chunked<Subject: Equatable>(
on projection: @escaping @Sendable (Element) -> Subject
) -> AsyncChunkedOnProjectionSequence<Self, Subject, [Element]> {
chunked(into: [Element].self, on: projection)
}
}
/// An `AsyncSequence` that chunks on a subject when it differs from the last element.
public struct AsyncChunkedOnProjectionSequence<Base: AsyncSequence, Subject: Equatable, Collected: RangeReplaceableCollection>: AsyncSequence where Collected.Element == Base.Element {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunkedOnProjectionSequence<
Base: AsyncSequence,
Subject: Equatable,
Collected: RangeReplaceableCollection
>: AsyncSequence where Collected.Element == Base.Element {
public typealias Element = (Subject, Collected)
/// The iterator for a `AsyncChunkedOnProjectionSequence` instance.
@@ -67,22 +80,21 @@ public struct AsyncChunkedOnProjectionSequence<Base: AsyncSequence, Subject: Equ
while let next = try await base.next() {
let subj = projection(next)
if subj == first.0 {
result.append(next)
} else {
guard subj == first.0 else {
hangingNext = (subj, next)
break
}
result.append(next)
}
return (first.0, result)
}
}
@usableFromInline
let base : Base
let base: Base
@usableFromInline
let projection : @Sendable (Base.Element) -> Subject
let projection: @Sendable (Base.Element) -> Subject
@usableFromInline
init(_ base: Base, projection: @escaping @Sendable (Base.Element) -> Subject) {
@@ -96,7 +108,8 @@ public struct AsyncChunkedOnProjectionSequence<Base: AsyncSequence, Subject: Equ
}
}
extension AsyncChunkedOnProjectionSequence : Sendable where Base : Sendable, Base.Element : Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunkedOnProjectionSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncChunkedOnProjectionSequence.Iterator: Sendable { }
extension AsyncChunkedOnProjectionSequence.Iterator: Sendable {}
@@ -9,54 +9,93 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type of a given count or when a signal `AsyncSequence` produces an element.
public func chunks<Signal, Collected: RangeReplaceableCollection>(ofCount count: Int, or signal: Signal, into: Collected.Type) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal> where Collected.Element == Element {
@available(AsyncAlgorithms 1.0, *)
public func chunks<Signal, Collected: RangeReplaceableCollection>(
ofCount count: Int,
or signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: count, signal: signal)
}
/// Creates an asynchronous sequence that creates chunks of a given count or when a signal `AsyncSequence` produces an element.
public func chunks<Signal>(ofCount count: Int, or signal: Signal) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal> {
@available(AsyncAlgorithms 1.0, *)
public func chunks<Signal>(
ofCount count: Int,
or signal: Signal
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal> {
chunks(ofCount: count, or: signal, into: [Element].self)
}
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type when a signal `AsyncSequence` produces an element.
public func chunked<Signal, Collected: RangeReplaceableCollection>(by signal: Signal, into: Collected.Type) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal> where Collected.Element == Element {
@available(AsyncAlgorithms 1.0, *)
public func chunked<Signal, Collected: RangeReplaceableCollection>(
by signal: Signal,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, Signal> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: nil, signal: signal)
}
/// Creates an asynchronous sequence that creates chunks when a signal `AsyncSequence` produces an element.
@available(AsyncAlgorithms 1.0, *)
public func chunked<Signal>(by signal: Signal) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], Signal> {
chunked(by: signal, into: [Element].self)
}
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type of a given count or when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunks<C: Clock, Collected: RangeReplaceableCollection>(ofCount count: Int, or timer: AsyncTimerSequence<C>, into: Collected.Type) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>> where Collected.Element == Element {
public func chunks<C: Clock, Collected: RangeReplaceableCollection>(
ofCount count: Int,
or timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: count, signal: timer)
}
/// Creates an asynchronous sequence that creates chunks of a given count or when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunks<C: Clock>(ofCount count: Int, or timer: AsyncTimerSequence<C>) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>> {
public func chunks<C: Clock>(
ofCount count: Int,
or timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>> {
chunks(ofCount: count, or: timer, into: [Element].self)
}
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` type when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(by timer: AsyncTimerSequence<C>, into: Collected.Type) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>> where Collected.Element == Element {
public func chunked<C: Clock, Collected: RangeReplaceableCollection>(
by timer: AsyncTimerSequence<C>,
into: Collected.Type
) -> AsyncChunksOfCountOrSignalSequence<Self, Collected, AsyncTimerSequence<C>> where Collected.Element == Element {
AsyncChunksOfCountOrSignalSequence(self, count: nil, signal: timer)
}
/// Creates an asynchronous sequence that creates chunks when an `AsyncTimerSequence` fires.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func chunked<C: Clock>(by timer: AsyncTimerSequence<C>) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>> {
public func chunked<C: Clock>(
by timer: AsyncTimerSequence<C>
) -> AsyncChunksOfCountOrSignalSequence<Self, [Element], AsyncTimerSequence<C>> {
chunked(by: timer, into: [Element].self)
}
}
/// An `AsyncSequence` that chunks elements into collected `RangeReplaceableCollection` instances by either count or a signal from another `AsyncSequence`.
public struct AsyncChunksOfCountOrSignalSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection, Signal: AsyncSequence>: AsyncSequence, Sendable where Collected.Element == Base.Element, Base: Sendable, Signal: Sendable, Base.Element: Sendable, Signal.Element: Sendable {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunksOfCountOrSignalSequence<
Base: AsyncSequence,
Collected: RangeReplaceableCollection,
Signal: AsyncSequence
>: AsyncSequence, Sendable
where
Collected.Element == Base.Element,
Base: Sendable,
Signal: Sendable,
Base.Element: Sendable,
Signal.Element: Sendable
{
public typealias Element = Collected
@@ -65,23 +104,23 @@ public struct AsyncChunksOfCountOrSignalSequence<Base: AsyncSequence, Collected:
case terminal
case signal
}
/// The iterator for a `AsyncChunksOfCountOrSignalSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
typealias EitherMappedBase = AsyncMapSequence<Base, Either>
typealias EitherMappedSignal = AsyncMapSequence<Signal, Either>
typealias ChainedBase = AsyncChain2Sequence<EitherMappedBase, AsyncSyncSequence<[Either]>>
typealias Merged = AsyncMerge2Sequence<ChainedBase, EitherMappedSignal>
let count: Int?
var iterator: Merged.AsyncIterator
var terminated = false
init(iterator: Merged.AsyncIterator, count: Int?) {
self.count = count
self.iterator = iterator
}
public mutating func next() async rethrows -> Collected? {
guard !terminated else {
return nil
@@ -124,10 +163,16 @@ public struct AsyncChunksOfCountOrSignalSequence<Base: AsyncSequence, Collected:
}
public func makeAsyncIterator() -> Iterator {
return Iterator(iterator: merge(chain(base.map { Either.element($0) }, [.terminal].async), signal.map { _ in Either.signal }).makeAsyncIterator(), count: count)
return Iterator(
iterator: merge(
chain(base.map { Either.element($0) }, [.terminal].async),
signal.map { _ in Either.signal }
).makeAsyncIterator(),
count: count
)
}
}
@available(*, unavailable)
extension AsyncChunksOfCountOrSignalSequence.Iterator: Sendable { }
extension AsyncChunksOfCountOrSignalSequence.Iterator: Sendable {}
@@ -9,14 +9,20 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that creates chunks of a given `RangeReplaceableCollection` of a given count.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunks<Collected: RangeReplaceableCollection>(ofCount count: Int, into: Collected.Type) -> AsyncChunksOfCountSequence<Self, Collected> where Collected.Element == Element {
public func chunks<Collected: RangeReplaceableCollection>(
ofCount count: Int,
into: Collected.Type
) -> AsyncChunksOfCountSequence<Self, Collected> where Collected.Element == Element {
AsyncChunksOfCountSequence(self, count: count)
}
/// Creates an asynchronous sequence that creates chunks of a given count.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func chunks(ofCount count: Int) -> AsyncChunksOfCountSequence<Self, [Element]> {
chunks(ofCount: count, into: [Element].self)
@@ -24,7 +30,9 @@ extension AsyncSequence {
}
/// An `AsyncSequence` that chunks elements into `RangeReplaceableCollection` instances of at least a given count.
public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence where Collected.Element == Base.Element {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeReplaceableCollection>: AsyncSequence
where Collected.Element == Base.Element {
public typealias Element = Collected
/// The iterator for a `AsyncChunksOfCountSequence` instance.
@@ -67,10 +75,10 @@ public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeRe
}
@usableFromInline
let base : Base
let base: Base
@usableFromInline
let count : Int
let count: Int
@usableFromInline
init(_ base: Base, count: Int) {
@@ -85,8 +93,7 @@ public struct AsyncChunksOfCountSequence<Base: AsyncSequence, Collected: RangeRe
}
}
extension AsyncChunksOfCountSequence : Sendable where Base : Sendable, Base.Element : Sendable { }
extension AsyncChunksOfCountSequence.Iterator : Sendable where Base.AsyncIterator : Sendable, Base.Element : Sendable { }
@available(*, unavailable)
extension AsyncChunksOfCountSequence.Iterator: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunksOfCountSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncChunksOfCountSequence.Iterator: Sendable where Base.AsyncIterator: Sendable, Base.Element: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns a new `AsyncSequence` that iterates over every non-nil element from the
/// original `AsyncSequence`.
@@ -18,22 +19,24 @@ extension AsyncSequence {
/// - Returns: An `AsyncSequence` where the element is the unwrapped original
/// element and iterates over every non-nil element from the original
/// `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func compacted<Unwrapped>() -> AsyncCompactedSequence<Self, Unwrapped>
where Element == Unwrapped? {
where Element == Unwrapped? {
AsyncCompactedSequence(self)
}
}
/// An `AsyncSequence` that iterates over every non-nil element from the original
/// `AsyncSequence`.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncCompactedSequence<Base: AsyncSequence, Element>: AsyncSequence
where Base.Element == Element? {
where Base.Element == Element? {
@usableFromInline
let base: Base
@inlinable
init(_ base: Base) {
self.base = base
@@ -44,12 +47,12 @@ public struct AsyncCompactedSequence<Base: AsyncSequence, Element>: AsyncSequenc
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var base: Base.AsyncIterator
@inlinable
init(_ base: Base.AsyncIterator) {
self.base = base
}
@inlinable
public mutating func next() async rethrows -> Element? {
while let wrapped = try await base.next() {
@@ -66,7 +69,8 @@ public struct AsyncCompactedSequence<Base: AsyncSequence, Element>: AsyncSequenc
}
}
extension AsyncCompactedSequence: Sendable where Base: Sendable, Base.Element: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncCompactedSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncCompactedSequence.Iterator: Sendable { }
extension AsyncCompactedSequence.Iterator: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given closure.
@@ -22,13 +23,17 @@ extension AsyncSequence {
/// the next element in the receiving asynchronous sequence, which it returns.
/// - Returns: An asynchronous sequence of the initial value followed by the reduced
/// elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(_ initial: Result, _ transform: @Sendable @escaping (Result, Element) async -> Result) -> AsyncExclusiveReductionsSequence<Self, Result> {
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async -> Result
) -> AsyncExclusiveReductionsSequence<Self, Result> {
reductions(into: initial) { result, element in
result = await transform(result, element)
}
}
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given closure.
///
@@ -42,25 +47,30 @@ extension AsyncSequence {
/// previous result instead of returning a value.
/// - Returns: An asynchronous sequence of the initial value followed by the reduced
/// elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(into initial: Result, _ transform: @Sendable @escaping (inout Result, Element) async -> Void) -> AsyncExclusiveReductionsSequence<Self, Result> {
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async -> Void
) -> AsyncExclusiveReductionsSequence<Self, Result> {
AsyncExclusiveReductionsSequence(self, initial: initial, transform: transform)
}
}
/// An asynchronous sequence of applying a transform to the element of an asynchronous sequence and the
/// previously transformed result.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncExclusiveReductionsSequence<Base: AsyncSequence, Element> {
@usableFromInline
let base: Base
@usableFromInline
let initial: Element
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async -> Void
@inlinable
init(_ base: Base, initial: Element, transform: @Sendable @escaping (inout Element, Base.Element) async -> Void) {
self.base = base
@@ -69,49 +79,55 @@ public struct AsyncExclusiveReductionsSequence<Base: AsyncSequence, Element> {
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncExclusiveReductionsSequence: AsyncSequence {
/// The iterator for an `AsyncExclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.AsyncIterator
@usableFromInline
var current: Element?
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async -> Void
@inlinable
init(_ iterator: Base.AsyncIterator, initial: Element, transform: @Sendable @escaping (inout Element, Base.Element) async -> Void) {
init(
_ iterator: Base.AsyncIterator,
initial: Element,
transform: @Sendable @escaping (inout Element, Base.Element) async -> Void
) {
self.iterator = iterator
self.current = initial
self.transform = transform
}
@inlinable
public mutating func next() async rethrows -> Element? {
guard let result = current else { return nil }
guard var result = current else { return nil }
let value = try await iterator.next()
if let value = value {
var result = result
await transform(&result, value)
current = result
return result
} else {
guard let value = value else {
current = nil
return nil
}
await transform(&result, value)
current = result
return result
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), initial: initial, transform: transform)
}
}
extension AsyncExclusiveReductionsSequence: Sendable where Base: Sendable, Element: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncExclusiveReductionsSequence: Sendable where Base: Sendable, Element: Sendable {}
@available(*, unavailable)
extension AsyncExclusiveReductionsSequence.Iterator: Sendable { }
extension AsyncExclusiveReductionsSequence.Iterator: Sendable {}
@@ -1,3 +1,15 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given closure.
@@ -17,6 +29,7 @@ extension AsyncSequence {
/// result and the next element in the receiving sequence, and returns
/// the result.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async -> Element
@@ -27,14 +40,15 @@ extension AsyncSequence {
/// An asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using a given closure.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncInclusiveReductionsSequence<Base: AsyncSequence> {
@usableFromInline
let base: Base
@usableFromInline
let transform: @Sendable (Base.Element, Base.Element) async -> Base.Element
@inlinable
init(_ base: Base, transform: @Sendable @escaping (Base.Element, Base.Element) async -> Base.Element) {
self.base = base
@@ -42,10 +56,12 @@ public struct AsyncInclusiveReductionsSequence<Base: AsyncSequence> {
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncInclusiveReductionsSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
@@ -78,13 +94,15 @@ extension AsyncInclusiveReductionsSequence: AsyncSequence {
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), transform: transform)
}
}
extension AsyncInclusiveReductionsSequence: Sendable where Base: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncInclusiveReductionsSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncInclusiveReductionsSequence.Iterator: Sendable { }
extension AsyncInclusiveReductionsSequence.Iterator: Sendable {}
@@ -9,16 +9,22 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Element: AsyncSequence {
/// Concatenate an `AsyncSequence` of `AsyncSequence` elements with a separator.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func joined<Separator: AsyncSequence>(separator: Separator) -> AsyncJoinedBySeparatorSequence<Self, Separator> {
public func joined<Separator: AsyncSequence>(
separator: Separator
) -> AsyncJoinedBySeparatorSequence<Self, Separator> {
return AsyncJoinedBySeparatorSequence(self, separator: separator)
}
}
/// An `AsyncSequence` that concatenates `AsyncSequence` elements with a separator.
public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: AsyncSequence>: AsyncSequence where Base.Element: AsyncSequence, Separator.Element == Base.Element.Element {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: AsyncSequence>: AsyncSequence
where Base.Element: AsyncSequence, Separator.Element == Base.Element.Element {
public typealias Element = Base.Element.Element
public typealias AsyncIterator = Iterator
@@ -36,31 +42,31 @@ public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: Asy
@usableFromInline
func startSeparator() -> SeparatorState {
switch self {
case .initial(let separatorSequence):
return .partialAsync(separatorSequence.makeAsyncIterator(), [])
case .cached(let array):
return .partialCached(array.makeIterator(), array)
default:
fatalError("Invalid separator sequence state")
case .initial(let separatorSequence):
return .partialAsync(separatorSequence.makeAsyncIterator(), [])
case .cached(let array):
return .partialCached(array.makeIterator(), array)
default:
fatalError("Invalid separator sequence state")
}
}
@usableFromInline
func next() async rethrows -> (Element?, SeparatorState) {
switch self {
case .partialAsync(var separatorIterator, var cache):
guard let next = try await separatorIterator.next() else {
return (nil, .cached(cache))
}
cache.append(next)
return (next, .partialAsync(separatorIterator, cache))
case .partialCached(var cacheIterator, let cache):
guard let next = cacheIterator.next() else {
return (nil, .cached(cache))
}
return (next, .partialCached(cacheIterator, cache))
default:
fatalError("Invalid separator sequence state")
case .partialAsync(var separatorIterator, var cache):
guard let next = try await separatorIterator.next() else {
return (nil, .cached(cache))
}
cache.append(next)
return (next, .partialAsync(separatorIterator, cache))
case .partialCached(var cacheIterator, let cache):
guard let next = cacheIterator.next() else {
return (nil, .cached(cache))
}
return (next, .partialCached(cacheIterator, cache))
default:
fatalError("Invalid separator sequence state")
}
}
}
@@ -83,37 +89,37 @@ public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: Asy
public mutating func next() async rethrows -> Base.Element.Element? {
do {
switch state {
case .terminal:
case .terminal:
return nil
case .initial(var outerIterator, let separator):
guard let innerSequence = try await outerIterator.next() else {
state = .terminal
return nil
case .initial(var outerIterator, let separator):
guard let innerSequence = try await outerIterator.next() else {
state = .terminal
return nil
}
let innerIterator = innerSequence.makeAsyncIterator()
state = .sequence(outerIterator, innerIterator, .initial(separator))
return try await next()
case .sequence(var outerIterator, var innerIterator, let separatorState):
if let item = try await innerIterator.next() {
state = .sequence(outerIterator, innerIterator, separatorState)
return item
}
guard let nextInner = try await outerIterator.next() else {
state = .terminal
return nil
}
state = .separator(outerIterator, separatorState.startSeparator(), nextInner)
return try await next()
case .separator(let iterator, let separatorState, let nextBase):
let (itemOpt, newSepState) = try await separatorState.next()
guard let item = itemOpt else {
state = .sequence(iterator, nextBase.makeAsyncIterator(), newSepState)
return try await next()
}
state = .separator(iterator, newSepState, nextBase)
}
let innerIterator = innerSequence.makeAsyncIterator()
state = .sequence(outerIterator, innerIterator, .initial(separator))
return try await next()
case .sequence(var outerIterator, var innerIterator, let separatorState):
if let item = try await innerIterator.next() {
state = .sequence(outerIterator, innerIterator, separatorState)
return item
}
guard let nextInner = try await outerIterator.next() else {
state = .terminal
return nil
}
state = .separator(outerIterator, separatorState.startSeparator(), nextInner)
return try await next()
case .separator(let iterator, let separatorState, let nextBase):
let (itemOpt, newSepState) = try await separatorState.next()
guard let item = itemOpt else {
state = .sequence(iterator, nextBase.makeAsyncIterator(), newSepState)
return try await next()
}
state = .separator(iterator, newSepState, nextBase)
return item
}
} catch {
state = .terminal
@@ -140,8 +146,9 @@ public struct AsyncJoinedBySeparatorSequence<Base: AsyncSequence, Separator: Asy
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncJoinedBySeparatorSequence: Sendable
where Base: Sendable, Base.Element: Sendable, Base.Element.Element: Sendable, Separator: Sendable { }
where Base: Sendable, Base.Element: Sendable, Base.Element.Element: Sendable, Separator: Sendable {}
@available(*, unavailable)
extension AsyncJoinedBySeparatorSequence.Iterator: Sendable { }
extension AsyncJoinedBySeparatorSequence.Iterator: Sendable {}
@@ -9,8 +9,10 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Element: AsyncSequence {
/// Concatenate an `AsyncSequence` of `AsyncSequence` elements
/// Concatenate an `AsyncSequence` of `AsyncSequence` elements
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func joined() -> AsyncJoinedSequence<Self> {
return AsyncJoinedSequence(self)
@@ -18,6 +20,7 @@ extension AsyncSequence where Element: AsyncSequence {
}
/// An `AsyncSequence` that concatenates`AsyncSequence` elements
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncJoinedSequence<Base: AsyncSequence>: AsyncSequence where Base.Element: AsyncSequence {
public typealias Element = Base.Element.Element
@@ -32,42 +35,42 @@ public struct AsyncJoinedSequence<Base: AsyncSequence>: AsyncSequence where Base
case sequence(Base.AsyncIterator, Base.Element.AsyncIterator)
case terminal
}
@usableFromInline
var state: State
@inlinable
init(_ iterator: Base.AsyncIterator) {
state = .initial(iterator)
}
@inlinable
public mutating func next() async rethrows -> Base.Element.Element? {
do {
switch state {
case .terminal:
case .terminal:
return nil
case .initial(var outerIterator):
guard let innerSequence = try await outerIterator.next() else {
state = .terminal
return nil
case .initial(var outerIterator):
guard let innerSequence = try await outerIterator.next() else {
state = .terminal
return nil
}
let innerIterator = innerSequence.makeAsyncIterator()
}
let innerIterator = innerSequence.makeAsyncIterator()
state = .sequence(outerIterator, innerIterator)
return try await next()
case .sequence(var outerIterator, var innerIterator):
if let item = try await innerIterator.next() {
state = .sequence(outerIterator, innerIterator)
return try await next()
case .sequence(var outerIterator, var innerIterator):
if let item = try await innerIterator.next() {
state = .sequence(outerIterator, innerIterator)
return item
}
guard let nextInner = try await outerIterator.next() else {
state = .terminal
return nil
}
return item
}
state = .sequence(outerIterator, nextInner.makeAsyncIterator())
return try await next()
guard let nextInner = try await outerIterator.next() else {
state = .terminal
return nil
}
state = .sequence(outerIterator, nextInner.makeAsyncIterator())
return try await next()
}
} catch {
state = .terminal
@@ -75,23 +78,24 @@ public struct AsyncJoinedSequence<Base: AsyncSequence>: AsyncSequence where Base
}
}
}
@usableFromInline
let base: Base
@usableFromInline
init(_ base: Base) {
self.base = base
}
@inlinable
public func makeAsyncIterator() -> Iterator {
return Iterator(base.makeAsyncIterator())
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncJoinedSequence: Sendable
where Base: Sendable, Base.Element: Sendable, Base.Element.Element: Sendable { }
where Base: Sendable, Base.Element: Sendable, Base.Element.Element: Sendable {}
@available(*, unavailable)
extension AsyncJoinedSequence.Iterator: Sendable { }
extension AsyncJoinedSequence.Iterator: Sendable {}
@@ -0,0 +1,106 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.0)
@available(AsyncAlgorithms 1.2, *)
extension AsyncSequence {
/// Converts any failure into a new error.
///
/// - Parameter transform: A closure that takes the failure as a parameter and returns a new error.
/// - Returns: An asynchronous sequence that maps the error thrown into the one produced by the transform closure.
///
/// Use the ``mapError(_:)`` operator when you need to replace one error type with another.
@available(AsyncAlgorithms 1.2, *)
public func mapError<MappedError: Error>(
_ transform: @Sendable @escaping (Self.Failure) -> MappedError
) -> some AsyncSequence<Self.Element, MappedError> {
AsyncMapErrorSequence(base: self, transform: transform)
}
/// Converts any failure into a new error.
///
/// - Parameter transform: A closure that takes the failure as a parameter and returns a new error.
/// - Returns: An asynchronous sequence that maps the error thrown into the one produced by the transform closure.
///
/// Use the ``mapError(_:)`` operator when you need to replace one error type with another.
@available(AsyncAlgorithms 1.2, *)
public func mapError<MappedError: Error>(
_ transform: @Sendable @escaping (Self.Failure) -> MappedError
) -> (some AsyncSequence<Self.Element, MappedError> & Sendable) where Self: Sendable, Self.Element: Sendable {
AsyncMapErrorSequence(base: self, transform: transform)
}
}
/// An asynchronous sequence that converts any failure into a new error.
@available(AsyncAlgorithms 1.2, *)
struct AsyncMapErrorSequence<Base: AsyncSequence, MappedError: Error> {
typealias Element = Base.Element
typealias Failure = MappedError
private let base: Base
private let transform: @Sendable (Base.Failure) async -> MappedError
init(
base: Base,
transform: @Sendable @escaping (Base.Failure) async -> MappedError
) {
self.base = base
self.transform = transform
}
}
@available(AsyncAlgorithms 1.2, *)
extension AsyncMapErrorSequence: AsyncSequence {
/// The iterator that produces elements of the map sequence.
struct Iterator: AsyncIteratorProtocol {
typealias Element = Base.Element
private var base: Base.AsyncIterator
private let transform: @Sendable (Base.Failure) async -> MappedError
init(
base: Base.AsyncIterator,
transform: @Sendable @escaping (Base.Failure) async -> MappedError
) {
self.base = base
self.transform = transform
}
mutating func next() async throws(MappedError) -> Element? {
try await self.next(isolation: nil)
}
mutating func next(isolation actor: isolated (any Actor)?) async throws(MappedError) -> Element? {
do {
return try await base.next(isolation: actor)
} catch {
throw await transform(error)
}
}
}
func makeAsyncIterator() -> Iterator {
Iterator(
base: base.makeAsyncIterator(),
transform: transform
)
}
}
@available(AsyncAlgorithms 1.2, *)
extension AsyncMapErrorSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncMapErrorSequence.Iterator: Sendable {}
#endif
@@ -9,8 +9,10 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Element: Equatable {
/// Creates an asynchronous sequence that omits repeated elements.
@available(AsyncAlgorithms 1.0, *)
public func removeDuplicates() -> AsyncRemoveDuplicatesSequence<Self> {
AsyncRemoveDuplicatesSequence(self) { lhs, rhs in
lhs == rhs
@@ -18,19 +20,27 @@ extension AsyncSequence where Element: Equatable {
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that omits repeated elements by testing them with a predicate.
public func removeDuplicates(by predicate: @escaping @Sendable (Element, Element) async -> Bool) -> AsyncRemoveDuplicatesSequence<Self> {
@available(AsyncAlgorithms 1.0, *)
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async -> Bool
) -> AsyncRemoveDuplicatesSequence<Self> {
return AsyncRemoveDuplicatesSequence(self, predicate: predicate)
}
/// Creates an asynchronous sequence that omits repeated elements by testing them with an error-throwing predicate.
public func removeDuplicates(by predicate: @escaping @Sendable (Element, Element) async throws -> Bool) -> AsyncThrowingRemoveDuplicatesSequence<Self> {
@available(AsyncAlgorithms 1.0, *)
public func removeDuplicates(
by predicate: @escaping @Sendable (Element, Element) async throws -> Bool
) -> AsyncThrowingRemoveDuplicatesSequence<Self> {
return AsyncThrowingRemoveDuplicatesSequence(self, predicate: predicate)
}
}
/// An asynchronous sequence that omits repeated elements by testing them with a predicate.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
@@ -73,7 +83,7 @@ public struct AsyncRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence
@usableFromInline
let predicate: @Sendable (Element, Element) async -> Bool
init(_ base: Base, predicate: @escaping @Sendable (Element, Element) async -> Bool) {
self.base = base
self.predicate = predicate
@@ -86,9 +96,10 @@ public struct AsyncRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence
}
/// An asynchronous sequence that omits repeated elements by testing them with an error-throwing predicate.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncThrowingRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncThrowingRemoveDuplicatesSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
@@ -128,7 +139,7 @@ public struct AsyncThrowingRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncS
@usableFromInline
let predicate: @Sendable (Element, Element) async throws -> Bool
init(_ base: Base, predicate: @escaping @Sendable (Element, Element) async throws -> Bool) {
self.base = base
self.predicate = predicate
@@ -140,12 +151,13 @@ public struct AsyncThrowingRemoveDuplicatesSequence<Base: AsyncSequence>: AsyncS
}
}
extension AsyncRemoveDuplicatesSequence: Sendable where Base: Sendable, Base.Element: Sendable { }
extension AsyncThrowingRemoveDuplicatesSequence: Sendable where Base: Sendable, Base.Element: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncRemoveDuplicatesSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingRemoveDuplicatesSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
extension AsyncRemoveDuplicatesSequence.Iterator: Sendable { }
extension AsyncRemoveDuplicatesSequence.Iterator: Sendable {}
@available(*, unavailable)
extension AsyncThrowingRemoveDuplicatesSequence.Iterator: Sendable { }
extension AsyncThrowingRemoveDuplicatesSequence.Iterator: Sendable {}
@@ -0,0 +1,726 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2022 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.2)
import Synchronization
import DequeModule
@available(AsyncAlgorithms 1.1, *)
extension AsyncSequence
where Element: Sendable, Self: SendableMetatype, AsyncIterator: SendableMetatype {
/// Creates a shared async sequence that allows multiple concurrent iterations over a single source.
///
/// The `share` method transforms an async sequence into a shareable sequence that can be safely
/// iterated by multiple concurrent tasks. This is useful when you want to broadcast elements from
/// a single source to multiple consumers without duplicating work or creating separate iterations.
///
/// Each element from the source sequence is delivered to all active iterators.
/// Elements are buffered according to the specified buffering policy to handle timing differences
/// between consumers.
///
/// The base sequence is iterated in it's own task to ensure that cancellation is not polluted from
/// one side of iteration to another.
///
/// ## Example Usage
///
/// ```swift
/// let numbers = [1, 2, 3, 4, 5].async.map {
/// try? await Task.sleep(for: .seconds(1))
/// return $0
/// }
///
/// let shared = numbers.share()
///
/// // Multiple tasks can iterate concurrently
/// let consumer1 = Task {
/// for await value in shared {
/// print("Consumer 1: \(value)")
/// }
/// }
///
/// let consumer2 = Task {
/// for await value in shared {
/// print("Consumer 2: \(value)")
/// }
/// }
///
/// await consumer1.value
/// await consumer2.value
/// ```
///
/// - Parameter bufferingPolicy: The policy controlling how elements are enqueued to the shared buffer. Defaults to `.bounded(1)`.
/// - `.bounded(n)`: Limits the buffer to `n` elements, applying backpressure to the source when that limit is reached
/// - `.bufferingOldest(n)`: Keeps the oldest `n` elements, discarding newer ones when full
/// - `.bufferingNewest(n)`: Keeps the newest `n` elements, discarding older ones when full
/// - `.unbounded`: Allows unlimited buffering (use with caution)
///
/// - Returns: A sendable async sequence that can be safely shared across multiple concurrent tasks.
///
public func share(
bufferingPolicy: AsyncBufferSequencePolicy = .bounded(1)
) -> some AsyncSequence<Element, Failure> & Sendable {
// The iterator is transferred to the isolation of the iterating task
// this has to be done "unsafely" since we cannot annotate the transfer
// however since iterating an AsyncSequence types twice has been defined
// as invalid and one creation of the iterator is virtually a consuming
// operation so this is safe at runtime.
// The general principal of `.share()` is to provide a mecahnism for non-
// shared AsyncSequence types to be shared. The parlance for those is
// that the base AsyncSequence type is not Sendable. If the iterator
// is not marked as `nonisolated(unsafe)` the compiler will claim that
// the value is "Capture of 'iterator' with non-Sendable type 'Self.AsyncIterator' in a '@Sendable' closure;"
// Since the closure returns a disconnected non-sendable value there is no
// distinct problem here and the compiler just needs to be informed
// that the diagnostic is overly pessimistic.
nonisolated(unsafe) let iterator = makeAsyncIterator()
return AsyncShareSequence<Self>(
{
iterator
},
bufferingPolicy: bufferingPolicy
)
}
}
// An async sequence that enables safe concurrent sharing of a single source sequence.
//
// `AsyncShareSequence` wraps a base async sequence and allows multiple concurrent iterators
// to consume elements from the same source. It handles all the complexity of coordinating
// between multiple consumers, buffering elements, and managing the lifecycle of the underlying
// iteration.
//
// ## Key Features
//
// **Single Source Iteration**: The base sequence's iterator is created and consumed only once
// **Concurrent Safe**: Multiple tasks can safely iterate simultaneously
// **Configurable Buffering**: Supports various buffering strategies for different use cases
// **Automatic Cleanup**: Properly manages resources and cancellation across all consumers
//
// ## Internal Architecture
//
// The implementation uses several key components:
// `Side`: Represents a single consumer's iteration state
// `Iteration`: Coordinates all consumers and manages the shared buffer
// `Extent`: Manages the overall lifecycle and cleanup
//
// This type is typically not used directly; instead, use the `share()` method on any
// async sequence that meets the sendability requirements.
@available(AsyncAlgorithms 1.1, *)
struct AsyncShareSequence<Base: AsyncSequence>: Sendable
where Base.Element: Sendable, Base: SendableMetatype, Base.AsyncIterator: SendableMetatype {
// Represents a single consumer's connection to the shared sequence.
//
// Each iterator of the shared sequence creates its own `Side` instance, which tracks
// that consumer's position in the shared buffer and manages its continuation for
// async iteration. The `Side` automatically registers itself with the central
// `Iteration` coordinator and cleans up when deallocated.
//
// ## Lifecycle
//
// **Creation**: Automatically registers with the iteration coordinator
// **Usage**: Tracks buffer position and manages async continuations
// **Cleanup**: Automatically unregisters and cancels pending operations on deinit
final class Side {
// Tracks the state of a single consumer's iteration.
//
// - `continuation`: The continuation waiting for the next element (nil if not waiting)
// - `position`: The consumer's current position in the shared buffer
struct State {
var continuation: UnsafeContinuation<Result<Element?, Failure>, Never>?
var position = 0
// Creates a new state with the position adjusted by the given offset.
//
// This is used when the shared buffer is trimmed to maintain correct
// relative positioning for this consumer.
//
// - Parameter adjustment: The number of positions to subtract from the current position
// - Returns: A new `State` with the adjusted position
func offset(_ adjustment: Int) -> State {
State(continuation: continuation, position: position - adjustment)
}
}
let iteration: Iteration
let id: Int
init(_ iteration: Iteration) {
self.iteration = iteration
id = iteration.registerSide()
}
deinit {
iteration.unregisterSide(id)
}
func next(isolation actor: isolated (any Actor)?) async throws(Failure) -> Element? {
try await iteration.next(isolation: actor, id: id)
}
}
// The central coordinator that manages the shared iteration state.
//
// `Iteration` is responsible for:
// Managing the single background task that consumes the source sequence
// Coordinating between multiple consumer sides
// Buffering elements according to the specified policy
// Handling backpressure and flow control
// Managing cancellation and cleanup
//
// ## Thread Safety
//
// All operations are synchronized using a `Mutex` to ensure thread-safe access
// to the shared state across multiple concurrent consumers.
final class Iteration: Sendable {
// Represents the state of the background task that consumes the source sequence.
//
// The iteration task goes through several states during its lifecycle:
// `pending`: Initial state, holds the factory to create the iterator
// `starting`: Transitional state while the task is being created
// `running`: Active state with a running background task
// `cancelled`: Terminal state when the iteration has been cancelled
enum IteratingTask {
case pending(@Sendable () -> sending Base.AsyncIterator)
case starting
case running(Task<Void, Never>)
case cancelled
var isStarting: Bool {
switch self {
case .starting: true
default: false
}
}
func cancel() {
switch self {
case .running(let task):
task.cancel()
default:
break
}
}
}
// The complete shared state for coordinating all aspects of the shared iteration.
//
// This state is protected by a mutex and contains all the information needed
// to coordinate between multiple consumers, manage buffering, and control
// the background iteration task.
struct State: Sendable {
// Defines how elements are stored and potentially discarded in the shared buffer.
//
// `unbounded`: Store all elements without limit (may cause memory growth)
// `bufferingOldest(Int)`: Keep only the oldest N elements, ignore newer ones when full
// `bufferingNewest(Int)`: Keep only the newest N elements, discard older ones when full
enum StoragePolicy: Sendable {
case unbounded
case bufferingOldest(Int)
case bufferingNewest(Int)
}
var generation = 0
var sides = [Int: Side.State]()
var iteratingTask: IteratingTask
private(set) var buffer = Deque<Element>()
private(set) var finished = false
private(set) var failure: Failure?
var cancelled = false
var limit: UnsafeContinuation<Bool, Never>?
var demand: UnsafeContinuation<Void, Never>?
let storagePolicy: StoragePolicy
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
self.iteratingTask = .pending(iteratorFactory)
switch bufferingPolicy.policy {
case .bounded: self.storagePolicy = .unbounded
case .bufferingOldest(let bound): self.storagePolicy = .bufferingOldest(bound)
case .bufferingNewest(let bound): self.storagePolicy = .bufferingNewest(bound)
case .unbounded: self.storagePolicy = .unbounded
}
}
// Removes elements from the front of the buffer that all consumers have already processed.
//
// This method finds the minimum position across all active consumers and removes
// that many elements from the front of the buffer. It then adjusts all consumer
// positions to account for the removed elements, maintaining their relative positions.
//
// This optimization prevents the buffer from growing indefinitely when all consumers
// are keeping pace with each other.
mutating func trimBuffer() {
if let minimumIndex = sides.values.map({ $0.position }).min(), minimumIndex > 0 {
buffer.removeFirst(minimumIndex)
sides = sides.mapValues {
$0.offset(minimumIndex)
}
}
}
// Private state machine transitions for the emission of a given value.
//
// This method ensures the continuations are properly consumed when emitting values
// and returns those continuations for resumption.
private mutating func _emit<T>(
_ value: T,
limit: Int
) -> (T, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) {
let belowLimit = buffer.count < limit || limit == 0
defer {
if belowLimit {
self.limit = nil
}
demand = nil
}
guard case .cancelled = iteratingTask else {
return (value, belowLimit ? self.limit : nil, demand, false)
}
return (value, belowLimit ? self.limit : nil, demand, true)
}
// Internal state machine transitions for the emission of a given value.
//
// This method ensures the continuations are properly consumed when emitting values
// and returns those continuations for resumption.
//
// If no limit is specified it interprets that as an unbounded limit.
mutating func emit<T>(
_ value: T,
limit: Int?
) -> (T, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) {
return _emit(value, limit: limit ?? .max)
}
// Adds an element to the buffer according to the configured storage policy.
//
// The behavior depends on the storage policy:
// **Unbounded**: Always appends the element
// **Buffering Oldest**: Appends only if under the limit, otherwise ignores the element
// **Buffering Newest**: Appends if under the limit, otherwise removes the oldest and appends
//
// - Parameter element: The element to add to the buffer
mutating func enqueue(_ element: Element) {
let count = buffer.count
switch storagePolicy {
case .unbounded:
buffer.append(element)
case .bufferingOldest(let limit):
if count < limit {
buffer.append(element)
}
case .bufferingNewest(let limit):
if count < limit {
buffer.append(element)
} else if count > 0 {
buffer.removeFirst()
buffer.append(element)
}
}
}
mutating func finish() {
finished = true
}
mutating func fail(_ error: Failure) {
finished = true
failure = error
}
}
let state: Mutex<State>
let limit: Int?
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
state = Mutex(State(iteratorFactory, bufferingPolicy: bufferingPolicy))
switch bufferingPolicy.policy {
case .bounded(let limit):
self.limit = limit
default:
self.limit = nil
}
}
func cancel() {
let (task, limitContinuation, demand, cancelled) = state.withLock {
state -> (IteratingTask?, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) in
guard state.sides.count == 0 else {
state.cancelled = true
return state.emit(nil, limit: limit)
}
defer {
state.iteratingTask = .cancelled
state.cancelled = true
}
return state.emit(state.iteratingTask, limit: limit)
}
task?.cancel()
limitContinuation?.resume(returning: cancelled)
demand?.resume()
}
func registerSide() -> Int {
state.withLock { state in
defer { state.generation += 1 }
state.sides[state.generation] = Side.State()
return state.generation
}
}
func unregisterSide(_ id: Int) {
let (side, continuation, cancelled, iteratingTaskToCancel) = state.withLock {
state -> (Side.State?, UnsafeContinuation<Bool, Never>?, Bool, IteratingTask?) in
let side = state.sides.removeValue(forKey: id)
state.trimBuffer()
let cancelRequested = state.sides.count == 0 && state.cancelled
guard let limit, state.buffer.count < limit else {
guard case .cancelled = state.iteratingTask else {
defer {
if cancelRequested {
state.iteratingTask = .cancelled
}
}
return (side, nil, false, cancelRequested ? state.iteratingTask : nil)
}
return (side, nil, true, nil)
}
defer { state.limit = nil }
guard case .cancelled = state.iteratingTask else {
defer {
if cancelRequested {
state.iteratingTask = .cancelled
}
}
return (side, state.limit, false, cancelRequested ? state.iteratingTask : nil)
}
return (side, state.limit, true, nil)
}
if let continuation {
continuation.resume(returning: cancelled)
}
if let side {
side.continuation?.resume(returning: .success(nil))
}
if let iteratingTaskToCancel {
iteratingTaskToCancel.cancel()
}
}
func iterate() async -> Bool {
if let limit, limit > 0 {
let cancelled = await withUnsafeContinuation { (continuation: UnsafeContinuation<Bool, Never>) in
let (resume, cancelled) = state.withLock { state -> (UnsafeContinuation<Bool, Never>?, Bool) in
guard state.buffer.count >= limit else {
assert(state.limit == nil)
guard case .cancelled = state.iteratingTask else {
return (continuation, false)
}
return (continuation, true)
}
state.limit = continuation
guard case .cancelled = state.iteratingTask else {
return (nil, false)
}
return (nil, true)
}
if let resume {
resume.resume(returning: cancelled)
}
}
if cancelled {
return false
}
}
// await a demand
await withUnsafeContinuation { (continuation: UnsafeContinuation<Void, Never>) in
let hasPendingDemand = state.withLock { state in
for (_, side) in state.sides {
if side.continuation != nil {
return true
}
}
state.demand = continuation
return false
}
if hasPendingDemand {
continuation.resume()
}
}
return state.withLock { state in
switch state.iteratingTask {
case .cancelled:
return false
default:
return true
}
}
}
func cancel(id: Int) {
unregisterSide(id) // doubly unregistering is idempotent but has a side effect of emitting nil if present
}
struct Resumption {
let continuation: UnsafeContinuation<Result<Element?, Failure>, Never>
let result: Result<Element?, Failure>
func resume() {
continuation.resume(returning: result)
}
}
func emit(_ result: Result<Element?, Failure>) {
let (resumptions, limitContinuation, demandContinuation, cancelled) = state.withLock {
state -> ([Resumption], UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool) in
var resumptions = [Resumption]()
switch result {
case .success(let element):
if let element {
state.enqueue(element)
} else {
state.finish()
}
case .failure(let failure):
state.fail(failure)
}
for (id, side) in state.sides {
if let continuation = side.continuation {
if side.position < state.buffer.count {
resumptions.append(Resumption(continuation: continuation, result: .success(state.buffer[side.position])))
state.sides[id]?.position += 1
state.sides[id]?.continuation = nil
} else if state.finished {
state.sides[id]?.continuation = nil
if let failure = state.failure {
resumptions.append(Resumption(continuation: continuation, result: .failure(failure)))
} else {
resumptions.append(Resumption(continuation: continuation, result: .success(nil)))
}
}
}
}
state.trimBuffer()
return state.emit(resumptions, limit: limit)
}
if let limitContinuation {
limitContinuation.resume(returning: cancelled)
}
if let demandContinuation {
demandContinuation.resume()
}
for resumption in resumptions {
resumption.resume()
}
}
private func nextIteration(
_ id: Int
) async -> Result<AsyncShareSequence<Base>.Element?, AsyncShareSequence<Base>.Failure> {
return await withTaskCancellationHandler {
await withUnsafeContinuation { continuation in
let (res, limitContinuation, demandContinuation, cancelled) = state.withLock {
state -> (
Result<Element?, Failure>?, UnsafeContinuation<Bool, Never>?, UnsafeContinuation<Void, Never>?, Bool
) in
guard let side = state.sides[id] else {
return state.emit(.success(nil), limit: limit)
}
if side.position < state.buffer.count {
// There's an element available at this position
let element = state.buffer[side.position]
state.sides[id]?.position += 1
state.trimBuffer()
return state.emit(.success(element), limit: limit)
} else {
// Position is beyond the buffer
if let failure = state.failure {
return state.emit(.failure(failure), limit: limit)
} else if state.finished {
return state.emit(.success(nil), limit: limit)
} else {
state.sides[id]?.continuation = continuation
return state.emit(nil, limit: limit)
}
}
}
if let limitContinuation {
limitContinuation.resume(returning: cancelled)
}
if let demandContinuation {
demandContinuation.resume()
}
if let res {
continuation.resume(returning: res)
}
}
} onCancel: {
cancel(id: id)
}
}
private func iterationLoop(factory: @Sendable () -> sending Base.AsyncIterator) async {
var iterator = factory()
do {
while await iterate() {
if let element = try await iterator.next() {
emit(.success(element))
} else {
emit(.success(nil))
}
}
} catch {
emit(.failure(error as! Failure))
}
}
func next(isolation actor: isolated (any Actor)?, id: Int) async throws(Failure) -> Element? {
let (factory, cancelled) = state.withLock { state -> ((@Sendable () -> sending Base.AsyncIterator)?, Bool) in
switch state.iteratingTask {
case .pending(let factory):
state.iteratingTask = .starting
return (factory, false)
case .cancelled:
return (nil, true)
default:
return (nil, false)
}
}
if cancelled { return nil }
if let factory {
let task: Task<Void, Never>
// for the fancy dance of availability and canImport see the comment on the next check for details
#if swift(>=6.2)
if #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) {
task = Task(name: "Share Iteration") { [factory, self] in
await iterationLoop(factory: factory)
}
} else {
task = Task.detached(name: "Share Iteration") { [factory, self] in
await iterationLoop(factory: factory)
}
}
#else
task = Task.detached { [factory, self] in
await iterationLoop(factory: factory)
}
#endif
// Known Issue: there is a very small race where the task may not get a priority escalation during startup
// this unfortuantely cannot be avoided since the task should ideally not be formed within the critical
// region of the state. Since that could lead to potential deadlocks in low-core-count systems.
// That window is relatively small and can be revisited if a suitable proof of safe behavior can be
// determined.
state.withLock { state in
precondition(state.iteratingTask.isStarting)
state.iteratingTask = .running(task)
}
}
// withTaskPriorityEscalationHandler is only available for the '26 releases and the 6.2 version of
// the _Concurrency library. This menas for Darwin based OSes we have to have a fallback at runtime,
// and for non-darwin OSes we need to verify against the ability to import that version.
// Using this priority escalation means that the base task can avoid being detached.
//
// This is disabled for now until the 9999 availability is removed from `withTaskPriorityEscalationHandler`
#if false // TODO: remove when this is resolved
guard #available(macOS 26.0, iOS 26.0, tvOS 26.0, visionOS 26.0, *) else {
return try await nextIteration(id).get()
}
return try await withTaskPriorityEscalationHandler {
return await nextIteration(id)
} onPriorityEscalated: { old, new in
let task = state.withLock { state -> Task<Void, Never>? in
switch state.iteratingTask {
case .running(let task):
return task
default:
return nil
}
}
task?.escalatePriority(to: new)
}.get()
#else
return try await nextIteration(id).get()
#endif
}
}
// Manages the lifecycle of the shared iteration.
//
// `Extent` serves as the ownership boundary for the shared sequence. When the
// `AsyncShareSequence` itself is deallocated, the `Extent` ensures that the
// background iteration task is properly cancelled and all resources are cleaned up.
//
// This design allows multiple iterators to safely reference the same underlying
// iteration coordinator while ensuring proper cleanup when the shared sequence
// is no longer needed.
final class Extent: Sendable {
let iteration: Iteration
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
iteration = Iteration(iteratorFactory, bufferingPolicy: bufferingPolicy)
}
deinit {
iteration.cancel()
}
}
let extent: Extent
init(
_ iteratorFactory: @escaping @Sendable () -> sending Base.AsyncIterator,
bufferingPolicy: AsyncBufferSequencePolicy
) {
extent = Extent(iteratorFactory, bufferingPolicy: bufferingPolicy)
}
}
@available(AsyncAlgorithms 1.1, *)
extension AsyncShareSequence: AsyncSequence {
typealias Element = Base.Element
typealias Failure = Base.Failure
struct Iterator: AsyncIteratorProtocol {
let side: Side
init(_ iteration: Iteration) {
side = Side(iteration)
}
mutating func next() async rethrows -> Element? {
try await side.next(isolation: nil)
}
mutating func next(isolation actor: isolated (any Actor)?) async throws(Failure) -> Element? {
try await side.next(isolation: actor)
}
}
func makeAsyncIterator() -> Iterator {
Iterator(extent.iteration)
}
}
#endif
+14 -11
View File
@@ -9,10 +9,12 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension Sequence {
/// An asynchronous sequence containing the same elements as this sequence,
/// but on which operations, such as `map` and `filter`, are
/// implemented asynchronously.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public var async: AsyncSyncSequence<Self> {
AsyncSyncSequence(self)
@@ -27,46 +29,47 @@ extension Sequence {
///
/// This functions similarly to `LazySequence` by accessing elements sequentially
/// in the iterator's `next()` method.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncSyncSequence<Base: Sequence>: AsyncSequence {
public typealias Element = Base.Element
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.Iterator?
@usableFromInline
init(_ iterator: Base.Iterator) {
self.iterator = iterator
}
@inlinable
public mutating func next() async -> Base.Element? {
if !Task.isCancelled, let value = iterator?.next() {
return value
} else {
guard !Task.isCancelled, let value = iterator?.next() else {
iterator = nil
return nil
}
return value
}
}
@usableFromInline
let base: Base
@usableFromInline
init(_ base: Base) {
self.base = base
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeIterator())
}
}
extension AsyncSyncSequence: Sendable where Base: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncSyncSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncSyncSequence.Iterator: Sendable { }
extension AsyncSyncSequence.Iterator: Sendable {}
@@ -9,35 +9,49 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle<C: Clock, Reduced>(for interval: C.Instant.Duration, clock: C, reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced) -> _AsyncThrottleSequence<Self, C, Reduced> {
_AsyncThrottleSequence(self, interval: interval, clock: clock, reducing: reducing)
public func _throttle<C: Clock, Reduced>(
for interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> _AsyncThrottleSequence<Self, C, Reduced> {
_AsyncThrottleSequence(self, interval: interval, clock: clock, reducing: reducing)
}
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle<Reduced>(for interval: Duration, reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced) -> _AsyncThrottleSequence<Self, ContinuousClock, Reduced> {
_throttle(for: interval, clock: .continuous, reducing: reducing)
public func _throttle<Reduced>(
for interval: Duration,
reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
) -> _AsyncThrottleSequence<Self, ContinuousClock, Reduced> {
_throttle(for: interval, clock: .continuous, reducing: reducing)
}
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle<C: Clock>(for interval: C.Instant.Duration, clock: C, latest: Bool = true) -> _AsyncThrottleSequence<Self, C, Element> {
_throttle(for: interval, clock: clock) { previous, element in
if latest {
return element
} else {
public func _throttle<C: Clock>(
for interval: C.Instant.Duration,
clock: C,
latest: Bool = true
) -> _AsyncThrottleSequence<Self, C, Element> {
_throttle(for: interval, clock: clock) { previous, element in
guard latest else {
return previous ?? element
}
return element
}
}
/// Create a rate-limited `AsyncSequence` by emitting values at most every specified interval.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func _throttle(for interval: Duration, latest: Bool = true) -> _AsyncThrottleSequence<Self, ContinuousClock, Element> {
_throttle(for: interval, clock: .continuous, latest: latest)
public func _throttle(
for interval: Duration,
latest: Bool = true
) -> _AsyncThrottleSequence<Self, ContinuousClock, Element> {
_throttle(for: interval, clock: .continuous, latest: latest)
}
}
@@ -48,8 +62,13 @@ public struct _AsyncThrottleSequence<Base: AsyncSequence, C: Clock, Reduced> {
let interval: C.Instant.Duration
let clock: C
let reducing: @Sendable (Reduced?, Base.Element) async -> Reduced
init(_ base: Base, interval: C.Instant.Duration, clock: C, reducing: @Sendable @escaping (Reduced?, Base.Element) async -> Reduced) {
init(
_ base: Base,
interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Base.Element) async -> Reduced
) {
self.base = base
self.interval = interval
self.clock = clock
@@ -60,7 +79,7 @@ public struct _AsyncThrottleSequence<Base: AsyncSequence, C: Clock, Reduced> {
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension _AsyncThrottleSequence: AsyncSequence {
public typealias Element = Reduced
/// The iterator for an `AsyncThrottleSequence` instance.
public struct Iterator: AsyncIteratorProtocol {
var base: Base.AsyncIterator
@@ -68,14 +87,19 @@ extension _AsyncThrottleSequence: AsyncSequence {
let interval: C.Instant.Duration
let clock: C
let reducing: @Sendable (Reduced?, Base.Element) async -> Reduced
init(_ base: Base.AsyncIterator, interval: C.Instant.Duration, clock: C, reducing: @Sendable @escaping (Reduced?, Base.Element) async -> Reduced) {
init(
_ base: Base.AsyncIterator,
interval: C.Instant.Duration,
clock: C,
reducing: @Sendable @escaping (Reduced?, Base.Element) async -> Reduced
) {
self.base = base
self.interval = interval
self.clock = clock
self.reducing = reducing
}
public mutating func next() async rethrows -> Reduced? {
var reduced: Reduced?
let start = last ?? clock.now
@@ -103,14 +127,14 @@ extension _AsyncThrottleSequence: AsyncSequence {
} while true
}
}
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), interval: interval, clock: clock, reducing: reducing)
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension _AsyncThrottleSequence: Sendable where Base: Sendable, Element: Sendable { }
extension _AsyncThrottleSequence: Sendable where Base: Sendable, Element: Sendable {}
@available(*, unavailable)
extension _AsyncThrottleSequence.Iterator: Sendable { }
extension _AsyncThrottleSequence.Iterator: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given error-throwing closure.
@@ -23,13 +24,17 @@ extension AsyncSequence {
/// the result. If the closure throws an error, the sequence throws.
/// - Returns: An asynchronous sequence of the initial value followed by the reduced
/// elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(_ initial: Result, _ transform: @Sendable @escaping (Result, Element) async throws -> Result) -> AsyncThrowingExclusiveReductionsSequence<Self, Result> {
public func reductions<Result>(
_ initial: Result,
_ transform: @Sendable @escaping (Result, Element) async throws -> Result
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result> {
reductions(into: initial) { result, element in
result = try await transform(result, element)
}
}
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given error-throwing closure.
///
@@ -44,81 +49,96 @@ extension AsyncSequence {
/// error, the sequence throws.
/// - Returns: An asynchronous sequence of the initial value followed by the reduced
/// elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions<Result>(into initial: Result, _ transform: @Sendable @escaping (inout Result, Element) async throws -> Void) -> AsyncThrowingExclusiveReductionsSequence<Self, Result> {
public func reductions<Result>(
into initial: Result,
_ transform: @Sendable @escaping (inout Result, Element) async throws -> Void
) -> AsyncThrowingExclusiveReductionsSequence<Self, Result> {
AsyncThrowingExclusiveReductionsSequence(self, initial: initial, transform: transform)
}
}
/// An asynchronous sequence of applying an error-throwing transform to the element of
/// an asynchronous sequence and the previously transformed result.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncThrowingExclusiveReductionsSequence<Base: AsyncSequence, Element> {
@usableFromInline
let base: Base
@usableFromInline
let initial: Element
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async throws -> Void
@inlinable
init(_ base: Base, initial: Element, transform: @Sendable @escaping (inout Element, Base.Element) async throws -> Void) {
init(
_ base: Base,
initial: Element,
transform: @Sendable @escaping (inout Element, Base.Element) async throws -> Void
) {
self.base = base
self.initial = initial
self.transform = transform
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingExclusiveReductionsSequence: AsyncSequence {
/// The iterator for an `AsyncThrowingExclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
var iterator: Base.AsyncIterator
@usableFromInline
var current: Element?
@usableFromInline
let transform: @Sendable (inout Element, Base.Element) async throws -> Void
@inlinable
init(_ iterator: Base.AsyncIterator, initial: Element, transform: @Sendable @escaping (inout Element, Base.Element) async throws -> Void) {
init(
_ iterator: Base.AsyncIterator,
initial: Element,
transform: @Sendable @escaping (inout Element, Base.Element) async throws -> Void
) {
self.iterator = iterator
self.current = initial
self.transform = transform
}
@inlinable
public mutating func next() async throws -> Element? {
guard let result = current else { return nil }
guard var result = current else { return nil }
let value = try await iterator.next()
if let value = value {
var result = result
do {
try await transform(&result, value)
current = result
return result
} catch {
current = nil
throw error
}
} else {
guard let value = value else {
current = nil
return nil
}
do {
try await transform(&result, value)
current = result
return result
} catch {
current = nil
throw error
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), initial: initial, transform: transform)
}
}
extension AsyncThrowingExclusiveReductionsSequence: Sendable where Base: Sendable, Element: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingExclusiveReductionsSequence: Sendable where Base: Sendable, Element: Sendable {}
@available(*, unavailable)
extension AsyncThrowingExclusiveReductionsSequence.Iterator: Sendable { }
extension AsyncThrowingExclusiveReductionsSequence.Iterator: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns an asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using the given error-throwing closure.
@@ -27,6 +28,7 @@ extension AsyncSequence {
/// result and the next element in the receiving sequence. If the closure
/// throws an error, the sequence throws.
/// - Returns: An asynchronous sequence of the reduced elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func reductions(
_ transform: @Sendable @escaping (Element, Element) async throws -> Element
@@ -37,14 +39,15 @@ extension AsyncSequence {
/// An asynchronous sequence containing the accumulated results of combining the
/// elements of the asynchronous sequence using a given error-throwing closure.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct AsyncThrowingInclusiveReductionsSequence<Base: AsyncSequence> {
@usableFromInline
let base: Base
@usableFromInline
let transform: @Sendable (Base.Element, Base.Element) async throws -> Base.Element
@inlinable
init(_ base: Base, transform: @Sendable @escaping (Base.Element, Base.Element) async throws -> Base.Element) {
self.base = base
@@ -52,10 +55,12 @@ public struct AsyncThrowingInclusiveReductionsSequence<Base: AsyncSequence> {
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInclusiveReductionsSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncThrowingInclusiveReductionsSequence` instance.
@available(AsyncAlgorithms 1.0, *)
@frozen
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
@@ -93,13 +98,15 @@ extension AsyncThrowingInclusiveReductionsSequence: AsyncSequence {
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(base.makeAsyncIterator(), transform: transform)
}
}
extension AsyncThrowingInclusiveReductionsSequence: Sendable where Base: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInclusiveReductionsSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncThrowingInclusiveReductionsSequence.Iterator: Sendable { }
extension AsyncThrowingInclusiveReductionsSequence.Iterator: Sendable {}
@@ -64,7 +64,11 @@ public struct AsyncTimerSequence<C: Clock>: AsyncSequence {
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncTimerSequence {
/// Create an `AsyncTimerSequence` with a given repeating interval.
public static func repeating(every interval: C.Instant.Duration, tolerance: C.Instant.Duration? = nil, clock: C) -> AsyncTimerSequence<C> {
public static func repeating(
every interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
) -> AsyncTimerSequence<C> {
return AsyncTimerSequence(interval: interval, tolerance: tolerance, clock: clock)
}
}
@@ -72,13 +76,16 @@ extension AsyncTimerSequence {
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncTimerSequence where C == SuspendingClock {
/// Create an `AsyncTimerSequence` with a given repeating interval.
public static func repeating(every interval: Duration, tolerance: Duration? = nil) -> AsyncTimerSequence<SuspendingClock> {
public static func repeating(
every interval: Duration,
tolerance: Duration? = nil
) -> AsyncTimerSequence<SuspendingClock> {
return AsyncTimerSequence(interval: interval, tolerance: tolerance, clock: SuspendingClock())
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncTimerSequence: Sendable { }
extension AsyncTimerSequence: Sendable {}
@available(*, unavailable)
extension AsyncTimerSequence.Iterator: Sendable { }
extension AsyncTimerSequence.Iterator: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence where Self: Sendable {
/// Creates an asynchronous sequence that buffers elements.
///
@@ -20,6 +21,7 @@ extension AsyncSequence where Self: Sendable {
///
/// - Parameter policy: A policy that drives the behaviour of the ``AsyncBufferSequence``
/// - Returns: An asynchronous sequence that buffers elements up to a given limit.
@available(AsyncAlgorithms 1.0, *)
public func buffer(
policy: AsyncBufferSequencePolicy
) -> AsyncBufferSequence<Self> {
@@ -28,6 +30,7 @@ extension AsyncSequence where Self: Sendable {
}
/// A policy dictating the buffering behaviour of an ``AsyncBufferSequence``
@available(AsyncAlgorithms 1.0, *)
public struct AsyncBufferSequencePolicy: Sendable {
enum _Policy {
case bounded(Int)
@@ -69,7 +72,16 @@ public struct AsyncBufferSequencePolicy: Sendable {
}
/// An `AsyncSequence` that buffers elements in regard to a policy.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncBufferSequence<Base: AsyncSequence & Sendable>: AsyncSequence {
// Internal implementation note:
// This type origianlly had no requirement that the element is actually Sendable. However,
// that is technically an implementation detail hole in the safety of the system, it needs
// to specify that the element is actually Sendable since the draining mechanism passes
// through the isolation that is in nature sending but cannot be marked as such for the
// isolated next method.
// In practice the users of this type are safe from isolation crossing since the Element
// is as sendable as it is required by the base sequences the buffer is constructed from.
enum StorageType {
case transparent(Base.AsyncIterator)
case bounded(storage: BoundedBufferStorage<Base>)
@@ -93,16 +105,16 @@ public struct AsyncBufferSequence<Base: AsyncSequence & Sendable>: AsyncSequence
public func makeAsyncIterator() -> Iterator {
let storageType: StorageType
switch self.policy.policy {
case .bounded(...0), .bufferingNewest(...0), .bufferingOldest(...0):
storageType = .transparent(self.base.makeAsyncIterator())
case .bounded(let limit):
storageType = .bounded(storage: BoundedBufferStorage(base: self.base, limit: limit))
case .unbounded:
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .unlimited))
case .bufferingNewest(let limit):
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .bufferingNewest(limit)))
case .bufferingOldest(let limit):
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .bufferingOldest(limit)))
case .bounded(...0), .bufferingNewest(...0), .bufferingOldest(...0):
storageType = .transparent(self.base.makeAsyncIterator())
case .bounded(let limit):
storageType = .bounded(storage: BoundedBufferStorage(base: self.base, limit: limit))
case .unbounded:
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .unlimited))
case .bufferingNewest(let limit):
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .bufferingNewest(limit)))
case .bufferingOldest(let limit):
storageType = .unbounded(storage: UnboundedBufferStorage(base: self.base, policy: .bufferingOldest(limit)))
}
return Iterator(storageType: storageType)
}
@@ -112,20 +124,21 @@ public struct AsyncBufferSequence<Base: AsyncSequence & Sendable>: AsyncSequence
public mutating func next() async rethrows -> Element? {
switch self.storageType {
case .transparent(var iterator):
let element = try await iterator.next()
self.storageType = .transparent(iterator)
return element
case .bounded(let storage):
return try await storage.next()?._rethrowGet()
case .unbounded(let storage):
return try await storage.next()?._rethrowGet()
case .transparent(var iterator):
let element = try await iterator.next()
self.storageType = .transparent(iterator)
return element
case .bounded(let storage):
return try await storage.next().wrapped?._rethrowGet()
case .unbounded(let storage):
return try await storage.next().wrapped?._rethrowGet()
}
}
}
}
extension AsyncBufferSequence: Sendable where Base: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncBufferSequence: Sendable where Base: Sendable {}
@available(*, unavailable)
extension AsyncBufferSequence.Iterator: Sendable { }
extension AsyncBufferSequence.Iterator: Sendable {}
@@ -11,21 +11,25 @@
import DequeModule
@available(AsyncAlgorithms 1.0, *)
struct BoundedBufferStateMachine<Base: AsyncSequence> {
typealias Element = Base.Element
typealias SuspendedProducer = UnsafeContinuation<Void, Never>
typealias SuspendedConsumer = UnsafeContinuation<Result<Base.Element, Error>?, Never>
typealias SuspendedConsumer = UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>
// We are using UnsafeTransfer here since we have to get the elements from the task
// into the consumer task. This is a transfer but we cannot prove this to the compiler at this point
// since next is not marked as transferring the return value.
fileprivate enum State {
case initial(base: Base)
case buffering(
task: Task<Void, Never>,
buffer: Deque<Result<Element, Error>>,
buffer: Deque<Result<UnsafeTransfer<Element>, Error>>,
suspendedProducer: SuspendedProducer?,
suspendedConsumer: SuspendedConsumer?
)
case modifying
case finished(buffer: Deque<Result<Element, Error>>)
case finished(buffer: Deque<Result<UnsafeTransfer<Element>, Error>>)
}
private var state: State
@@ -38,52 +42,52 @@ struct BoundedBufferStateMachine<Base: AsyncSequence> {
var task: Task<Void, Never>? {
switch self.state {
case .buffering(let task, _, _, _):
return task
default:
return nil
case .buffering(let task, _, _, _):
return task
default:
return nil
}
}
mutating func taskStarted(task: Task<Void, Never>) {
switch self.state {
case .initial:
self.state = .buffering(task: task, buffer: [], suspendedProducer: nil, suspendedConsumer: nil)
case .buffering:
preconditionFailure("Invalid state.")
case .initial:
self.state = .buffering(task: task, buffer: [], suspendedProducer: nil, suspendedConsumer: nil)
case .modifying:
preconditionFailure("Invalid state.")
case .buffering:
preconditionFailure("Invalid state.")
case .finished:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
preconditionFailure("Invalid state.")
}
}
mutating func shouldSuspendProducer() -> Bool {
switch state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(_, let buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if there are free slots, we should directly request the next element
return buffer.count >= self.limit
case .buffering(_, let buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if there are free slots, we should directly request the next element
return buffer.count >= self.limit
case .buffering(_, _, .none, .some):
// we have an awaiting consumer, we should not suspended the producer, we should
// directly request the next element
return false
case .buffering(_, _, .none, .some):
// we have an awaiting consumer, we should not suspended the producer, we should
// directly request the next element
return false
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There is already a suspended producer.")
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There is already a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return false
case .finished:
return false
}
}
@@ -94,107 +98,117 @@ struct BoundedBufferStateMachine<Base: AsyncSequence> {
mutating func producerSuspended(continuation: SuspendedProducer) -> ProducerSuspendedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(let task, let buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if the buffer is available we resume the producer so it can we can request the next element
// otherwise we confirm the suspension
if buffer.count < limit {
return .resumeProducer
} else {
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: continuation, suspendedConsumer: nil)
return .none
}
case .buffering(let task, let buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if the buffer is available we resume the producer so it can we can request the next element
// otherwise we confirm the suspension
guard buffer.count < limit else {
self.state = .buffering(
task: task,
buffer: buffer,
suspendedProducer: continuation,
suspendedConsumer: nil
)
return .none
}
return .resumeProducer
case .buffering(_, let buffer, .none, .some):
// we have an awaiting consumer, we can resume the producer so the next element can be requested
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty as we have an awaiting consumer already.")
return .resumeProducer
case .buffering(_, let buffer, .none, .some):
// we have an awaiting consumer, we can resume the producer so the next element can be requested
precondition(
buffer.isEmpty,
"Invalid state. The buffer should be empty as we have an awaiting consumer already."
)
return .resumeProducer
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There is already a suspended producer.")
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There is already a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .resumeProducer
case .finished:
return .resumeProducer
}
}
enum ElementProducedAction {
case none
case resumeConsumer(continuation: SuspendedConsumer, result: Result<Element, Error>)
case resumeConsumer(continuation: SuspendedConsumer, result: UnsafeTransfer<Result<Base.Element, Error>?>)
}
mutating func elementProduced(element: Element) -> ElementProducedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(let task, var buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// we have to stack the new element or suspend the producer if the buffer is full
precondition(buffer.count < limit, "Invalid state. The buffer should be available for stacking a new element.")
self.state = .modifying
buffer.append(.success(element))
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .none
case .buffering(let task, var buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// we have to stack the new element or suspend the producer if the buffer is full
precondition(
buffer.count < limit,
"Invalid state. The buffer should be available for stacking a new element."
)
self.state = .modifying
buffer.append(.success(.init(element)))
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .none
case .buffering(let task, let buffer, .none, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it with the element and exit
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .resumeConsumer(continuation: suspendedConsumer, result: .success(element))
case .buffering(let task, let buffer, .none, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it with the element and exit
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .resumeConsumer(continuation: suspendedConsumer, result: UnsafeTransfer(.success(element)))
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There should not be a suspended producer.")
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There should not be a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
case .finished:
return .none
}
}
enum FinishAction {
case none
case resumeConsumer(
continuation: UnsafeContinuation<Result<Base.Element, Error>?, Never>?
continuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>?
)
}
mutating func finish(error: Error?) -> FinishAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(_, var buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if we have an error we stack it in the buffer so it can be consumed later
if let error {
buffer.append(.failure(error))
}
self.state = .finished(buffer: buffer)
return .none
case .buffering(_, var buffer, .none, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if we have an error we stack it in the buffer so it can be consumed later
if let error {
buffer.append(.failure(error))
}
self.state = .finished(buffer: buffer)
return .none
case .buffering(_, let buffer, .none, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .finished(buffer: [])
return .resumeConsumer(continuation: suspendedConsumer)
case .buffering(_, let buffer, .none, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .finished(buffer: [])
return .resumeConsumer(continuation: suspendedConsumer)
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There should not be a suspended producer.")
case .buffering(_, _, .some, _):
preconditionFailure("Invalid state. There should not be a suspended producer.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
case .finished:
return .none
}
}
@@ -206,34 +220,34 @@ struct BoundedBufferStateMachine<Base: AsyncSequence> {
mutating func next() -> NextAction {
switch state {
case .initial(let base):
return .startTask(base: base)
case .initial(let base):
return .startTask(base: base)
case .buffering(_, let buffer, .none, .none) where buffer.isEmpty:
// we are idle, we must suspend the consumer
return .suspend
case .buffering(_, let buffer, .none, .none) where buffer.isEmpty:
// we are idle, we must suspend the consumer
return .suspend
case .buffering(let task, var buffer, let suspendedProducer, .none):
// we have values in the buffer, we unstack the oldest one and resume a potential suspended producer
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .returnResult(producerContinuation: suspendedProducer, result: result)
case .buffering(let task, var buffer, let suspendedProducer, .none):
// we have values in the buffer, we unstack the oldest one and resume a potential suspended producer
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .returnResult(producerContinuation: suspendedProducer, result: result.map { $0.wrapped })
case .buffering(_, _, _, .some):
preconditionFailure("Invalid states. There is already a suspended consumer.")
case .buffering(_, _, _, .some):
preconditionFailure("Invalid states. There is already a suspended consumer.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(producerContinuation: nil, result: nil)
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(producerContinuation: nil, result: nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(producerContinuation: nil, result: result)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(producerContinuation: nil, result: result.map { $0.wrapped })
}
}
@@ -244,35 +258,35 @@ struct BoundedBufferStateMachine<Base: AsyncSequence> {
mutating func nextSuspended(continuation: SuspendedConsumer) -> NextSuspendedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .initial:
preconditionFailure("Invalid state. The task should already be started.")
case .buffering(let task, let buffer, .none, .none) where buffer.isEmpty:
// we are idle, we confirm the suspension of the consumer
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: continuation)
return .none
case .buffering(let task, let buffer, .none, .none) where buffer.isEmpty:
// we are idle, we confirm the suspension of the consumer
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: continuation)
return .none
case .buffering(let task, var buffer, let suspendedProducer, .none):
// we have values in the buffer, we unstack the oldest one and resume a potential suspended producer
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .returnResult(producerContinuation: suspendedProducer, result: result)
case .buffering(let task, var buffer, let suspendedProducer, .none):
// we have values in the buffer, we unstack the oldest one and resume a potential suspended producer
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedProducer: nil, suspendedConsumer: nil)
return .returnResult(producerContinuation: suspendedProducer, result: result.map { $0.wrapped })
case .buffering(_, _, _, .some):
preconditionFailure("Invalid states. There is already a suspended consumer.")
case .buffering(_, _, _, .some):
preconditionFailure("Invalid states. There is already a suspended consumer.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(producerContinuation: nil, result: nil)
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(producerContinuation: nil, result: nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(producerContinuation: nil, result: result)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(producerContinuation: nil, result: result.map { $0.wrapped })
}
}
@@ -281,33 +295,35 @@ struct BoundedBufferStateMachine<Base: AsyncSequence> {
case resumeProducerAndConsumer(
task: Task<Void, Never>,
producerContinuation: UnsafeContinuation<Void, Never>?,
consumerContinuation: UnsafeContinuation<Result<Base.Element, Error>?, Never>?
consumerContinuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>?
)
}
mutating func interrupted() -> InterruptedAction {
switch self.state {
case .initial:
self.state = .finished(buffer: [])
return .none
case .initial:
self.state = .finished(buffer: [])
return .none
case .buffering(let task, _, let suspendedProducer, let suspendedConsumer):
self.state = .finished(buffer: [])
return .resumeProducerAndConsumer(
task: task,
producerContinuation: suspendedProducer,
consumerContinuation: suspendedConsumer
)
case .buffering(let task, _, let suspendedProducer, let suspendedConsumer):
self.state = .finished(buffer: [])
return .resumeProducerAndConsumer(
task: task,
producerContinuation: suspendedProducer,
consumerContinuation: suspendedConsumer
)
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
self.state = .finished(buffer: [])
return .none
case .finished:
self.state = .finished(buffer: [])
return .none
}
}
}
extension BoundedBufferStateMachine: Sendable where Base: Sendable { }
extension BoundedBufferStateMachine.State: Sendable where Base: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension BoundedBufferStateMachine: Sendable where Base: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension BoundedBufferStateMachine.State: Sendable where Base: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Sendable {
private let stateMachine: ManagedCriticalState<BoundedBufferStateMachine<Base>>
@@ -16,49 +17,51 @@ final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Send
self.stateMachine = ManagedCriticalState(BoundedBufferStateMachine(base: base, limit: limit))
}
func next() async -> Result<Base.Element, Error>? {
func next() async -> UnsafeTransfer<Result<Base.Element, Error>?> {
return await withTaskCancellationHandler {
let action: BoundedBufferStateMachine<Base>.NextAction? = self.stateMachine.withCriticalRegion { stateMachine in
let action: BoundedBufferStateMachine<Base>.NextAction? = self.stateMachine.withCriticalRegion {
stateMachine in
let action = stateMachine.next()
switch action {
case .startTask(let base):
self.startTask(stateMachine: &stateMachine, base: base)
return nil
case .startTask(let base):
self.startTask(stateMachine: &stateMachine, base: base)
return nil
case .suspend:
return action
case .returnResult:
return action
case .suspend:
return action
case .returnResult:
return action
}
}
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .suspend:
break
case .suspend:
break
case .returnResult(let producerContinuation, let result):
producerContinuation?.resume()
return result
case .returnResult(let producerContinuation, let result):
producerContinuation?.resume()
return UnsafeTransfer(result)
case .none:
break
}
return await withUnsafeContinuation { (continuation: UnsafeContinuation<Result<Base.Element, Error>?, Never>) in
return await withUnsafeContinuation {
(continuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.nextSuspended(continuation: continuation)
}
switch action {
case .none:
break
case .returnResult(let producerContinuation, let result):
producerContinuation?.resume()
continuation.resume(returning: result)
case .none:
break
case .returnResult(let producerContinuation, let result):
producerContinuation?.resume()
continuation.resume(returning: UnsafeTransfer(result))
}
}
} onCancel: {
@@ -86,10 +89,10 @@ final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Send
}
switch action {
case .none:
break
case .resumeProducer:
continuation.resume()
case .none:
break
case .resumeProducer:
continuation.resume()
}
}
}
@@ -103,10 +106,10 @@ final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Send
stateMachine.elementProduced(element: element)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation, let result):
continuation.resume(returning: result)
case .none:
break
case .resumeConsumer(let continuation, let result):
continuation.resume(returning: result)
}
}
@@ -114,20 +117,20 @@ final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Send
stateMachine.finish(error: nil)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: nil)
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(nil))
}
} catch {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: error)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: .failure(error))
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(Result<Base.Element, Error>.failure(error)))
}
}
}
@@ -140,12 +143,12 @@ final class BoundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Send
stateMachine.interrupted()
}
switch action {
case .none:
break
case .resumeProducerAndConsumer(let task, let producerContinuation, let consumerContinuation):
task.cancel()
producerContinuation?.resume()
consumerContinuation?.resume(returning: nil)
case .none:
break
case .resumeProducerAndConsumer(let task, let producerContinuation, let consumerContinuation):
task.cancel()
producerContinuation?.resume()
consumerContinuation?.resume(returning: UnsafeTransfer(nil))
}
}
@@ -11,9 +11,10 @@
import DequeModule
@available(AsyncAlgorithms 1.0, *)
struct UnboundedBufferStateMachine<Base: AsyncSequence> {
typealias Element = Base.Element
typealias SuspendedConsumer = UnsafeContinuation<Result<Element, Error>?, Never>
typealias SuspendedConsumer = UnsafeContinuation<UnsafeTransfer<Result<Element, Error>?>, Never>
enum Policy {
case unlimited
@@ -21,15 +22,18 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
case bufferingOldest(Int)
}
// We are using UnsafeTransfer here since we have to get the elements from the task
// into the consumer task. This is a transfer but we cannot prove this to the compiler at this point
// since next is not marked as transferring the return value.
fileprivate enum State {
case initial(base: Base)
case buffering(
task: Task<Void, Never>,
buffer: Deque<Result<Element, Error>>,
buffer: Deque<Result<UnsafeTransfer<Element>, Error>>,
suspendedConsumer: SuspendedConsumer?
)
case modifying
case finished(buffer: Deque<Result<Element, Error>>)
case finished(buffer: Deque<Result<UnsafeTransfer<Element>, Error>>)
}
private var state: State
@@ -42,26 +46,26 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
var task: Task<Void, Never>? {
switch self.state {
case .buffering(let task, _, _):
return task
default:
return nil
case .buffering(let task, _, _):
return task
default:
return nil
}
}
mutating func taskStarted(task: Task<Void, Never>) {
switch self.state {
case .initial:
self.state = .buffering(task: task, buffer: [], suspendedConsumer: nil)
case .initial:
self.state = .buffering(task: task, buffer: [], suspendedConsumer: nil)
case .buffering:
preconditionFailure("Invalid state.")
case .buffering:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
preconditionFailure("Invalid state.")
case .finished:
preconditionFailure("Invalid state.")
}
}
@@ -69,49 +73,49 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
case none
case resumeConsumer(
continuation: SuspendedConsumer,
result: Result<Element, Error>
result: UnsafeTransfer<Result<Element, Error>?>
)
}
mutating func elementProduced(element: Element) -> ElementProducedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(let task, var buffer, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// we have to apply the policy when stacking the new element
self.state = .modifying
switch self.policy {
case .unlimited:
buffer.append(.success(element))
case .bufferingNewest(let limit):
if buffer.count >= limit {
_ = buffer.popFirst()
}
buffer.append(.success(element))
case .bufferingOldest(let limit):
if buffer.count < limit {
buffer.append(.success(element))
}
case .buffering(let task, var buffer, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// we have to apply the policy when stacking the new element
self.state = .modifying
switch self.policy {
case .unlimited:
buffer.append(.success(.init(element)))
case .bufferingNewest(let limit):
if buffer.count >= limit {
_ = buffer.popFirst()
}
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .none
buffer.append(.success(.init(element)))
case .bufferingOldest(let limit):
if buffer.count < limit {
buffer.append(.success(.init(element)))
}
}
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .none
case .buffering(let task, let buffer, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it with the element
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .resumeConsumer(
continuation: suspendedConsumer,
result: .success(element)
)
case .buffering(let task, let buffer, .some(let suspendedConsumer)):
// we have an awaiting consumer, we can resume it with the element
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .resumeConsumer(
continuation: suspendedConsumer,
result: UnsafeTransfer(.success(element))
)
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
case .finished:
return .none
}
}
@@ -122,29 +126,29 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
mutating func finish(error: Error?) -> FinishAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(_, var buffer, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if we have an error we stack it in the buffer so it can be consumed later
if let error {
buffer.append(.failure(error))
}
self.state = .finished(buffer: buffer)
return .none
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(_, let buffer, let suspendedConsumer):
// we have an awaiting consumer, we can resume it with nil or the error
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .finished(buffer: [])
return .resumeConsumer(continuation: suspendedConsumer)
case .buffering(_, var buffer, .none):
// we are either idle or the buffer is already in use (no awaiting consumer)
// if we have an error we stack it in the buffer so it can be consumed later
if let error {
buffer.append(.failure(error))
}
self.state = .finished(buffer: buffer)
return .none
case .modifying:
preconditionFailure("Invalid state.")
case .buffering(_, let buffer, let suspendedConsumer):
// we have an awaiting consumer, we can resume it with nil or the error
precondition(buffer.isEmpty, "Invalid state. The buffer should be empty.")
self.state = .finished(buffer: [])
return .resumeConsumer(continuation: suspendedConsumer)
case .finished:
return .none
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
return .none
}
}
@@ -156,33 +160,33 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
mutating func next() -> NextAction {
switch self.state {
case .initial(let base):
return .startTask(base: base)
case .buffering(_, let buffer, let suspendedConsumer) where buffer.isEmpty:
// we are idle, we have to suspend the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
return .suspend
case .initial(let base):
return .startTask(base: base)
case .buffering(let task, var buffer, let suspendedConsumer):
// the buffer is already in use, we can unstack a value and directly resume the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .returnResult(result)
case .buffering(_, let buffer, let suspendedConsumer) where buffer.isEmpty:
// we are idle, we have to suspend the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
return .suspend
case .modifying:
preconditionFailure("Invalid state.")
case .buffering(let task, var buffer, let suspendedConsumer):
// the buffer is already in use, we can unstack a value and directly resume the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .returnResult(result.map { $0.wrapped })
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(nil)
case .modifying:
preconditionFailure("Invalid state.")
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(result)
case .finished(let buffer) where buffer.isEmpty:
return .returnResult(nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .returnResult(result.map { $0.wrapped })
}
}
@@ -193,34 +197,34 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
mutating func nextSuspended(continuation: SuspendedConsumer) -> NextSuspendedAction {
switch self.state {
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .initial:
preconditionFailure("Invalid state. The task should already by started.")
case .buffering(let task, let buffer, let suspendedConsumer) where buffer.isEmpty:
// we are idle, we confirm the suspension of the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: continuation)
return .none
case .buffering(let task, let buffer, let suspendedConsumer) where buffer.isEmpty:
// we are idle, we confirm the suspension of the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: continuation)
return .none
case .buffering(let task, var buffer, let suspendedConsumer):
// the buffer is already in use, we can unstack a value and directly resume the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .resumeConsumer(result)
case .buffering(let task, var buffer, let suspendedConsumer):
// the buffer is already in use, we can unstack a value and directly resume the consumer
precondition(suspendedConsumer == nil, "Invalid states. There is already a suspended consumer.")
self.state = .modifying
let result = buffer.popFirst()!
self.state = .buffering(task: task, buffer: buffer, suspendedConsumer: nil)
return .resumeConsumer(result.map { $0.wrapped })
case .modifying:
preconditionFailure("Invalid state.")
case .modifying:
preconditionFailure("Invalid state.")
case .finished(let buffer) where buffer.isEmpty:
return .resumeConsumer(nil)
case .finished(let buffer) where buffer.isEmpty:
return .resumeConsumer(nil)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .resumeConsumer(result)
case .finished(var buffer):
self.state = .modifying
let result = buffer.popFirst()!
self.state = .finished(buffer: buffer)
return .resumeConsumer(result.map { $0.wrapped })
}
}
@@ -231,23 +235,25 @@ struct UnboundedBufferStateMachine<Base: AsyncSequence> {
mutating func interrupted() -> InterruptedAction {
switch self.state {
case .initial:
state = .finished(buffer: [])
return .none
case .buffering(let task, _, let suspendedConsumer):
self.state = .finished(buffer: [])
return .resumeConsumer(task: task, continuation: suspendedConsumer)
case .initial:
state = .finished(buffer: [])
return .none
case .modifying:
preconditionFailure("Invalid state.")
case .buffering(let task, _, let suspendedConsumer):
self.state = .finished(buffer: [])
return .resumeConsumer(task: task, continuation: suspendedConsumer)
case .finished:
self.state = .finished(buffer: [])
return .none
case .modifying:
preconditionFailure("Invalid state.")
case .finished:
self.state = .finished(buffer: [])
return .none
}
}
}
extension UnboundedBufferStateMachine: Sendable where Base: Sendable { }
extension UnboundedBufferStateMachine.State: Sendable where Base: Sendable { }
@available(AsyncAlgorithms 1.0, *)
extension UnboundedBufferStateMachine: Sendable where Base: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension UnboundedBufferStateMachine.State: Sendable where Base: Sendable {}
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class UnboundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Sendable {
private let stateMachine: ManagedCriticalState<UnboundedBufferStateMachine<Base>>
@@ -16,44 +17,46 @@ final class UnboundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Se
self.stateMachine = ManagedCriticalState(UnboundedBufferStateMachine<Base>(base: base, policy: policy))
}
func next() async -> Result<Base.Element, Error>? {
func next() async -> UnsafeTransfer<Result<Base.Element, Error>?> {
return await withTaskCancellationHandler {
let action: UnboundedBufferStateMachine<Base>.NextAction? = self.stateMachine.withCriticalRegion { stateMachine in
let action: UnboundedBufferStateMachine<Base>.NextAction? = self.stateMachine.withCriticalRegion {
stateMachine in
let action = stateMachine.next()
switch action {
case .startTask(let base):
self.startTask(stateMachine: &stateMachine, base: base)
return nil
case .suspend:
return action
case .returnResult:
return action
case .startTask(let base):
self.startTask(stateMachine: &stateMachine, base: base)
return nil
case .suspend:
return action
case .returnResult:
return action
}
}
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .suspend:
break
case .returnResult(let result):
return result
case .none:
break
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .suspend:
break
case .returnResult(let result):
return UnsafeTransfer(result)
case .none:
break
}
return await withUnsafeContinuation { (continuation: UnsafeContinuation<Result<Base.Element, Error>?, Never>) in
return await withUnsafeContinuation {
(continuation: UnsafeContinuation<UnsafeTransfer<Result<Base.Element, Error>?>, Never>) in
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.nextSuspended(continuation: continuation)
}
switch action {
case .none:
break
case .resumeConsumer(let result):
continuation.resume(returning: result)
case .none:
break
case .resumeConsumer(let result):
continuation.resume(returning: UnsafeTransfer(result))
}
}
} onCancel: {
@@ -72,10 +75,10 @@ final class UnboundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Se
stateMachine.elementProduced(element: element)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation, let result):
continuation.resume(returning: result)
case .none:
break
case .resumeConsumer(let continuation, let result):
continuation.resume(returning: result)
}
}
@@ -83,20 +86,20 @@ final class UnboundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Se
stateMachine.finish(error: nil)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: nil)
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(nil))
}
} catch {
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.finish(error: error)
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: .failure(error))
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: UnsafeTransfer(Result<Base.Element, Error>.failure(error)))
}
}
}
@@ -109,11 +112,11 @@ final class UnboundedBufferStorage<Base: AsyncSequence>: Sendable where Base: Se
stateMachine.interrupted()
}
switch action {
case .none:
break
case .resumeConsumer(let task, let continuation):
task.cancel()
continuation?.resume(returning: nil)
case .none:
break
case .resumeConsumer(let task, let continuation):
task.cancel()
continuation?.resume(returning: UnsafeTransfer(nil))
}
}
@@ -19,6 +19,7 @@
/// on the `Iterator` is made, or when `finish()` is called from another Task.
/// As `finish()` induces a terminal state, there is no more need for a back pressure management.
/// This function does not suspend and will finish all the pending iterations.
@available(AsyncAlgorithms 1.0, *)
public final class AsyncChannel<Element: Sendable>: AsyncSequence, Sendable {
public typealias Element = Element
public typealias AsyncIterator = Iterator
@@ -60,4 +61,4 @@ public final class AsyncChannel<Element: Sendable>: AsyncSequence, Sendable {
}
@available(*, unavailable)
extension AsyncChannel.Iterator: Sendable { }
extension AsyncChannel.Iterator: Sendable {}
@@ -18,6 +18,7 @@
/// and is resumed when the next call to `next()` on the `Iterator` is made, or when `finish()`/`fail(_:)` is called
/// from another Task. As `finish()` and `fail(_:)` induce a terminal state, there is no more need for a back pressure management.
/// Those functions do not suspend and will finish all the pending iterations.
@available(AsyncAlgorithms 1.0, *)
public final class AsyncThrowingChannel<Element: Sendable, Failure: Error>: AsyncSequence, Sendable {
public typealias Element = Element
public typealias AsyncIterator = Iterator
@@ -63,4 +64,4 @@ public final class AsyncThrowingChannel<Element: Sendable, Failure: Error>: Asyn
}
@available(*, unavailable)
extension AsyncThrowingChannel.Iterator: Sendable { }
extension AsyncThrowingChannel.Iterator: Sendable {}
@@ -10,9 +10,7 @@
//===----------------------------------------------------------------------===//
import OrderedCollections
// NOTE: this is only marked as unchecked since the swift-collections tag is before auditing for Sendable
extension OrderedSet: @unchecked Sendable where Element: Sendable { }
@available(AsyncAlgorithms 1.0, *)
struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
private struct SuspendedProducer: Hashable, Sendable {
let id: UInt64
@@ -64,7 +62,12 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
case terminated(Termination)
}
private var state: State = .channeling(suspendedProducers: [], cancelledProducers: [], suspendedConsumers: [], cancelledConsumers: [])
private var state: State = .channeling(
suspendedProducers: [],
cancelledProducers: [],
suspendedConsumers: [],
cancelledConsumers: []
)
enum SendAction {
case resumeConsumer(continuation: UnsafeContinuation<Element?, any Error>?)
@@ -73,23 +76,23 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
mutating func send() -> SendAction {
switch self.state {
case .channeling(_, _, let suspendedConsumers, _) where suspendedConsumers.isEmpty:
// we are idle or waiting for consumers, we have to suspend the producer
return .suspend
case .channeling(_, _, let suspendedConsumers, _) where suspendedConsumers.isEmpty:
// we are idle or waiting for consumers, we have to suspend the producer
return .suspend
case .channeling(let suspendedProducers, let cancelledProducers, var suspendedConsumers, let cancelledConsumers):
// we are waiting for producers, we can resume the first available consumer
let suspendedConsumer = suspendedConsumers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(continuation: suspendedConsumer.continuation)
case .channeling(let suspendedProducers, let cancelledProducers, var suspendedConsumers, let cancelledConsumers):
// we are waiting for producers, we can resume the first available consumer
let suspendedConsumer = suspendedConsumers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(continuation: suspendedConsumer.continuation)
case .terminated:
return .resumeConsumer(continuation: nil)
case .terminated:
return .resumeConsumer(continuation: nil)
}
}
@@ -104,45 +107,44 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
producerID: UInt64
) -> SendSuspendedAction? {
switch self.state {
case .channeling(var suspendedProducers, var cancelledProducers, var suspendedConsumers, let cancelledConsumers):
let suspendedProducer = SuspendedProducer(id: producerID, continuation: continuation, element: element)
if let _ = cancelledProducers.remove(suspendedProducer) {
// the producer was already cancelled, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer
}
if suspendedConsumers.isEmpty {
// we are idle or waiting for consumers
// we stack the incoming producer in a suspended state
suspendedProducers.append(suspendedProducer)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
} else {
// we are waiting for producers
// we resume the first consumer
let suspendedConsumer = suspendedConsumers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducerAndConsumer(continuation: suspendedConsumer.continuation)
}
case .terminated:
case .channeling(var suspendedProducers, var cancelledProducers, var suspendedConsumers, let cancelledConsumers):
let suspendedProducer = SuspendedProducer(id: producerID, continuation: continuation, element: element)
if let _ = cancelledProducers.remove(suspendedProducer) {
// the producer was already cancelled, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer
}
guard suspendedConsumers.isEmpty else {
// we are waiting for producers
// we resume the first consumer
let suspendedConsumer = suspendedConsumers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducerAndConsumer(continuation: suspendedConsumer.continuation)
}
// we are idle or waiting for consumers
// we stack the incoming producer in a suspended state
suspendedProducers.append(suspendedProducer)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated:
return .resumeProducer
}
}
@@ -153,33 +155,33 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
mutating func sendCancelled(producerID: UInt64) -> SendCancelledAction {
switch self.state {
case .channeling(var suspendedProducers, var cancelledProducers, let suspendedConsumers, let cancelledConsumers):
// the cancelled producer might be part of the waiting list
let placeHolder = SuspendedProducer.placeHolder(id: producerID)
case .channeling(var suspendedProducers, var cancelledProducers, let suspendedConsumers, let cancelledConsumers):
// the cancelled producer might be part of the waiting list
let placeHolder = SuspendedProducer.placeHolder(id: producerID)
if let removed = suspendedProducers.remove(placeHolder) {
// the producer was cancelled after being added to the suspended ones, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer(continuation: removed.continuation)
}
// the producer was cancelled before being added to the suspended ones
cancelledProducers.update(with: placeHolder)
if let removed = suspendedProducers.remove(placeHolder) {
// the producer was cancelled after being added to the suspended ones, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
return .resumeProducer(continuation: removed.continuation)
}
case .terminated:
return .none
// the producer was cancelled before being added to the suspended ones
cancelledProducers.update(with: placeHolder)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated:
return .none
}
}
@@ -193,24 +195,24 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
mutating func finish(error: Failure?) -> FinishAction {
switch self.state {
case .channeling(let suspendedProducers, _, let suspendedConsumers, _):
// no matter if we are idle, waiting for producers or waiting for consumers, we resume every thing that is suspended
if let error {
if suspendedConsumers.isEmpty {
self.state = .terminated(.failed(error))
} else {
self.state = .terminated(.finished)
}
case .channeling(let suspendedProducers, _, let suspendedConsumers, _):
// no matter if we are idle, waiting for producers or waiting for consumers, we resume every thing that is suspended
if let error {
if suspendedConsumers.isEmpty {
self.state = .terminated(.failed(error))
} else {
self.state = .terminated(.finished)
}
return .resumeProducersAndConsumers(
producerSontinuations: suspendedProducers.map { $0.continuation },
consumerContinuations: suspendedConsumers.map { $0.continuation }
)
} else {
self.state = .terminated(.finished)
}
return .resumeProducersAndConsumers(
producerSontinuations: suspendedProducers.map { $0.continuation },
consumerContinuations: suspendedConsumers.map { $0.continuation }
)
case .terminated:
return .none
case .terminated:
return .none
}
}
@@ -221,30 +223,30 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
mutating func next() -> NextAction {
switch self.state {
case .channeling(let suspendedProducers, _, _, _) where suspendedProducers.isEmpty:
// we are idle or waiting for producers, we must suspend
return .suspend
case .channeling(let suspendedProducers, _, _, _) where suspendedProducers.isEmpty:
// we are idle or waiting for producers, we must suspend
return .suspend
case .channeling(var suspendedProducers, let cancelledProducers, let suspendedConsumers, let cancelledConsumers):
// we are waiting for consumers, we can resume the first awaiting producer
let suspendedProducer = suspendedProducers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer(
continuation: suspendedProducer.continuation,
result: .success(suspendedProducer.element)
)
case .channeling(var suspendedProducers, let cancelledProducers, let suspendedConsumers, let cancelledConsumers):
// we are waiting for consumers, we can resume the first awaiting producer
let suspendedProducer = suspendedProducers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducer(
continuation: suspendedProducer.continuation,
result: .success(suspendedProducer.element)
)
case .terminated(.failed(let error)):
self.state = .terminated(.finished)
return .resumeProducer(continuation: nil, result: .failure(error))
case .terminated(.failed(let error)):
self.state = .terminated(.finished)
return .resumeProducer(continuation: nil, result: .failure(error))
case .terminated:
return .resumeProducer(continuation: nil, result: .success(nil))
case .terminated:
return .resumeProducer(continuation: nil, result: .success(nil))
}
}
@@ -259,52 +261,51 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
consumerID: UInt64
) -> NextSuspendedAction? {
switch self.state {
case .channeling(var suspendedProducers, let cancelledProducers, var suspendedConsumers, var cancelledConsumers):
let suspendedConsumer = SuspendedConsumer(id: consumerID, continuation: continuation)
if let _ = cancelledConsumers.remove(suspendedConsumer) {
// the consumer was already cancelled, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(element: nil)
}
if suspendedProducers.isEmpty {
// we are idle or waiting for producers
// we stack the incoming consumer in a suspended state
suspendedConsumers.append(suspendedConsumer)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
} else {
// we are waiting for consumers
// we resume the first producer
let suspendedProducer = suspendedProducers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducerAndConsumer(
continuation: suspendedProducer.continuation,
element: suspendedProducer.element
)
}
case .terminated(.finished):
case .channeling(var suspendedProducers, let cancelledProducers, var suspendedConsumers, var cancelledConsumers):
let suspendedConsumer = SuspendedConsumer(id: consumerID, continuation: continuation)
if let _ = cancelledConsumers.remove(suspendedConsumer) {
// the consumer was already cancelled, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(element: nil)
}
case .terminated(.failed(let error)):
self.state = .terminated(.finished)
return .resumeConsumerWithError(error: error)
guard suspendedProducers.isEmpty else {
// we are waiting for consumers
// we resume the first producer
let suspendedProducer = suspendedProducers.removeFirst()
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeProducerAndConsumer(
continuation: suspendedProducer.continuation,
element: suspendedProducer.element
)
}
// we are idle or waiting for producers
// we stack the incoming consumer in a suspended state
suspendedConsumers.append(suspendedConsumer)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated(.finished):
return .resumeConsumer(element: nil)
case .terminated(.failed(let error)):
self.state = .terminated(.finished)
return .resumeConsumerWithError(error: error)
}
}
@@ -315,33 +316,33 @@ struct ChannelStateMachine<Element: Sendable, Failure: Error>: Sendable {
mutating func nextCancelled(consumerID: UInt64) -> NextCancelledAction {
switch self.state {
case .channeling(let suspendedProducers, let cancelledProducers, var suspendedConsumers, var cancelledConsumers):
// the cancelled consumer might be part of the suspended ones
let placeHolder = SuspendedConsumer.placeHolder(id: consumerID)
case .channeling(let suspendedProducers, let cancelledProducers, var suspendedConsumers, var cancelledConsumers):
// the cancelled consumer might be part of the suspended ones
let placeHolder = SuspendedConsumer.placeHolder(id: consumerID)
if let removed = suspendedConsumers.remove(placeHolder) {
// the consumer was cancelled after being added to the suspended ones, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .resumeConsumer(continuation: removed.continuation)
}
// the consumer was cancelled before being added to the suspended ones
cancelledConsumers.update(with: placeHolder)
if let removed = suspendedConsumers.remove(placeHolder) {
// the consumer was cancelled after being added to the suspended ones, we resume it
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
return .resumeConsumer(continuation: removed.continuation)
}
case .terminated:
return .none
// the consumer was cancelled before being added to the suspended ones
cancelledConsumers.update(with: placeHolder)
self.state = .channeling(
suspendedProducers: suspendedProducers,
cancelledProducers: cancelledProducers,
suspendedConsumers: suspendedConsumers,
cancelledConsumers: cancelledConsumers
)
return .none
case .terminated:
return .none
}
}
}
@@ -8,6 +8,7 @@
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
private let stateMachine: ManagedCriticalState<ChannelStateMachine<Element, Failure>>
private let ids = ManagedCriticalState<UInt64>(0)
@@ -30,12 +31,12 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .suspend:
case .suspend:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: element)
return
case .resumeConsumer(let continuation):
continuation?.resume(returning: element)
return
}
let producerID = self.generateId()
@@ -48,13 +49,13 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .none:
break
case .resumeProducer:
continuation.resume()
case .resumeProducerAndConsumer(let consumerContinuation):
continuation.resume()
consumerContinuation?.resume(returning: element)
case .none:
break
case .resumeProducer:
continuation.resume()
case .resumeProducerAndConsumer(let consumerContinuation):
continuation.resume()
consumerContinuation?.resume(returning: element)
}
}
} onCancel: {
@@ -63,10 +64,10 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .none:
break
case .resumeProducer(let continuation):
continuation?.resume()
case .none:
break
case .resumeProducer(let continuation):
continuation?.resume()
}
}
}
@@ -77,15 +78,15 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .none:
break
case .resumeProducersAndConsumers(let producerContinuations, let consumerContinuations):
producerContinuations.forEach { $0?.resume() }
if let error {
consumerContinuations.forEach { $0?.resume(throwing: error) }
} else {
consumerContinuations.forEach { $0?.resume(returning: nil) }
}
case .none:
break
case .resumeProducersAndConsumers(let producerContinuations, let consumerContinuations):
producerContinuations.forEach { $0?.resume() }
if let error {
consumerContinuations.forEach { $0?.resume(throwing: error) }
} else {
consumerContinuations.forEach { $0?.resume(returning: nil) }
}
}
}
@@ -95,12 +96,12 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .suspend:
break
case .suspend:
break
case .resumeProducer(let producerContinuation, let result):
producerContinuation?.resume()
return try result._rethrowGet()
case .resumeProducer(let producerContinuation, let result):
producerContinuation?.resume()
return try result._rethrowGet()
}
let consumerID = self.generateId()
@@ -115,15 +116,15 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .none:
break
case .resumeConsumer(let element):
continuation.resume(returning: element)
case .resumeConsumerWithError(let error):
continuation.resume(throwing: error)
case .resumeProducerAndConsumer(let producerContinuation, let element):
producerContinuation?.resume()
continuation.resume(returning: element)
case .none:
break
case .resumeConsumer(let element):
continuation.resume(returning: element)
case .resumeConsumerWithError(let error):
continuation.resume(throwing: error)
case .resumeProducerAndConsumer(let producerContinuation, let element):
producerContinuation?.resume()
continuation.resume(returning: element)
}
}
} onCancel: {
@@ -132,10 +133,10 @@ struct ChannelStorage<Element: Sendable, Failure: Error>: Sendable {
}
switch action {
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: nil)
case .none:
break
case .resumeConsumer(let continuation):
continuation?.resume(returning: nil)
}
}
}
@@ -20,26 +20,32 @@
/// Throws:
/// ``combineLatest(_:_:)`` throws when one of the bases throws. If one of the bases threw any buffered and not yet consumed
/// values will be dropped.
@available(AsyncAlgorithms 1.0, *)
public func combineLatest<
Base1: AsyncSequence,
Base2: AsyncSequence
>(_ base1: Base1, _ base2: Base2) -> AsyncCombineLatest2Sequence<Base1, Base2> where
>(_ base1: Base1, _ base2: Base2) -> AsyncCombineLatest2Sequence<Base1, Base2>
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable {
Base2.Element: Sendable
{
AsyncCombineLatest2Sequence(base1, base2)
}
/// An `AsyncSequence` that combines the latest values produced from two asynchronous sequences into an asynchronous sequence of tuples.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncCombineLatest2Sequence<
Base1: AsyncSequence,
Base2: AsyncSequence
>: AsyncSequence, Sendable where
>: AsyncSequence, Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable {
Base2.Element: Sendable
{
public typealias Element = (Base1.Element, Base2.Element)
public typealias AsyncIterator = Iterator
@@ -89,4 +95,4 @@ public struct AsyncCombineLatest2Sequence<
}
@available(*, unavailable)
extension AsyncCombineLatest2Sequence.Iterator: Sendable { }
extension AsyncCombineLatest2Sequence.Iterator: Sendable {}
@@ -20,32 +20,38 @@
/// Throws:
/// ``combineLatest(_:_:_:)`` throws when one of the bases throws. If one of the bases threw any buffered and not yet consumed
/// values will be dropped.
@available(AsyncAlgorithms 1.0, *)
public func combineLatest<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncCombineLatest3Sequence<Base1, Base2, Base3> where
>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncCombineLatest3Sequence<Base1, Base2, Base3>
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable {
Base3.Element: Sendable
{
AsyncCombineLatest3Sequence(base1, base2, base3)
}
/// An `AsyncSequence` that combines the latest values produced from three asynchronous sequences into an asynchronous sequence of tuples.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncCombineLatest3Sequence<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: AsyncSequence, Sendable where
>: AsyncSequence, Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable {
Base3.Element: Sendable
{
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public typealias AsyncIterator = Iterator
@@ -60,7 +66,8 @@ public struct AsyncCombineLatest3Sequence<
}
public func makeAsyncIterator() -> AsyncIterator {
Iterator(storage: .init(self.base1, self.base2, self.base3)
Iterator(
storage: .init(self.base1, self.base2, self.base3)
)
}
@@ -99,4 +106,4 @@ public struct AsyncCombineLatest3Sequence<
}
@available(*, unavailable)
extension AsyncCombineLatest3Sequence.Iterator: Sendable { }
extension AsyncCombineLatest3Sequence.Iterator: Sendable {}
@@ -12,22 +12,29 @@
import DequeModule
/// State machine for combine latest
@available(AsyncAlgorithms 1.0, *)
struct CombineLatestStateMachine<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable where
>: Sendable
where
Base1: Sendable,
Base2: Sendable,
Base3: Sendable,
Base1.Element: Sendable,
Base2.Element: Sendable,
Base3.Element: Sendable {
typealias DownstreamContinuation = UnsafeContinuation<Result<(
Base1.Element,
Base2.Element,
Base3.Element?
)?, Error>, Never>
Base3.Element: Sendable
{
typealias DownstreamContinuation = UnsafeContinuation<
Result<
(
Base1.Element,
Base2.Element,
Base3.Element?
)?, Error
>, Never
>
private enum State: Sendable {
/// Small wrapper for the state of an upstream sequence.
@@ -115,7 +122,9 @@ struct CombineLatestStateMachine<
case .combining:
// An iterator was deinitialized while we have a suspended continuation.
preconditionFailure("Internal inconsistency current state \(self.state) and received iteratorDeinitialized()")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received iteratorDeinitialized()"
)
case .waitingForDemand(let task, let upstreams, _):
// The iterator was dropped which signals that the consumer is finished.
@@ -124,7 +133,8 @@ struct CombineLatestStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .upstreamThrew, .upstreamsFinished:
@@ -180,7 +190,10 @@ struct CombineLatestStateMachine<
)
}
mutating func childTaskSuspended(baseIndex: Int, continuation: UnsafeContinuation<Void, Error>) -> ChildTaskSuspendedAction? {
mutating func childTaskSuspended(
baseIndex: Int,
continuation: UnsafeContinuation<Void, Error>
) -> ChildTaskSuspendedAction? {
switch self.state {
case .initial:
// Child tasks are only created after we transitioned to `zipping`
@@ -203,7 +216,9 @@ struct CombineLatestStateMachine<
upstreams.2.continuation = continuation
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)"
)
}
self.state = .waitingForDemand(
@@ -283,7 +298,10 @@ struct CombineLatestStateMachine<
return .none
case .combining(let task, var upstreams, let downstreamContinuation, let buffer):
precondition(buffer.isEmpty, "Internal inconsistency current state \(self.state) and the buffer is not empty")
precondition(
buffer.isEmpty,
"Internal inconsistency current state \(self.state) and the buffer is not empty"
)
self.state = .modifying
switch result {
@@ -302,8 +320,9 @@ struct CombineLatestStateMachine<
// Implementing this for the two arities without variadic generics is a bit awkward sadly.
if let first = upstreams.0.element,
let second = upstreams.1.element,
let third = upstreams.2.element {
let second = upstreams.1.element,
let third = upstreams.2.element
{
// We got an element from each upstream so we can resume the downstream now
self.state = .waitingForDemand(
task: task,
@@ -317,8 +336,9 @@ struct CombineLatestStateMachine<
)
} else if let first = upstreams.0.element,
let second = upstreams.1.element,
self.numberOfUpstreamSequences == 2 {
let second = upstreams.1.element,
self.numberOfUpstreamSequences == 2
{
// We got an element from each upstream so we can resume the downstream now
self.state = .waitingForDemand(
task: task,
@@ -335,9 +355,21 @@ struct CombineLatestStateMachine<
self.state = .combining(
task: task,
upstreams: (
.init(continuation: upstreams.0.continuation, element: upstreams.0.element, isFinished: upstreams.0.isFinished),
.init(continuation: upstreams.1.continuation, element: upstreams.1.element, isFinished: upstreams.1.isFinished),
.init(continuation: upstreams.2.continuation, element: upstreams.2.element, isFinished: upstreams.2.isFinished)
.init(
continuation: upstreams.0.continuation,
element: upstreams.0.element,
isFinished: upstreams.0.isFinished
),
.init(
continuation: upstreams.1.continuation,
element: upstreams.1.element,
isFinished: upstreams.1.isFinished
),
.init(
continuation: upstreams.2.continuation,
element: upstreams.2.element,
isFinished: upstreams.2.isFinished
)
),
downstreamContinuation: downstreamContinuation,
buffer: buffer
@@ -397,7 +429,9 @@ struct CombineLatestStateMachine<
upstreams.2.isFinished = true
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished() with base index \(baseIndex)")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received upstreamFinished() with base index \(baseIndex)"
)
}
if upstreams.0.isFinished && upstreams.1.isFinished && upstreams.2.isFinished {
@@ -410,7 +444,9 @@ struct CombineLatestStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else if upstreams.0.isFinished && upstreams.1.isFinished && self.numberOfUpstreamSequences == 2 {
// All upstreams finished we can transition to either finished or upstreamsFinished now
@@ -422,7 +458,9 @@ struct CombineLatestStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else {
self.state = .waitingForDemand(
@@ -455,7 +493,9 @@ struct CombineLatestStateMachine<
emptyUpstreamFinished = upstreams.2.element == nil
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received upstreamFinished() with base index \(baseIndex)")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received upstreamFinished() with base index \(baseIndex)"
)
}
// Implementing this for the two arities without variadic generics is a bit awkward sadly.
@@ -466,7 +506,9 @@ struct CombineLatestStateMachine<
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else if upstreams.0.isFinished && upstreams.1.isFinished && upstreams.2.isFinished {
@@ -476,7 +518,9 @@ struct CombineLatestStateMachine<
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else if upstreams.0.isFinished && upstreams.1.isFinished && self.numberOfUpstreamSequences == 2 {
@@ -486,7 +530,9 @@ struct CombineLatestStateMachine<
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
)
} else {
self.state = .combining(
@@ -542,7 +588,8 @@ struct CombineLatestStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .combining(let task, let upstreams, let downstreamContinuation, _):
@@ -555,7 +602,8 @@ struct CombineLatestStateMachine<
downstreamContinuation: downstreamContinuation,
error: error,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .upstreamThrew, .finished:
@@ -597,7 +645,8 @@ struct CombineLatestStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .combining(let task, let upstreams, let downstreamContinuation, _):
@@ -608,7 +657,8 @@ struct CombineLatestStateMachine<
return .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .upstreamsFinished:
@@ -661,19 +711,10 @@ struct CombineLatestStateMachine<
// If not we have to transition to combining and need to resume all upstream continuations now
self.state = .modifying
if let element = buffer.popFirst() {
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .resumeContinuation(
downstreamContinuation: continuation,
result: .success(element)
)
} else {
let upstreamContinuations = [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
guard let element = buffer.popFirst() else {
let upstreamContinuations = [
upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation,
].compactMap { $0 }
upstreams.0.continuation = nil
upstreams.1.continuation = nil
upstreams.2.continuation = nil
@@ -689,22 +730,31 @@ struct CombineLatestStateMachine<
upstreamContinuation: upstreamContinuations
)
}
self.state = .waitingForDemand(
task: task,
upstreams: upstreams,
buffer: buffer
)
return .resumeContinuation(
downstreamContinuation: continuation,
result: .success(element)
)
case .upstreamsFinished(var buffer):
self.state = .modifying
if let element = buffer.popFirst() {
self.state = .upstreamsFinished(buffer: buffer)
return .resumeContinuation(
downstreamContinuation: continuation,
result: .success(element)
)
} else {
guard let element = buffer.popFirst() else {
self.state = .finished
return .resumeDownstreamContinuationWithNil(continuation)
}
self.state = .upstreamsFinished(buffer: buffer)
return .resumeContinuation(
downstreamContinuation: continuation,
result: .success(element)
)
case .upstreamThrew(let error):
// One of the upstreams threw and we have to return this error now.
@@ -9,17 +9,20 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class CombineLatestStorage<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable where
>: Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable {
Base3.Element: Sendable
{
typealias StateMachine = CombineLatestStateMachine<Base1, Base2, Base3>
private let stateMachine: ManagedCriticalState<StateMachine>
@@ -340,33 +343,33 @@ final class CombineLatestStorage<
}
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let error,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .none:
break
}
group.cancelAll()
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .cancelTaskAndUpstreamContinuations(let task, let upstreamContinuations):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let error,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .none:
break
}
group.cancelAll()
}
}
}
}
@@ -9,94 +9,102 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Creates an asynchronous sequence that emits the latest element after a given quiescence period
/// has elapsed by using a specified Clock.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func debounce<C: Clock>(for interval: C.Instant.Duration, tolerance: C.Instant.Duration? = nil, clock: C) -> AsyncDebounceSequence<Self, C> where Self: Sendable, Self.Element: Sendable {
AsyncDebounceSequence(self, interval: interval, tolerance: tolerance, clock: clock)
}
/// Creates an asynchronous sequence that emits the latest element after a given quiescence period
/// has elapsed by using a specified Clock.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func debounce<C: Clock>(
for interval: C.Instant.Duration,
tolerance: C.Instant.Duration? = nil,
clock: C
) -> AsyncDebounceSequence<Self, C> where Self: Sendable, Self.Element: Sendable {
AsyncDebounceSequence(self, interval: interval, tolerance: tolerance, clock: clock)
}
/// Creates an asynchronous sequence that emits the latest element after a given quiescence period
/// has elapsed.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func debounce(for interval: Duration, tolerance: Duration? = nil) -> AsyncDebounceSequence<Self, ContinuousClock> where Self: Sendable, Self.Element: Sendable {
self.debounce(for: interval, tolerance: tolerance, clock: .continuous)
}
/// Creates an asynchronous sequence that emits the latest element after a given quiescence period
/// has elapsed.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public func debounce(
for interval: Duration,
tolerance: Duration? = nil
) -> AsyncDebounceSequence<Self, ContinuousClock> where Self: Sendable, Self.Element: Sendable {
self.debounce(for: interval, tolerance: tolerance, clock: .continuous)
}
}
/// An `AsyncSequence` that emits the latest element after a given quiescence period
/// has elapsed.
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public struct AsyncDebounceSequence<Base: AsyncSequence & Sendable, C: Clock>: Sendable where Base.Element: Sendable {
private let base: Base
private let clock: C
private let interval: C.Instant.Duration
private let tolerance: C.Instant.Duration?
private let base: Base
private let clock: C
private let interval: C.Instant.Duration
private let tolerance: C.Instant.Duration?
/// Initializes a new ``AsyncDebounceSequence``.
///
/// - Parameters:
/// - base: The base sequence.
/// - interval: The interval to debounce.
/// - tolerance: The tolerance of the clock.
/// - clock: The clock.
init(_ base: Base, interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.base = base
self.interval = interval
self.tolerance = tolerance
self.clock = clock
}
/// Initializes a new ``AsyncDebounceSequence``.
///
/// - Parameters:
/// - base: The base sequence.
/// - interval: The interval to debounce.
/// - tolerance: The tolerance of the clock.
/// - clock: The clock.
init(_ base: Base, interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.base = base
self.interval = interval
self.tolerance = tolerance
self.clock = clock
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncDebounceSequence: AsyncSequence {
public typealias Element = Base.Element
public typealias Element = Base.Element
public func makeAsyncIterator() -> Iterator {
let storage = DebounceStorage(
base: self.base,
interval: self.interval,
tolerance: self.tolerance,
clock: self.clock
)
return Iterator(storage: storage)
}
public func makeAsyncIterator() -> Iterator {
let storage = DebounceStorage(
base: self.base,
interval: self.interval,
tolerance: self.tolerance,
clock: self.clock
)
return Iterator(storage: storage)
}
}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncDebounceSequence {
public struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: DebounceStorage<Base, C>
public struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: DebounceStorage<Base, C>
fileprivate init(storage: DebounceStorage<Base, C>) {
self.storage = storage
}
fileprivate init(storage: DebounceStorage<Base, C>) {
self.storage = storage
}
deinit {
self.storage.iteratorDeinitialized()
}
deinit {
self.storage.iteratorDeinitialized()
}
func next() async rethrows -> Element? {
try await self.storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: DebounceStorage<Base, C>) {
self.internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await self.internalClass.next()
}
func next() async rethrows -> Element? {
try await self.storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: DebounceStorage<Base, C>) {
self.internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await self.internalClass.next()
}
}
}
@available(*, unavailable)
extension AsyncDebounceSequence.Iterator: Sendable { }
extension AsyncDebounceSequence.Iterator: Sendable {}
File diff suppressed because it is too large. Load diff
@@ -11,312 +11,317 @@
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
final class DebounceStorage<Base: AsyncSequence & Sendable, C: Clock>: Sendable where Base.Element: Sendable {
typealias Element = Base.Element
typealias Element = Base.Element
/// The state machine protected with a lock.
private let stateMachine: ManagedCriticalState<DebounceStateMachine<Base, C>>
/// The interval to debounce.
private let interval: C.Instant.Duration
/// The tolerance for the clock.
private let tolerance: C.Instant.Duration?
/// The clock.
private let clock: C
/// The state machine protected with a lock.
private let stateMachine: ManagedCriticalState<DebounceStateMachine<Base, C>>
/// The interval to debounce.
private let interval: C.Instant.Duration
/// The tolerance for the clock.
private let tolerance: C.Instant.Duration?
/// The clock.
private let clock: C
init(base: Base, interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.stateMachine = .init(.init(base: base, clock: clock, interval: interval))
self.interval = interval
self.tolerance = tolerance
self.clock = clock
init(base: Base, interval: C.Instant.Duration, tolerance: C.Instant.Duration?, clock: C) {
self.stateMachine = .init(.init(base: base, clock: clock, interval: interval))
self.interval = interval
self.tolerance = tolerance
self.clock = clock
}
func iteratorDeinitialized() {
let action = self.stateMachine.withCriticalRegion { $0.iteratorDeinitialized() }
switch action {
case .cancelTaskAndUpstreamAndClockContinuations(
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
case .none:
break
}
}
func iteratorDeinitialized() {
let action = self.stateMachine.withCriticalRegion { $0.iteratorDeinitialized() }
func next() async rethrows -> Element? {
// We need to handle cancellation here because we are creating a continuation
// and because we need to cancel the `Task` we created to consume the upstream
return try await withTaskCancellationHandler {
// We always suspend since we can never return an element right away
let result: Result<Element?, Error> = await withUnsafeContinuation { continuation in
let action: DebounceStateMachine<Base, C>.NextAction? = self.stateMachine.withCriticalRegion {
let action = $0.next(for: continuation)
switch action {
case .startTask(let base):
self.startTask(
stateMachine: &$0,
base: base,
downstreamContinuation: continuation
)
return nil
case .resumeUpstreamContinuation:
return action
case .resumeUpstreamAndClockContinuation:
return action
case .resumeDownstreamContinuationWithNil:
return action
case .resumeDownstreamContinuationWithError:
return action
}
}
switch action {
case .cancelTaskAndUpstreamAndClockContinuations(
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
task.cancel()
case .resumeUpstreamContinuation(let upstreamContinuation):
// This is signalling the upstream task that is consuming the upstream
// sequence to signal demand.
upstreamContinuation?.resume(returning: ())
case .resumeUpstreamAndClockContinuation(let upstreamContinuation, let clockContinuation, let deadline):
// This is signalling the upstream task that is consuming the upstream
// sequence to signal demand and start the clock task.
upstreamContinuation?.resume(returning: ())
clockContinuation?.resume(returning: deadline)
case .resumeDownstreamContinuationWithNil(let continuation):
continuation.resume(returning: .success(nil))
case .resumeDownstreamContinuationWithError(let continuation, let error):
continuation.resume(returning: .failure(error))
case .none:
break
break
}
}
return try result._rethrowGet()
} onCancel: {
let action = self.stateMachine.withCriticalRegion { $0.cancelled() }
switch action {
case .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
case .none:
break
}
}
}
func next() async rethrows -> Element? {
// We need to handle cancellation here because we are creating a continuation
// and because we need to cancel the `Task` we created to consume the upstream
return try await withTaskCancellationHandler {
// We always suspend since we can never return an element right away
private func startTask(
stateMachine: inout DebounceStateMachine<Base, C>,
base: Base,
downstreamContinuation: UnsafeContinuation<Result<Base.Element?, Error>, Never>
) {
let task = Task {
await withThrowingTaskGroup(of: Void.self) { group in
// The task that consumes the upstream sequence
group.addTask {
var iterator = base.makeAsyncIterator()
let result: Result<Element?, Error> = await withUnsafeContinuation { continuation in
let action: DebounceStateMachine<Base, C>.NextAction? = self.stateMachine.withCriticalRegion {
let action = $0.next(for: continuation)
switch action {
case .startTask(let base):
self.startTask(
stateMachine: &$0,
base: base,
downstreamContinuation: continuation
)
return nil
case .resumeUpstreamContinuation:
return action
case .resumeUpstreamAndClockContinuation:
return action
case .resumeDownstreamContinuationWithNil:
return action
case .resumeDownstreamContinuationWithError:
return action
}
}
// This is our upstream consumption loop
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand
// and until the Clock sleep finished.
try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { $0.upstreamTaskSuspended(continuation) }
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .resumeContinuation(let continuation):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: ())
case .resumeUpstreamContinuation(let upstreamContinuation):
// This is signalling the upstream task that is consuming the upstream
// sequence to signal demand.
upstreamContinuation?.resume(returning: ())
case .resumeUpstreamAndClockContinuation(let upstreamContinuation, let clockContinuation, let deadline):
// This is signalling the upstream task that is consuming the upstream
// sequence to signal demand and start the clock task.
upstreamContinuation?.resume(returning: ())
clockContinuation?.resume(returning: deadline)
case .resumeDownstreamContinuationWithNil(let continuation):
continuation.resume(returning: .success(nil))
case .resumeDownstreamContinuationWithError(let continuation, let error):
continuation.resume(returning: .failure(error))
case .resumeContinuationWithError(let continuation, let error):
// This happens if the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
}
return try result._rethrowGet()
} onCancel: {
let action = self.stateMachine.withCriticalRegion { $0.cancelled() }
// We got signalled from the downstream that we have demand so let's
// request a new element from the upstream
if let element = try await iterator.next() {
let action = self.stateMachine.withCriticalRegion {
let deadline = self.clock.now.advanced(by: self.interval)
return $0.elementProduced(element, deadline: deadline)
}
switch action {
case .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
switch action {
case .resumeClockContinuation(let clockContinuation, let deadline):
clockContinuation?.resume(returning: deadline)
case .none:
break
}
} else {
// The upstream returned `nil` which indicates that it finished
let action = self.stateMachine.withCriticalRegion { $0.upstreamFinished() }
// All of this is mostly cleanup around the Task and the outstanding
// continuations used for signalling.
switch action {
case .cancelTaskAndClockContinuation(let task, let clockContinuation):
task.cancel()
clockContinuation?.resume(throwing: CancellationError())
break loop
case .resumeContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let task,
let upstreamContinuation,
let clockContinuation
):
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
case .none:
break loop
case .resumeContinuationWithElementAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let element,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(element))
break loop
case .none:
break loop
}
}
}
}
group.addTask {
// This is our clock scheduling loop
loop: while true {
do {
// We are creating a continuation sleeping on the Clock.
// This continuation is only resumed if the downstream consumer called `next`.
let deadline: C.Instant = try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion {
$0.clockTaskSuspended(continuation)
}
switch action {
case .resumeContinuation(let continuation, let deadline):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: deadline)
case .resumeContinuationWithError(let continuation, let error):
// This happens if the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
try await self.clock.sleep(until: deadline, tolerance: self.tolerance)
let action = self.stateMachine.withCriticalRegion { $0.clockSleepFinished() }
switch action {
case .resumeDownstreamContinuation(let downstreamContinuation, let element):
downstreamContinuation.resume(returning: .success(element))
case .none:
break
}
} catch {
// The only error that we expect is the `CancellationError`
// thrown from the Clock.sleep or from the withUnsafeContinuation.
// This happens if we are cleaning everything up. We can just drop that error and break our loop
precondition(
error is CancellationError,
"Received unexpected error \(error) in the Clock loop"
)
break loop
}
}
}
private func startTask(
stateMachine: inout DebounceStateMachine<Base, C>,
base: Base,
downstreamContinuation: UnsafeContinuation<Result<Base.Element?, Error>, Never>
) {
let task = Task {
await withThrowingTaskGroup(of: Void.self) { group in
// The task that consumes the upstream sequence
group.addTask {
var iterator = base.makeAsyncIterator()
// This is our upstream consumption loop
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand
// and until the Clock sleep finished.
try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { $0.upstreamTaskSuspended(continuation) }
switch action {
case .resumeContinuation(let continuation):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: ())
case .resumeContinuationWithError(let continuation, let error):
// This happens if the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
// We got signalled from the downstream that we have demand so let's
// request a new element from the upstream
if let element = try await iterator.next() {
let action = self.stateMachine.withCriticalRegion {
let deadline = self.clock.now.advanced(by: self.interval)
return $0.elementProduced(element, deadline: deadline)
}
switch action {
case .resumeClockContinuation(let clockContinuation, let deadline):
clockContinuation?.resume(returning: deadline)
case .none:
break
}
} else {
// The upstream returned `nil` which indicates that it finished
let action = self.stateMachine.withCriticalRegion { $0.upstreamFinished() }
// All of this is mostly cleanup around the Task and the outstanding
// continuations used for signalling.
switch action {
case .cancelTaskAndClockContinuation(let task, let clockContinuation):
task.cancel()
clockContinuation?.resume(throwing: CancellationError())
break loop
case .resumeContinuationWithNilAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(nil))
break loop
case .resumeContinuationWithElementAndCancelTaskAndUpstreamAndClockContinuation(
let downstreamContinuation,
let element,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .success(element))
break loop
case .none:
break loop
}
}
}
}
group.addTask {
// This is our clock scheduling loop
loop: while true {
do {
// We are creating a continuation sleeping on the Clock.
// This continuation is only resumed if the downstream consumer called `next`.
let deadline: C.Instant = try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.withCriticalRegion { $0.clockTaskSuspended(continuation) }
switch action {
case .resumeContinuation(let continuation, let deadline):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: deadline)
case .resumeContinuationWithError(let continuation, let error):
// This happens if the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
try await self.clock.sleep(until: deadline, tolerance: self.tolerance)
let action = self.stateMachine.withCriticalRegion { $0.clockSleepFinished() }
switch action {
case .resumeDownstreamContinuation(let downstreamContinuation, let element):
downstreamContinuation.resume(returning: .success(element))
case .none:
break
}
} catch {
// The only error that we expect is the `CancellationError`
// thrown from the Clock.sleep or from the withUnsafeContinuation.
// This happens if we are cleaning everything up. We can just drop that error and break our loop
precondition(error is CancellationError, "Received unexpected error \(error) in the Clock loop")
break loop
}
}
}
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuation(
let downstreamContinuation,
let error,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .cancelTaskAndClockContinuation(
let task,
let clockContinuation
):
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
case .none:
break
}
}
group.cancelAll()
}
}
}
}
stateMachine.taskStarted(task, downstreamContinuation: downstreamContinuation)
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuation(
let downstreamContinuation,
let error,
let task,
let upstreamContinuation,
let clockContinuation
):
upstreamContinuation?.resume(throwing: CancellationError())
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .cancelTaskAndClockContinuation(
let task,
let clockContinuation
):
clockContinuation?.resume(throwing: CancellationError())
task.cancel()
case .none:
break
}
}
group.cancelAll()
}
}
}
stateMachine.taskStarted(task, downstreamContinuation: downstreamContinuation)
}
}
+14 -7
View File
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension Dictionary {
/// Creates a new dictionary from the key-value pairs in the given asynchronous sequence.
///
@@ -20,14 +21,14 @@ extension Dictionary {
///
/// - Parameter keysAndValues: An asynchronous sequence of key-value pairs to use for
/// the new dictionary. Every key in `keysAndValues` must be unique.
/// - Returns: A new dictionary initialized with the elements of
/// `keysAndValues`.
/// - Precondition: The sequence must not have duplicate keys.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public init<S: AsyncSequence>(uniqueKeysWithValues keysAndValues: S) async rethrows where S.Element == (Key, Value) {
public init<S: AsyncSequence>(uniqueKeysWithValues keysAndValues: S) async rethrows
where S.Element == (Key, Value) {
self.init(uniqueKeysWithValues: try await Array(keysAndValues))
}
/// Creates a new dictionary from the key-value pairs in the given asynchronous sequence,
/// using a combining closure to determine the value for any duplicate keys.
///
@@ -46,8 +47,12 @@ extension Dictionary {
/// keys that are encountered. The closure returns the desired value for
/// the final dictionary, or throws an error if building the dictionary
/// can't proceed.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public init<S: AsyncSequence>(_ keysAndValues: S, uniquingKeysWith combine: (Value, Value) async throws -> Value) async rethrows where S.Element == (Key, Value) {
public init<S: AsyncSequence>(
_ keysAndValues: S,
uniquingKeysWith combine: (Value, Value) async throws -> Value
) async rethrows where S.Element == (Key, Value) {
self.init()
for try await (key, value) in keysAndValues {
if let existing = self[key] {
@@ -57,7 +62,7 @@ extension Dictionary {
}
}
}
/// Creates a new dictionary whose keys are the groupings returned by the
/// given closure and whose values are arrays of the elements that returned
/// each key.
@@ -70,8 +75,10 @@ extension Dictionary {
/// - values: An asynchronous sequence of values to group into a dictionary.
/// - keyForValue: A closure that returns a key for each element in
/// `values`.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public init<S: AsyncSequence>(grouping values: S, by keyForValue: (S.Element) async throws -> Key) async rethrows where Value == [S.Element] {
public init<S: AsyncSequence>(grouping values: S, by keyForValue: (S.Element) async throws -> Key) async rethrows
where Value == [S.Element] {
self.init()
for try await value in values {
let key = try await keyForValue(value)
@@ -0,0 +1,89 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.3, *)
extension AsyncSequence where Self: Sendable {
/// Transforms elements into new asynchronous sequences, emitting elements
/// from the most recent inner sequence.
///
/// When a new element is emitted by this sequence, the `transform`
/// is called to produce a new inner sequence. Iteration on the
/// previous inner sequence is cancelled, and iteration begins
/// on the new one.
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable
where T.Failure == Failure, T.Element: Sendable, Element: Sendable {
return AsyncFlatMapLatestSequence(self, transform: transform)
}
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, Failure> & Sendable
where T.Failure == Never, T.Element: Sendable, Element: Sendable {
return AsyncFlatMapLatestSequence(self) {
transform($0).mapError { _ -> Failure in
fatalError()
}
}
}
@_disfavoredOverload
public func flatMapLatest<T: AsyncSequence & Sendable>(
_ transform: @escaping @Sendable (Element) -> T
) -> some AsyncSequence<T.Element, T.Failure> & Sendable
where Failure == Never, T.Element: Sendable, Element: Sendable {
return AsyncFlatMapLatestSequence(
self.mapError { _ -> T.Failure in
fatalError()
},
transform: transform
)
}
}
@available(AsyncAlgorithms 1.3, *)
struct AsyncFlatMapLatestSequence<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>: AsyncSequence,
Sendable
where Base.Element: Sendable, Inner.Element: Sendable, Base.Failure == Inner.Failure {
typealias Element = Inner.Element
typealias Failure = Inner.Failure
let base: Base
let transform: @Sendable (Base.Element) -> Inner
init(_ base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.base = base
self.transform = transform
}
func makeAsyncIterator() -> Iterator {
return Iterator(base: base, transform: transform)
}
struct Iterator: AsyncIteratorProtocol, Sendable {
let storage: FlatMapLatestStorage<Base, Inner>
init(base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.storage = FlatMapLatestStorage(base: base, transform: transform)
}
func next() async rethrows -> Element? {
return try await storage.next(isolation: nil)
}
func next(isolation: isolated (any Actor)? = #isolation) async throws(Failure) -> Element? {
return try await storage.next(isolation: isolation)
}
}
}
@@ -0,0 +1,531 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
import DequeModule
@available(AsyncAlgorithms 1.3, *)
struct FlatMapLatestStateMachine<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>
where Base.Element: Sendable, Inner.Element: Sendable {
typealias Element = Inner.Element
private enum State {
case initial(Base)
case running(
outerTask: Task<Void, Never>?,
outerContinuation: UnsafeContinuation<Void, Error>?,
innerTask: Task<Void, Never>?,
innerContinuation: UnsafeContinuation<Void, Error>?,
downstreamContinuation: UnsafeContinuation<Element?, Error>?,
buffer: Deque<Result<Element, Error>>,
generation: Int,
outerFinished: Bool
)
case finished
}
private var state: State
private let transform: @Sendable (Base.Element) -> Inner
init(base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.state = .initial(base)
self.transform = transform
}
enum NextAction {
case returnElement(Element)
case returnNil
case throwError(Error)
case startOuterTask(Base)
case suspend
}
enum Action {
case startInnerTask(
Inner,
generation: Int,
previousTask: Task<Void, Never>?,
previousContinuation: UnsafeContinuation<Void, Error>?
)
case cancelInnerTask(Task<Void, Never>, UnsafeContinuation<Void, Error>?)
case resumeDownstream(UnsafeContinuation<Element?, Error>, Result<Element?, Error>)
case resumeOuterContinuation(UnsafeContinuation<Void, Error>)
case cancelTasks(
Task<Void, Never>?,
Task<Void, Never>?,
UnsafeContinuation<Void, Error>?,
UnsafeContinuation<Void, Error>?
)
case none
}
enum SuspendAction {
case resumeOuterContinuation(UnsafeContinuation<Void, Error>)
case resumeInnerContinuation(UnsafeContinuation<Void, Error>)
case none
}
mutating func next() -> NextAction {
switch state {
case .initial(let base):
return .startOuterTask(base)
case .running(
let outerTask,
let outerCont,
let innerTask,
let innerCont,
let downstreamCont,
var buffer,
let generation,
let outerFinished
):
if let result = buffer.popFirst() {
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
switch result {
case .success(let element): return .returnElement(element)
case .failure(let error): return .throwError(error)
}
} else if outerFinished && innerTask == nil {
state = .finished
return .returnNil
} else {
return .suspend
}
case .finished:
return .returnNil
}
}
mutating func next(for continuation: UnsafeContinuation<Element?, Error>) -> SuspendAction {
switch state {
case .initial:
fatalError("Should be started")
case .running(
let outerTask,
let outerCont,
let innerTask,
let innerCont,
let downstreamCont,
let buffer,
let generation,
let outerFinished
):
precondition(downstreamCont == nil, "Already have downstream continuation")
precondition(buffer.isEmpty, "Buffer should be empty if suspending")
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: continuation,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
// Resume waiting tasks to produce elements for downstream demand
if let innerCont = innerCont {
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: nil,
downstreamContinuation: continuation,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .resumeInnerContinuation(innerCont)
}
if let outerCont = outerCont {
state = .running(
outerTask: outerTask,
outerContinuation: nil,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: continuation,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .resumeOuterContinuation(outerCont)
}
return .none
case .finished:
continuation.resume(returning: nil)
return .none
}
}
mutating func outerTaskStarted(_ task: Task<Void, Never>) {
switch state {
case .initial:
// Transition to running
state = .running(
outerTask: task,
outerContinuation: nil,
innerTask: nil,
innerContinuation: nil,
downstreamContinuation: nil,
buffer: Deque(),
generation: 0,
outerFinished: false
)
default:
fatalError("Invalid state transition")
}
}
mutating func outerTaskSuspended(_ continuation: UnsafeContinuation<Void, Error>) -> SuspendAction {
switch state {
case .running(
let outerTask,
_,
let innerTask,
let innerCont,
let downstreamCont,
let buffer,
let generation,
let outerFinished
):
// If we have downstream demand, resume immediately
if downstreamCont != nil {
return .resumeOuterContinuation(continuation)
}
state = .running(
outerTask: outerTask,
outerContinuation: continuation,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .none
case .finished:
continuation.resume(throwing: CancellationError())
return .none
default:
fatalError("Invalid state")
}
}
mutating func outerElementProduced(_ element: Base.Element) -> Action {
switch state {
case .running(
let outerTask,
_,
let innerTask,
let innerCont,
let downstreamCont,
let buffer,
var generation,
let outerFinished
):
// New element from outer -> Cancel previous inner, start new inner
let newInner = transform(element)
generation += 1
state = .running(
outerTask: outerTask,
outerContinuation: nil, // We just consumed the continuation by producing
innerTask: nil, // Will be set by innerTaskStarted
innerContinuation: nil,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: outerFinished
)
return .startInnerTask(newInner, generation: generation, previousTask: innerTask, previousContinuation: innerCont)
case .finished:
return .none
default:
fatalError("Invalid state")
}
}
mutating func innerTaskStarted(_ task: Task<Void, Never>, generation: Int) {
switch state {
case .running(
let outerTask,
let outerCont,
_,
let innerCont,
let downstreamCont,
let buffer,
let currentGen,
let outerFinished
):
if generation != currentGen {
// Stale task from previous generation, ignore it
return
}
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: task,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
default:
// If finished, we don't care
break
}
}
mutating func innerTaskSuspended(_ continuation: UnsafeContinuation<Void, Error>, generation: Int) -> SuspendAction {
switch state {
case .running(
let outerTask,
let outerCont,
let innerTask,
_,
let downstreamCont,
let buffer,
let currentGen,
let outerFinished
):
if generation != currentGen {
// Stale generation
continuation.resume(throwing: CancellationError())
return .none
}
if downstreamCont != nil {
return .resumeInnerContinuation(continuation)
}
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: continuation,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .none
case .finished:
continuation.resume(throwing: CancellationError())
return .none
default:
fatalError("Invalid state")
}
}
mutating func innerElementProduced(_ element: Element, generation: Int) -> Action {
switch state {
case .running(
let outerTask,
let outerCont,
let innerTask,
_,
let downstreamCont,
var buffer,
let currentGen,
let outerFinished
):
if generation != currentGen {
return .none
}
guard let downstreamCont = downstreamCont else {
buffer.append(.success(element))
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: nil,
downstreamContinuation: nil,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .none
}
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: innerTask,
innerContinuation: nil, // Consumed
downstreamContinuation: nil,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .resumeDownstream(downstreamCont, .success(element))
case .finished:
return .none
default:
fatalError("Invalid state")
}
}
mutating func innerFinished(generation: Int) -> Action {
switch state {
case .running(let outerTask, let outerCont, _, _, let downstreamCont, let buffer, let currentGen, let outerFinished):
if generation != currentGen {
return .none
}
// Inner finished.
if outerFinished {
// Both finished
state = .finished
if let downstreamCont = downstreamCont {
return .resumeDownstream(downstreamCont, .success(nil))
}
} else {
// Just this inner finished. Wait for next outer.
state = .running(
outerTask: outerTask,
outerContinuation: outerCont,
innerTask: nil,
innerContinuation: nil,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
// Resume outer sequence if downstream is waiting for more elements
if downstreamCont != nil, let outerCont = outerCont {
state = .running(
outerTask: outerTask,
outerContinuation: nil, // Consumed
innerTask: nil,
innerContinuation: nil,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: currentGen,
outerFinished: outerFinished
)
return .resumeOuterContinuation(outerCont)
}
}
return .none
default:
return .none
}
}
mutating func innerThrew(_ error: Error, generation: Int) -> Action {
switch state {
case .running(let outerTask, let outerCont, let innerTask, let innerCont, let downstreamCont, _, let currentGen, _):
if generation != currentGen {
return .none
}
state = .finished
let action: Action = .cancelTasks(outerTask, innerTask, outerCont, innerCont)
guard let downstreamCont = downstreamCont else {
return action
}
return .resumeDownstream(downstreamCont, .failure(error))
default:
return .none
}
}
mutating func outerFinished() -> Action {
switch state {
case .running(let outerTask, _, let innerTask, let innerCont, let downstreamCont, let buffer, let generation, _):
if innerTask == nil && buffer.isEmpty {
state = .finished
if let downstreamCont = downstreamCont {
return .resumeDownstream(downstreamCont, .success(nil))
}
} else {
state = .running(
outerTask: outerTask,
outerContinuation: nil,
innerTask: innerTask,
innerContinuation: innerCont,
downstreamContinuation: downstreamCont,
buffer: buffer,
generation: generation,
outerFinished: true
)
}
return .none
default:
return .none
}
}
mutating func outerThrew(_ error: Error) -> Action {
switch state {
case .running(let outerTask, let outerCont, let innerTask, let innerCont, let downstreamCont, _, _, _):
state = .finished
let action: Action = .cancelTasks(outerTask, innerTask, outerCont, innerCont)
guard let downstreamCont = downstreamCont else {
return action
}
return .resumeDownstream(downstreamCont, .failure(error))
default:
return .none
}
}
mutating func cancelled() -> Action {
switch state {
case .running(let outerTask, let outerCont, let innerTask, let innerCont, let downstreamCont, _, _, _):
state = .finished
let action: Action = .cancelTasks(outerTask, innerTask, outerCont, innerCont)
if let downstreamCont = downstreamCont {
return .resumeDownstream(downstreamCont, .success(nil))
}
return action
default:
state = .finished
return .none
}
}
}
@@ -0,0 +1,199 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2025 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.3, *)
final class FlatMapLatestStorage<Base: AsyncSequence & Sendable, Inner: AsyncSequence & Sendable>: @unchecked Sendable
where Base.Element: Sendable, Inner.Element: Sendable {
typealias Element = Inner.Element
private let lock = Lock.allocate()
private var stateMachine: FlatMapLatestStateMachine<Base, Inner>
init(base: Base, transform: @escaping @Sendable (Base.Element) -> Inner) {
self.stateMachine = FlatMapLatestStateMachine(base: base, transform: transform)
}
deinit {
lock.deinitialize()
}
func next(isolation: isolated (any Actor)? = #isolation) async throws(Inner.Failure) -> Inner.Element? {
do {
return try await withTaskCancellationHandler {
lock.lock()
let action = stateMachine.next()
switch action {
case .returnElement(let element):
lock.unlock()
return element
case .returnNil:
lock.unlock()
return nil
case .throwError(let error):
lock.unlock()
throw error
case .startOuterTask(let base):
// We need to start the outer task and then suspend
startOuterTask(base)
return try await suspend()
case .suspend:
lock.unlock()
return try await suspend()
}
} onCancel: {
let action = lock.withLock { stateMachine.cancelled() }
handleAction(action)
}
} catch {
throw error as! Inner.Failure
}
}
private func suspend() async throws -> Element? {
return try await withUnsafeThrowingContinuation { continuation in
let action = lock.withLock { stateMachine.next(for: continuation) }
switch action {
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .resumeInnerContinuation(let continuation):
continuation.resume()
case .none:
break
}
}
}
private func startOuterTask(_ base: Base) {
let task = Task {
var iterator = base.makeAsyncIterator()
loop: while true {
// Create a continuation to wait for demand
do {
try await withUnsafeThrowingContinuation { continuation in
let action = lock.withLock { stateMachine.outerTaskSuspended(continuation) }
switch action {
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .resumeInnerContinuation(let continuation):
continuation.resume()
case .none:
break
}
}
} catch {
// Cancellation or other error during suspension
let action = lock.withLock { stateMachine.outerThrew(error) }
handleAction(action)
break loop
}
do {
guard let element = try await iterator.next() else {
let action = lock.withLock { stateMachine.outerFinished() }
handleAction(action)
break loop
}
let action = lock.withLock { stateMachine.outerElementProduced(element) }
handleAction(action)
} catch {
let action = lock.withLock { stateMachine.outerThrew(error) }
handleAction(action)
break loop
}
}
}
stateMachine.outerTaskStarted(task)
lock.unlock()
}
private func startInnerTask(_ inner: Inner, generation: Int) {
let task = Task {
var iterator = inner.makeAsyncIterator()
loop: while true {
// Wait for demand
do {
try await withUnsafeThrowingContinuation { continuation in
let action = lock.withLock { stateMachine.innerTaskSuspended(continuation, generation: generation) }
switch action {
case .resumeInnerContinuation(let continuation):
continuation.resume()
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .none:
break
}
}
} catch {
// Cancellation
let action = lock.withLock { stateMachine.innerThrew(error, generation: generation) }
handleAction(action)
break loop
}
do {
guard let element = try await iterator.next() else {
let action = lock.withLock { stateMachine.innerFinished(generation: generation) }
handleAction(action)
break loop
}
let action = lock.withLock { stateMachine.innerElementProduced(element, generation: generation) }
handleAction(action)
} catch {
let action = lock.withLock { stateMachine.innerThrew(error, generation: generation) }
handleAction(action)
break loop
}
}
}
stateMachine.innerTaskStarted(task, generation: generation)
}
private func handleAction(_ action: FlatMapLatestStateMachine<Base, Inner>.Action) {
switch action {
case .startInnerTask(let inner, let generation, let previousTask, let previousCont):
if let previousTask = previousTask {
previousTask.cancel()
previousCont?.resume(throwing: CancellationError())
}
startInnerTask(inner, generation: generation)
case .cancelInnerTask(let task, let continuation):
task.cancel()
continuation?.resume(throwing: CancellationError())
case .resumeDownstream(let continuation, let result):
continuation.resume(with: result)
case .resumeOuterContinuation(let continuation):
continuation.resume()
case .cancelTasks(let outer, let inner, let outerCont, let innerCont):
outer?.cancel()
inner?.cancel()
outerCont?.resume(throwing: CancellationError())
innerCont?.resume(throwing: CancellationError())
case .none:
break
}
}
}
@@ -0,0 +1,38 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2026 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.1)
// This is a helper type to move a non-Sendable value across isolation regions.
@usableFromInline
struct Disconnected<Value: ~Copyable>: ~Copyable, Sendable {
// This is safe since we take the value as sending and take consumes it
// and returns it as sending.
private nonisolated(unsafe) var value: Value?
@usableFromInline
init(value: consuming sending Value) {
self.value = .some(value)
}
@usableFromInline
consuming func take() -> sending Value {
nonisolated(unsafe) let value = self.value.take()!
return value
}
@usableFromInline
mutating func swap(newValue: consuming sending Value) -> sending Value {
nonisolated(unsafe) let value = self.value.take()!
self.value = consume newValue
return value
}
}
#endif
@@ -0,0 +1,329 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
//===----------------------------------------------------------------------===//
//
// This source file is part of the SwiftCertificates open source project
//
// Copyright (c) 2023 Apple Inc. and the SwiftCertificates project authors
// Licensed under Apache License v2.0
//
// See LICENSE.txt for license information
// See CONTRIBUTORS.txt for the list of SwiftCertificates project authors
//
// SPDX-License-Identifier: Apache-2.0
//
//===----------------------------------------------------------------------===//
/// ``_TinyArray`` is a ``RandomAccessCollection`` optimised to store zero or one ``Element``.
/// It supports arbitrary many elements but if only up to one ``Element`` is stored it does **not** allocate separate storage on the heap
/// and instead stores the ``Element`` inline.
@usableFromInline
struct _TinyArray<Element> {
@usableFromInline
enum Storage {
case one(Element)
case arbitrary([Element])
}
@usableFromInline
var storage: Storage
}
// MARK: - TinyArray "public" interface
extension _TinyArray: Equatable where Element: Equatable {}
extension _TinyArray: Hashable where Element: Hashable {}
extension _TinyArray: Sendable where Element: Sendable {}
extension _TinyArray: RandomAccessCollection {
@usableFromInline
typealias Element = Element
@usableFromInline
typealias Index = Int
@inlinable
subscript(position: Int) -> Element {
get {
self.storage[position]
}
set {
self.storage[position] = newValue
}
}
@inlinable
var startIndex: Int {
self.storage.startIndex
}
@inlinable
var endIndex: Int {
self.storage.endIndex
}
}
extension _TinyArray {
@inlinable
init(_ elements: some Sequence<Element>) {
self.storage = .init(elements)
}
@inlinable
init() {
self.storage = .init()
}
@inlinable
mutating func append(_ newElement: Element) {
self.storage.append(newElement)
}
@inlinable
mutating func append(contentsOf newElements: some Sequence<Element>) {
self.storage.append(contentsOf: newElements)
}
@discardableResult
@inlinable
mutating func remove(at index: Int) -> Element {
self.storage.remove(at: index)
}
@inlinable
mutating func removeAll(where shouldBeRemoved: (Element) throws -> Bool) rethrows {
try self.storage.removeAll(where: shouldBeRemoved)
}
@inlinable
mutating func sort(by areInIncreasingOrder: (Element, Element) throws -> Bool) rethrows {
try self.storage.sort(by: areInIncreasingOrder)
}
}
// MARK: - TinyArray.Storage "private" implementation
extension _TinyArray.Storage: Equatable where Element: Equatable {
@inlinable
static func == (lhs: Self, rhs: Self) -> Bool {
switch (lhs, rhs) {
case (.one(let lhs), .one(let rhs)):
return lhs == rhs
case (.arbitrary(let lhs), .arbitrary(let rhs)):
// we don't use lhs.elementsEqual(rhs) so we can hit the fast path from Array
// if both arrays share the same underlying storage: https://github.com/apple/swift/blob/b42019005988b2d13398025883e285a81d323efa/stdlib/public/core/Array.swift#L1775
return lhs == rhs
case (.one(let element), .arbitrary(let array)),
(.arbitrary(let array), .one(let element)):
guard array.count == 1 else {
return false
}
return element == array[0]
}
}
}
extension _TinyArray.Storage: Hashable where Element: Hashable {
@inlinable
func hash(into hasher: inout Hasher) {
// same strategy as Array: https://github.com/apple/swift/blob/b42019005988b2d13398025883e285a81d323efa/stdlib/public/core/Array.swift#L1801
hasher.combine(count)
for element in self {
hasher.combine(element)
}
}
}
extension _TinyArray.Storage: Sendable where Element: Sendable {}
extension _TinyArray.Storage: RandomAccessCollection {
@inlinable
subscript(position: Int) -> Element {
get {
switch self {
case .one(let element):
guard position == 0 else {
fatalError("index \(position) out of bounds")
}
return element
case .arbitrary(let elements):
return elements[position]
}
}
set {
switch self {
case .one:
guard position == 0 else {
fatalError("index \(position) out of bounds")
}
self = .one(newValue)
case .arbitrary(var elements):
elements[position] = newValue
self = .arbitrary(elements)
}
}
}
@inlinable
var startIndex: Int {
0
}
@inlinable
var endIndex: Int {
switch self {
case .one: return 1
case .arbitrary(let elements): return elements.endIndex
}
}
}
extension _TinyArray.Storage {
@inlinable
init(_ elements: some Sequence<Element>) {
var iterator = elements.makeIterator()
guard let firstElement = iterator.next() else {
self = .arbitrary([])
return
}
guard let secondElement = iterator.next() else {
// newElements just contains a single element
// and we hit the fast path
self = .one(firstElement)
return
}
var elements: [Element] = []
elements.reserveCapacity(elements.underestimatedCount)
elements.append(firstElement)
elements.append(secondElement)
while let nextElement = iterator.next() {
elements.append(nextElement)
}
self = .arbitrary(elements)
}
@inlinable
init() {
self = .arbitrary([])
}
@inlinable
mutating func append(_ newElement: Element) {
self.append(contentsOf: CollectionOfOne(newElement))
}
@inlinable
mutating func append(contentsOf newElements: some Sequence<Element>) {
switch self {
case .one(let firstElement):
var iterator = newElements.makeIterator()
guard let secondElement = iterator.next() else {
// newElements is empty, nothing to do
return
}
var elements: [Element] = []
elements.reserveCapacity(1 + newElements.underestimatedCount)
elements.append(firstElement)
elements.append(secondElement)
elements.appendRemainingElements(from: &iterator)
self = .arbitrary(elements)
case .arbitrary(var elements):
if elements.isEmpty {
// if `self` is currently empty and `newElements` just contains a single
// element, we skip allocating an array and set `self` to `.one(firstElement)`
var iterator = newElements.makeIterator()
guard let firstElement = iterator.next() else {
// newElements is empty, nothing to do
return
}
guard let secondElement = iterator.next() else {
// newElements just contains a single element
// and we hit the fast path
self = .one(firstElement)
return
}
elements.reserveCapacity(elements.count + newElements.underestimatedCount)
elements.append(firstElement)
elements.append(secondElement)
elements.appendRemainingElements(from: &iterator)
self = .arbitrary(elements)
} else {
elements.append(contentsOf: newElements)
self = .arbitrary(elements)
}
}
}
@discardableResult
@inlinable
mutating func remove(at index: Int) -> Element {
switch self {
case .one(let oldElement):
guard index == 0 else {
fatalError("index \(index) out of bounds")
}
self = .arbitrary([])
return oldElement
case .arbitrary(var elements):
defer {
self = .arbitrary(elements)
}
return elements.remove(at: index)
}
}
@inlinable
mutating func removeAll(where shouldBeRemoved: (Element) throws -> Bool) rethrows {
switch self {
case .one(let oldElement):
if try shouldBeRemoved(oldElement) {
self = .arbitrary([])
}
case .arbitrary(var elements):
defer {
self = .arbitrary(elements)
}
return try elements.removeAll(where: shouldBeRemoved)
}
}
@inlinable
mutating func sort(by areInIncreasingOrder: (Element, Element) throws -> Bool) rethrows {
switch self {
case .one:
// a collection of just one element is always sorted, nothing to do
break
case .arbitrary(var elements):
defer {
self = .arbitrary(elements)
}
try elements.sort(by: areInIncreasingOrder)
}
}
}
extension Array {
@inlinable
mutating func appendRemainingElements(from iterator: inout some IteratorProtocol<Element>) {
while let nextElement = iterator.next() {
append(nextElement)
}
}
}
@@ -9,145 +9,213 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequence {
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: The value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: "-")
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: The value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func interspersed(every: Int = 1, with separator: Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: @Sendable @escaping () -> Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func interspersed(
every: Int = 1,
with separator: @Sendable @escaping () -> Element
) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: @Sendable @escaping () async -> Element) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func interspersed(
every: Int = 1,
with separator: @Sendable @escaping () async -> Element
) -> AsyncInterspersedSequence<Self> {
AsyncInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: @Sendable @escaping () throws -> Element) -> AsyncThrowingInterspersedSequence<Self> {
AsyncThrowingInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func interspersed(
every: Int = 1,
with separator: @Sendable @escaping () throws -> Element
) -> AsyncThrowingInterspersedSequence<Self> {
AsyncThrowingInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@inlinable
public func interspersed(every: Int = 1, with separator: @Sendable @escaping () async throws -> Element) -> AsyncThrowingInterspersedSequence<Self> {
AsyncThrowingInterspersedSequence(self, every: every, separator: separator)
}
/// Returns a new asynchronous sequence containing the elements of this asynchronous sequence, inserting
/// the given separator between each element.
///
/// Any value of this asynchronous sequence's element type can be used as the separator.
///
/// The following example shows how an async sequences of `String`s can be interspersed using `-` as the separator:
///
/// ```
/// let input = ["A", "B", "C"].async
/// let interspersed = input.interspersed(with: { "-" })
/// for await element in interspersed {
/// print(element)
/// }
/// // Prints "A" "-" "B" "-" "C"
/// ```
///
/// - Parameters:
/// - every: Dictates after how many elements a separator should be inserted.
/// - separator: A closure that produces the value to insert in between each of this async sequence’s elements.
/// - Returns: The interspersed asynchronous sequence of elements.
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func interspersed(
every: Int = 1,
with separator: @Sendable @escaping () async throws -> Element
) -> AsyncThrowingInterspersedSequence<Self> {
AsyncThrowingInterspersedSequence(self, every: every, separator: separator)
}
}
/// An asynchronous sequence that presents the elements of a base asynchronous sequence of
/// elements with a separator between each of those elements.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncInterspersedSequence<Base: AsyncSequence> {
@usableFromInline
internal enum Separator {
case element(Element)
case syncClosure(@Sendable () -> Element)
case asyncClosure(@Sendable () async -> Element)
}
@usableFromInline
internal let base: Base
@usableFromInline
internal let separator: Separator
@usableFromInline
internal let every: Int
@usableFromInline
internal init(_ base: Base, every: Int, separator: Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .element(separator)
self.every = every
}
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () -> Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .syncClosure(separator)
self.every = every
}
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () async -> Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .asyncClosure(separator)
self.every = every
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncInterspersedSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInterspersedSequence` asynchronous sequence.
@available(AsyncAlgorithms 1.0, *)
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal enum Separator {
case element(Element)
case syncClosure(@Sendable () -> Element)
case asyncClosure(@Sendable () async -> Element)
internal enum State {
case start(Element?)
case element(Int)
case separator
case finished
}
@usableFromInline
internal let base: Base
internal var iterator: Base.AsyncIterator
@usableFromInline
internal let separator: Separator
@@ -156,151 +224,144 @@ public struct AsyncInterspersedSequence<Base: AsyncSequence> {
internal let every: Int
@usableFromInline
internal init(_ base: Base, every: Int, separator: Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
self.base = base
self.separator = .element(separator)
self.every = every
}
internal var state = State.start(nil)
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () -> Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
self.base = base
self.separator = .syncClosure(separator)
self.every = every
internal init(_ iterator: Base.AsyncIterator, every: Int, separator: Separator) {
self.iterator = iterator
self.separator = separator
self.every = every
}
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () async -> Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
self.base = base
self.separator = .asyncClosure(separator)
self.every = every
}
}
public mutating func next() async rethrows -> Base.Element? {
switch self.state {
case .start(var element):
do {
if element == nil {
element = try await self.iterator.next()
}
extension AsyncInterspersedSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInterspersedSequence` asynchronous sequence.
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal enum State {
case start(Element?)
case element(Int)
case separator
case finished
guard let element = element else {
self.state = .finished
return nil
}
if self.every == 1 {
self.state = .separator
} else {
self.state = .element(1)
}
return element
} catch {
self.state = .finished
throw error
}
@usableFromInline
internal var iterator: Base.AsyncIterator
case .separator:
do {
guard let element = try await iterator.next() else {
self.state = .finished
return nil
}
self.state = .start(element)
switch self.separator {
case .element(let element):
return element
@usableFromInline
internal let separator: Separator
case .syncClosure(let closure):
return closure()
@usableFromInline
internal let every: Int
@usableFromInline
internal var state = State.start(nil)
@usableFromInline
internal init(_ iterator: Base.AsyncIterator, every: Int, separator: Separator) {
self.iterator = iterator
self.separator = separator
self.every = every
case .asyncClosure(let closure):
return await closure()
}
} catch {
self.state = .finished
throw error
}
public mutating func next() async rethrows -> Base.Element? {
switch self.state {
case .start(var element):
do {
if element == nil {
element = try await self.iterator.next()
}
if let element = element {
if self.every == 1 {
self.state = .separator
} else {
self.state = .element(1)
}
return element
} else {
self.state = .finished
return nil
}
} catch {
self.state = .finished
throw error
}
case .separator:
do {
if let element = try await iterator.next() {
self.state = .start(element)
switch self.separator {
case .element(let element):
return element
case .syncClosure(let closure):
return closure()
case .asyncClosure(let closure):
return await closure()
}
} else {
self.state = .finished
return nil
}
} catch {
self.state = .finished
throw error
}
case .element(let count):
do {
if let element = try await iterator.next() {
let newCount = count + 1
if self.every == newCount {
self.state = .separator
} else {
self.state = .element(newCount)
}
return element
} else {
self.state = .finished
return nil
}
} catch {
self.state = .finished
throw error
}
case .finished:
return nil
}
case .element(let count):
do {
guard let element = try await iterator.next() else {
self.state = .finished
return nil
}
let newCount = count + 1
if self.every == newCount {
self.state = .separator
} else {
self.state = .element(newCount)
}
return element
} catch {
self.state = .finished
throw error
}
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(self.base.makeAsyncIterator(), every: self.every, separator: self.separator)
case .finished:
return nil
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(self.base.makeAsyncIterator(), every: self.every, separator: self.separator)
}
}
/// An asynchronous sequence that presents the elements of a base asynchronous sequence of
/// elements with a separator between each of those elements.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncThrowingInterspersedSequence<Base: AsyncSequence> {
@usableFromInline
internal enum Separator {
case syncClosure(@Sendable () throws -> Element)
case asyncClosure(@Sendable () async throws -> Element)
}
@usableFromInline
internal let base: Base
@usableFromInline
internal let separator: Separator
@usableFromInline
internal let every: Int
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () throws -> Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .syncClosure(separator)
self.every = every
}
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () async throws -> Element) {
precondition(every > 0, "Separators can only be interspersed every 1+ elements")
self.base = base
self.separator = .asyncClosure(separator)
self.every = every
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInterspersedSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInterspersedSequence` asynchronous sequence.
@available(AsyncAlgorithms 1.0, *)
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal enum Separator {
case syncClosure(@Sendable () throws -> Element)
case asyncClosure(@Sendable () async throws -> Element)
internal enum State {
case start(Element?)
case element(Int)
case separator
case finished
}
@usableFromInline
internal let base: Base
internal var iterator: Base.AsyncIterator
@usableFromInline
internal let separator: Separator
@@ -309,133 +370,96 @@ public struct AsyncThrowingInterspersedSequence<Base: AsyncSequence> {
internal let every: Int
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () throws -> Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
self.base = base
self.separator = .syncClosure(separator)
self.every = every
}
internal var state = State.start(nil)
@usableFromInline
internal init(_ base: Base, every: Int, separator: @Sendable @escaping () async throws -> Element) {
precondition(every > 0, "Separators can only be interspersed ever 1+ elements")
self.base = base
self.separator = .asyncClosure(separator)
self.every = every
}
}
extension AsyncThrowingInterspersedSequence: AsyncSequence {
public typealias Element = Base.Element
/// The iterator for an `AsyncInterspersedSequence` asynchronous sequence.
public struct Iterator: AsyncIteratorProtocol {
@usableFromInline
internal enum State {
case start(Element?)
case element(Int)
case separator
case finished
}
@usableFromInline
internal var iterator: Base.AsyncIterator
@usableFromInline
internal let separator: Separator
@usableFromInline
internal let every: Int
@usableFromInline
internal var state = State.start(nil)
@usableFromInline
internal init(_ iterator: Base.AsyncIterator, every: Int, separator: Separator) {
self.iterator = iterator
self.separator = separator
self.every = every
}
public mutating func next() async throws -> Base.Element? {
switch self.state {
case .start(var element):
do {
if element == nil {
element = try await self.iterator.next()
}
if let element = element {
if self.every == 1 {
self.state = .separator
} else {
self.state = .element(1)
}
return element
} else {
self.state = .finished
return nil
}
} catch {
self.state = .finished
throw error
}
case .separator:
do {
if let element = try await iterator.next() {
self.state = .start(element)
switch self.separator {
case .syncClosure(let closure):
return try closure()
case .asyncClosure(let closure):
return try await closure()
}
} else {
self.state = .finished
return nil
}
} catch {
self.state = .finished
throw error
}
case .element(let count):
do {
if let element = try await iterator.next() {
let newCount = count + 1
if self.every == newCount {
self.state = .separator
} else {
self.state = .element(newCount)
}
return element
} else {
self.state = .finished
return nil
}
} catch {
self.state = .finished
throw error
}
case .finished:
return nil
}
}
}
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(self.base.makeAsyncIterator(), every: self.every, separator: self.separator)
internal init(_ iterator: Base.AsyncIterator, every: Int, separator: Separator) {
self.iterator = iterator
self.separator = separator
self.every = every
}
public mutating func next() async throws -> Base.Element? {
switch self.state {
case .start(var element):
do {
if element == nil {
element = try await self.iterator.next()
}
guard let element = element else {
self.state = .finished
return nil
}
if self.every == 1 {
self.state = .separator
} else {
self.state = .element(1)
}
return element
} catch {
self.state = .finished
throw error
}
case .separator:
do {
guard let element = try await iterator.next() else {
self.state = .finished
return nil
}
self.state = .start(element)
switch self.separator {
case .syncClosure(let closure):
return try closure()
case .asyncClosure(let closure):
return try await closure()
}
} catch {
self.state = .finished
throw error
}
case .element(let count):
do {
guard let element = try await iterator.next() else {
self.state = .finished
return nil
}
let newCount = count + 1
if self.every == newCount {
self.state = .separator
} else {
self.state = .element(newCount)
}
return element
} catch {
self.state = .finished
throw error
}
case .finished:
return nil
}
}
}
@available(AsyncAlgorithms 1.0, *)
@inlinable
public func makeAsyncIterator() -> Iterator {
Iterator(self.base.makeAsyncIterator(), every: self.every, separator: self.separator)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncInterspersedSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncInterspersedSequence.Separator: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInterspersedSequence: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncThrowingInterspersedSequence.Separator: Sendable where Base: Sendable, Base.Element: Sendable {}
@available(*, unavailable)
+94 -47
View File
@@ -13,66 +13,108 @@
import Darwin
#elseif canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#elseif canImport(WinSDK)
import WinSDK
#elseif canImport(Bionic)
import Bionic
#elseif canImport(wasi_pthread)
import wasi_pthread
#elseif canImport(WASILibc)
// Here, we can not import wasi_pthread, but we can import WASILibc. So
// this is a variation of WASILibc without pthread support. As such, the
// platform does not have locks or mechanisms to become multi-threaded,
// so no locks are needed.
#else
#error("Unsupported platform")
#endif
internal struct Lock {
#if canImport(Darwin)
#if canImport(Darwin)
typealias Primitive = os_unfair_lock
#elseif canImport(Glibc)
#elseif canImport(Glibc) || canImport(Musl) || canImport(Bionic) || canImport(wasi_pthread)
#if os(FreeBSD) || os(OpenBSD)
typealias Primitive = pthread_mutex_t?
#else
typealias Primitive = pthread_mutex_t
#elseif canImport(WinSDK)
#endif
#elseif canImport(WASILibc)
// This WASILibc variation is single threaded, provides no locks
#elseif canImport(WinSDK)
typealias Primitive = SRWLOCK
#else
typealias Primitive = Int
#endif
#else
#error("Unsupported platform")
#endif
typealias PlatformLock = UnsafeMutablePointer<Primitive>
let platformLock: PlatformLock
private init(_ platformLock: PlatformLock) {
self.platformLock = platformLock
}
fileprivate static func initialize(_ platformLock: PlatformLock) {
#if canImport(Darwin)
#if canImport(Darwin)
platformLock.initialize(to: os_unfair_lock())
#elseif canImport(Glibc)
#elseif canImport(Glibc) || canImport(Musl) || canImport(Bionic) || canImport(wasi_pthread)
let result = pthread_mutex_init(platformLock, nil)
precondition(result == 0, "pthread_mutex_init failed")
#elseif canImport(WinSDK)
#elseif canImport(WASILibc)
// Here, we can not import wasi_pthread, but we can import WASILibc. So
// this is a variation of WASILibc without pthread support. As such, the
// platform does not have locks or mechanisms to become multi-threaded,
// so no locks are needed.
#elseif canImport(WinSDK)
InitializeSRWLock(platformLock)
#endif
#else
#error("Unsupported platform")
#endif
}
fileprivate static func deinitialize(_ platformLock: PlatformLock) {
#if canImport(Glibc)
#if canImport(Glibc) || canImport(Musl) || canImport(Bionic) || canImport(wasi_pthread)
let result = pthread_mutex_destroy(platformLock)
precondition(result == 0, "pthread_mutex_destroy failed")
#endif
#endif
platformLock.deinitialize(count: 1)
}
fileprivate static func lock(_ platformLock: PlatformLock) {
#if canImport(Darwin)
#if canImport(Darwin)
os_unfair_lock_lock(platformLock)
#elseif canImport(Glibc)
#elseif canImport(Glibc) || canImport(Musl) || canImport(Bionic) || canImport(wasi_pthread)
pthread_mutex_lock(platformLock)
#elseif canImport(WinSDK)
#elseif canImport(WASILibc)
// Here, we can not import wasi_pthread, but we can import WASILibc. So
// this is a variation of WASILibc without pthread support. As such, the
// platform does not have locks or mechanisms to become multi-threaded,
// so no locks are needed.
return
#elseif canImport(WinSDK)
AcquireSRWLockExclusive(platformLock)
#endif
#else
#error("Unsupported platform")
#endif
}
fileprivate static func unlock(_ platformLock: PlatformLock) {
#if canImport(Darwin)
#if canImport(Darwin)
os_unfair_lock_unlock(platformLock)
#elseif canImport(Glibc)
#elseif canImport(Glibc) || canImport(Musl) || canImport(Bionic) || canImport(wasi_pthread)
let result = pthread_mutex_unlock(platformLock)
precondition(result == 0, "pthread_mutex_unlock failed")
#elseif canImport(WinSDK)
#elseif canImport(WASILibc)
// Here, we can not import wasi_pthread, but we can import WASILibc. So
// this is a variation of WASILibc without pthread support. As such, the
// platform does not have locks or mechanisms to become multi-threaded,
// so no locks are needed.
return
#elseif canImport(WinSDK)
ReleaseSRWLockExclusive(platformLock)
#endif
#else
#error("Unsupported platform")
#endif
}
static func allocate() -> Lock {
@@ -83,6 +125,7 @@ internal struct Lock {
func deinitialize() {
Lock.deinitialize(platformLock)
platformLock.deallocate()
}
func lock() {
@@ -93,26 +136,26 @@ internal struct Lock {
Lock.unlock(platformLock)
}
/// Acquire the lock for the duration of the given block.
///
/// This convenience method should be preferred to `lock` and `unlock` in
/// most situations, as it ensures that the lock will be released regardless
/// of how `body` exits.
///
/// - Parameter body: The block to execute while holding the lock.
/// - Returns: The value returned by the block.
func withLock<T>(_ body: () throws -> T) rethrows -> T {
self.lock()
defer {
self.unlock()
}
return try body()
/// Acquire the lock for the duration of the given block.
///
/// This convenience method should be preferred to `lock` and `unlock` in
/// most situations, as it ensures that the lock will be released regardless
/// of how `body` exits.
///
/// - Parameter body: The block to execute while holding the lock.
/// - Returns: The value returned by the block.
func withLock<T>(_ body: () throws -> T) rethrows -> T {
self.lock()
defer {
self.unlock()
}
return try body()
}
// specialise Void return (for performance)
func withLockVoid(_ body: () throws -> Void) rethrows -> Void {
try self.withLock(body)
}
// specialise Void return (for performance)
func withLockVoid(_ body: () throws -> Void) rethrows {
try self.withLock(body)
}
}
struct ManagedCriticalState<State> {
@@ -121,16 +164,16 @@ struct ManagedCriticalState<State> {
withUnsafeMutablePointerToElements { Lock.deinitialize($0) }
}
}
private let buffer: ManagedBuffer<State, Lock.Primitive>
init(_ initial: State) {
buffer = LockedBuffer.create(minimumCapacity: 1) { buffer in
buffer.withUnsafeMutablePointerToElements { Lock.initialize($0) }
return initial
}
}
func withCriticalRegion<R>(_ critical: (inout State) throws -> R) rethrows -> R {
try buffer.withUnsafeMutablePointers { header, lock in
Lock.lock(lock)
@@ -138,6 +181,10 @@ struct ManagedCriticalState<State> {
return try critical(&header.pointee)
}
}
func withLock<R>(_ critical: (inout State) throws -> R) rethrows -> R {
return try withCriticalRegion(critical)
}
}
extension ManagedCriticalState: @unchecked Sendable where State: Sendable { }
extension ManagedCriticalState: @unchecked Sendable where State: Sendable {}
@@ -12,86 +12,98 @@
import DequeModule
/// Creates an asynchronous sequence of elements from two underlying asynchronous sequences
public func merge<Base1: AsyncSequence, Base2: AsyncSequence>(_ base1: Base1, _ base2: Base2) -> AsyncMerge2Sequence<Base1, Base2>
where
Base1.Element == Base2.Element,
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable
@available(AsyncAlgorithms 1.0, *)
public func merge<Base1: AsyncSequence, Base2: AsyncSequence>(
_ base1: Base1,
_ base2: Base2
) -> AsyncMerge2Sequence<Base1, Base2>
where
Base1.Element == Base2.Element,
Base1: Sendable,
Base2: Sendable,
Base1.Element: Sendable
{
return AsyncMerge2Sequence(base1, base2)
return AsyncMerge2Sequence(base1, base2)
}
/// An ``Swift/AsyncSequence`` that takes two upstream ``Swift/AsyncSequence``s and combines their elements.
/// An `AsyncSequence` that takes two upstream `AsyncSequence`s and combines their elements.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncMerge2Sequence<
Base1: AsyncSequence,
Base2: AsyncSequence
>: Sendable where
Base1.Element == Base2.Element,
Base1: Sendable, Base2: Sendable,
Base1.Element: Sendable
Base1: AsyncSequence,
Base2: AsyncSequence
>: Sendable
where
Base1.Element == Base2.Element,
Base1: Sendable,
Base2: Sendable,
Base1.Element: Sendable
{
public typealias Element = Base1.Element
public typealias Element = Base1.Element
private let base1: Base1
private let base2: Base2
private let base1: Base1
private let base2: Base2
/// Initializes a new ``AsyncMerge2Sequence``.
///
/// - Parameters:
/// - base1: The first upstream ``Swift/AsyncSequence``.
/// - base2: The second upstream ``Swift/AsyncSequence``.
init(
_ base1: Base1,
_ base2: Base2
) {
self.base1 = base1
self.base2 = base2
}
/// Initializes a new ``AsyncMerge2Sequence``.
///
/// - Parameters:
/// - base1: The first upstream ``Swift/AsyncSequence``.
/// - base2: The second upstream ``Swift/AsyncSequence``.
init(
_ base1: Base1,
_ base2: Base2
) {
self.base1 = base1
self.base2 = base2
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncMerge2Sequence: AsyncSequence {
public func makeAsyncIterator() -> Iterator {
let storage = MergeStorage<Base1, Base2, Base1>(
base1: base1,
base2: base2,
base3: nil
)
return Iterator(storage: storage)
}
@available(AsyncAlgorithms 1.0, *)
public func makeAsyncIterator() -> Iterator {
let storage = MergeStorage<Base1, Base2, Base1>(
base1: base1,
base2: base2,
base3: nil
)
return Iterator(storage: storage)
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncMerge2Sequence {
public struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: MergeStorage<Base1, Base2, Base1>
@available(AsyncAlgorithms 1.0, *)
public struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: MergeStorage<Base1, Base2, Base1>
fileprivate init(storage: MergeStorage<Base1, Base2, Base1>) {
self.storage = storage
}
fileprivate init(storage: MergeStorage<Base1, Base2, Base1>) {
self.storage = storage
}
deinit {
self.storage.iteratorDeinitialized()
}
deinit {
self.storage.iteratorDeinitialized()
}
func next() async rethrows -> Element? {
try await storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: MergeStorage<Base1, Base2, Base1>) {
internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await internalClass.next()
}
func next() async rethrows -> Element? {
try await storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: MergeStorage<Base1, Base2, Base1>) {
internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await internalClass.next()
}
}
}
@available(*, unavailable)
extension AsyncMerge2Sequence.Iterator: Sendable { }
extension AsyncMerge2Sequence.Iterator: Sendable {}
@@ -12,97 +12,108 @@
import DequeModule
/// Creates an asynchronous sequence of elements from two underlying asynchronous sequences
@available(AsyncAlgorithms 1.0, *)
public func merge<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>(_ base1: Base1, _ base2: Base2, _ base3: Base3) -> AsyncMerge3Sequence<Base1, Base2, Base3>
where
Base1.Element == Base2.Element,
Base1.Element == Base3.Element,
Base1: Sendable, Base2: Sendable, Base3: Sendable,
Base1.Element: Sendable
where
Base1.Element == Base2.Element,
Base1.Element == Base3.Element,
Base1: Sendable,
Base2: Sendable,
Base3: Sendable,
Base1.Element: Sendable
{
return AsyncMerge3Sequence(base1, base2, base3)
return AsyncMerge3Sequence(base1, base2, base3)
}
/// An ``Swift/AsyncSequence`` that takes three upstream ``Swift/AsyncSequence``s and combines their elements.
/// An `AsyncSequence` that takes three upstream `AsyncSequence`s and combines their elements.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncMerge3Sequence<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable where
Base1.Element == Base2.Element,
Base1.Element == Base3.Element,
Base1: Sendable, Base2: Sendable, Base3: Sendable,
Base1.Element: Sendable
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable
where
Base1.Element == Base2.Element,
Base1.Element == Base3.Element,
Base1: Sendable,
Base2: Sendable,
Base3: Sendable,
Base1.Element: Sendable
{
public typealias Element = Base1.Element
public typealias Element = Base1.Element
private let base1: Base1
private let base2: Base2
private let base3: Base3
private let base1: Base1
private let base2: Base2
private let base3: Base3
/// Initializes a new ``AsyncMerge2Sequence``.
///
/// - Parameters:
/// - base1: The first upstream ``Swift/AsyncSequence``.
/// - base2: The second upstream ``Swift/AsyncSequence``.
/// - base3: The third upstream ``Swift/AsyncSequence``.
init(
_ base1: Base1,
_ base2: Base2,
_ base3: Base3
) {
self.base1 = base1
self.base2 = base2
self.base3 = base3
}
/// Initializes a new ``AsyncMerge2Sequence``.
///
/// - Parameters:
/// - base1: The first upstream ``Swift/AsyncSequence``.
/// - base2: The second upstream ``Swift/AsyncSequence``.
/// - base3: The third upstream ``Swift/AsyncSequence``.
init(
_ base1: Base1,
_ base2: Base2,
_ base3: Base3
) {
self.base1 = base1
self.base2 = base2
self.base3 = base3
}
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncMerge3Sequence: AsyncSequence {
public func makeAsyncIterator() -> Iterator {
let storage = MergeStorage(
base1: base1,
base2: base2,
base3: base3
)
return Iterator(storage: storage)
}
@available(AsyncAlgorithms 1.0, *)
public func makeAsyncIterator() -> Iterator {
let storage = MergeStorage(
base1: base1,
base2: base2,
base3: base3
)
return Iterator(storage: storage)
}
}
public extension AsyncMerge3Sequence {
struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: MergeStorage<Base1, Base2, Base3>
@available(AsyncAlgorithms 1.0, *)
extension AsyncMerge3Sequence {
@available(AsyncAlgorithms 1.0, *)
public struct Iterator: AsyncIteratorProtocol {
/// This class is needed to hook the deinit to observe once all references to the ``AsyncIterator`` are dropped.
///
/// If we get move-only types we should be able to drop this class and use the `deinit` of the ``AsyncIterator`` struct itself.
final class InternalClass: Sendable {
private let storage: MergeStorage<Base1, Base2, Base3>
fileprivate init(storage: MergeStorage<Base1, Base2, Base3>) {
self.storage = storage
}
fileprivate init(storage: MergeStorage<Base1, Base2, Base3>) {
self.storage = storage
}
deinit {
self.storage.iteratorDeinitialized()
}
deinit {
self.storage.iteratorDeinitialized()
}
func next() async rethrows -> Element? {
try await storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: MergeStorage<Base1, Base2, Base3>) {
internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await internalClass.next()
}
func next() async rethrows -> Element? {
try await storage.next()
}
}
let internalClass: InternalClass
fileprivate init(storage: MergeStorage<Base1, Base2, Base3>) {
internalClass = InternalClass(storage: storage)
}
public mutating func next() async rethrows -> Element? {
try await internalClass.next()
}
}
}
@available(*, unavailable)
extension AsyncMerge3Sequence.Iterator: Sendable { }
extension AsyncMerge3Sequence.Iterator: Sendable {}
File diff suppressed because it is too large. Load diff
+269 -261
View File
@@ -9,280 +9,288 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class MergeStorage<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: @unchecked Sendable where
Base1.Element == Base2.Element,
Base1.Element == Base3.Element,
Base1: Sendable, Base2: Sendable, Base3: Sendable,
Base1.Element: Sendable
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: @unchecked Sendable
where
Base1.Element == Base2.Element,
Base1.Element == Base3.Element,
Base1: Sendable,
Base2: Sendable,
Base3: Sendable,
Base1.Element: Sendable
{
typealias Element = Base1.Element
typealias Element = Base1.Element
/// The lock that protects our state.
private let lock = Lock.allocate()
/// The state machine.
private var stateMachine: MergeStateMachine<Base1, Base2, Base3>
/// The lock that protects our state.
private let lock = Lock.allocate()
/// The state machine.
private var stateMachine: MergeStateMachine<Base1, Base2, Base3>
init(
base1: Base1,
base2: Base2,
base3: Base3?
) {
stateMachine = .init(base1: base1, base2: base2, base3: base3)
init(
base1: Base1,
base2: Base2,
base3: Base3?
) {
stateMachine = .init(base1: base1, base2: base2, base3: base3)
}
deinit {
self.lock.deinitialize()
}
func iteratorDeinitialized() {
let action = lock.withLock { self.stateMachine.iteratorDeinitialized() }
switch action {
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
}
func next() async rethrows -> Element? {
// We need to handle cancellation here because we are creating a continuation
// and because we need to cancel the `Task` we created to consume the upstream
try await withTaskCancellationHandler {
self.lock.lock()
let action = self.stateMachine.next()
switch action {
case .startTaskAndSuspendDownstreamTask(let base1, let base2, let base3):
self.startTask(
stateMachine: &self.stateMachine,
base1: base1,
base2: base2,
base3: base3
)
// It is safe to hold the lock across this method
// since the closure is guaranteed to be run straight away
return try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.next(for: continuation)
self.lock.unlock()
switch action {
case let .resumeUpstreamContinuations(upstreamContinuations):
// This is signalling the child tasks that are consuming the upstream
// sequences to signal demand.
upstreamContinuations.forEach { $0.resume(returning: ()) }
}
}
case let .returnElement(element):
self.lock.unlock()
return try element._rethrowGet()
case .returnNil:
self.lock.unlock()
return nil
case let .throwError(error):
self.lock.unlock()
throw error
case .suspendDownstreamTask:
// It is safe to hold the lock across this method
// since the closure is guaranteed to be run straight away
return try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.next(for: continuation)
self.lock.unlock()
switch action {
case let .resumeUpstreamContinuations(upstreamContinuations):
// This is signalling the child tasks that are consuming the upstream
// sequences to signal demand.
upstreamContinuations.forEach { $0.resume(returning: ()) }
}
}
}
} onCancel: {
let action = self.lock.withLock { self.stateMachine.cancelled() }
switch action {
case let .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation,
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: nil)
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
}
}
private func startTask(
stateMachine: inout MergeStateMachine<Base1, Base2, Base3>,
base1: Base1,
base2: Base2,
base3: Base3?
) {
// This creates a new `Task` that is iterating the upstream
// sequences. We must store it to cancel it at the right times.
let task = Task {
await withThrowingTaskGroup(of: Void.self) { group in
self.iterateAsyncSequence(base1, in: &group)
self.iterateAsyncSequence(base2, in: &group)
// Copy from the above just using the base3 sequence
if let base3 = base3 {
self.iterateAsyncSequence(base3, in: &group)
}
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.lock.withLock {
self.stateMachine.upstreamThrew(error)
}
switch action {
case let .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
downstreamContinuation,
error,
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(throwing: error)
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
group.cancelAll()
}
}
}
}
deinit {
self.lock.deinitialize()
}
// We need to inform our state machine that we started the Task
stateMachine.taskStarted(task)
}
func iteratorDeinitialized() {
let action = lock.withLock { self.stateMachine.iteratorDeinitialized() }
private func iterateAsyncSequence<AsyncSequence: _Concurrency.AsyncSequence>(
_ base: AsyncSequence,
in taskGroup: inout ThrowingTaskGroup<Void, Error>
) where AsyncSequence.Element == Base1.Element, AsyncSequence: Sendable {
// For each upstream sequence we are adding a child task that
// is consuming the upstream sequence
taskGroup.addTask {
var iterator = base.makeAsyncIterator()
switch action {
case let .cancelTaskAndUpstreamContinuations(
// This is our upstream consumption loop
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand.
try await withUnsafeThrowingContinuation { continuation in
let action = self.lock.withLock {
self.stateMachine.childTaskSuspended(continuation)
}
switch action {
case let .resumeContinuation(continuation):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: ())
case let .resumeContinuationWithError(continuation, error):
// This happens if another upstream already failed or if
// the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
// We got signalled from the downstream that we have demand so let's
// request a new element from the upstream
if let element1 = try await iterator.next() {
let action = self.lock.withLock {
self.stateMachine.elementProduced(element1)
}
switch action {
case let .resumeContinuation(continuation, element):
// We had an outstanding demand and where the first
// upstream to produce an element so we can forward it to
// the downstream
continuation.resume(returning: element)
case .none:
break
}
} else {
// The upstream returned `nil` which indicates that it finished
let action = self.lock.withLock {
self.stateMachine.upstreamFinished()
}
// All of this is mostly cleanup around the Task and the outstanding
// continuations used for signalling.
switch action {
case let .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation,
task,
upstreamContinuations
):
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
}
func next() async rethrows -> Element? {
// We need to handle cancellation here because we are creating a continuation
// and because we need to cancel the `Task` we created to consume the upstream
try await withTaskCancellationHandler {
self.lock.lock()
let action = self.stateMachine.next()
switch action {
case .startTaskAndSuspendDownstreamTask(let base1, let base2, let base3):
self.startTask(
stateMachine: &self.stateMachine,
base1: base1,
base2: base2,
base3: base3
)
// It is safe to hold the lock across this method
// since the closure is guaranteed to be run straight away
return try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.next(for: continuation)
self.lock.unlock()
switch action {
case let .resumeUpstreamContinuations(upstreamContinuations):
// This is signalling the child tasks that are consuming the upstream
// sequences to signal demand.
upstreamContinuations.forEach { $0.resume(returning: ()) }
}
}
case let .returnElement(element):
self.lock.unlock()
return try element._rethrowGet()
case .returnNil:
self.lock.unlock()
return nil
case let .throwError(error):
self.lock.unlock()
throw error
case .suspendDownstreamTask:
// It is safe to hold the lock across this method
// since the closure is guaranteed to be run straight away
return try await withUnsafeThrowingContinuation { continuation in
let action = self.stateMachine.next(for: continuation)
self.lock.unlock()
switch action {
case let .resumeUpstreamContinuations(upstreamContinuations):
// This is signalling the child tasks that are consuming the upstream
// sequences to signal demand.
upstreamContinuations.forEach { $0.resume(returning: ()) }
}
}
}
} onCancel: {
let action = self.lock.withLock { self.stateMachine.cancelled() }
switch action {
case let .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation,
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: nil)
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
}
}
private func startTask(stateMachine: inout MergeStateMachine<Base1, Base2, Base3>, base1: Base1, base2: Base2, base3: Base3?) {
// This creates a new `Task` that is iterating the upstream
// sequences. We must store it to cancel it at the right times.
let task = Task {
await withThrowingTaskGroup(of: Void.self) { group in
self.iterateAsyncSequence(base1, in: &group)
self.iterateAsyncSequence(base2, in: &group)
// Copy from the above just using the base3 sequence
if let base3 = base3 {
self.iterateAsyncSequence(base3, in: &group)
}
while !group.isEmpty {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.lock.withLock {
self.stateMachine.upstreamThrew(error)
}
switch action {
case let .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
downstreamContinuation,
error,
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(throwing: error)
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
case .none:
break
}
group.cancelAll()
}
}
}
}
// We need to inform our state machine that we started the Task
stateMachine.taskStarted(task)
}
private func iterateAsyncSequence<AsyncSequence: _Concurrency.AsyncSequence>(
_ base: AsyncSequence,
in taskGroup: inout ThrowingTaskGroup<Void, Error>
) where AsyncSequence.Element == Base1.Element, AsyncSequence: Sendable {
// For each upstream sequence we are adding a child task that
// is consuming the upstream sequence
taskGroup.addTask {
var iterator = base.makeAsyncIterator()
// This is our upstream consumption loop
loop: while true {
// We are creating a continuation before requesting the next
// element from upstream. This continuation is only resumed
// if the downstream consumer called `next` to signal his demand.
try await withUnsafeThrowingContinuation { continuation in
let action = self.lock.withLock {
self.stateMachine.childTaskSuspended(continuation)
}
switch action {
case let .resumeContinuation(continuation):
// This happens if there is outstanding demand
// and we need to demand from upstream right away
continuation.resume(returning: ())
case let .resumeContinuationWithError(continuation, error):
// This happens if another upstream already failed or if
// the task got cancelled.
continuation.resume(throwing: error)
case .none:
break
}
}
// We got signalled from the downstream that we have demand so let's
// request a new element from the upstream
if let element1 = try await iterator.next() {
let action = self.lock.withLock {
self.stateMachine.elementProduced(element1)
}
switch action {
case let .resumeContinuation(continuation, element):
// We had an outstanding demand and where the first
// upstream to produce an element so we can forward it to
// the downstream
continuation.resume(returning: element)
case .none:
break
}
} else {
// The upstream returned `nil` which indicates that it finished
let action = self.lock.withLock {
self.stateMachine.upstreamFinished()
}
// All of this is mostly cleanup around the Task and the outstanding
// continuations used for signalling.
switch action {
case let .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation,
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: nil)
break loop
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
break loop
case .none:
break loop
}
}
}
downstreamContinuation.resume(returning: nil)
break loop
case let .cancelTaskAndUpstreamContinuations(
task,
upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
break loop
case .none:
break loop
}
}
}
}
}
}
@@ -0,0 +1,728 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2023 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
#if compiler(>=6.1)
/// An error that is thrown from the various `send` methods of the
/// ``MultiProducerSingleConsumerAsyncChannel/Source``.
///
/// This error is thrown when the channel is already finished when
/// trying to send new elements to the source.
@available(AsyncAlgorithms 1.1, *)
public struct MultiProducerSingleConsumerAsyncChannelAlreadyFinishedError: Error {
@usableFromInline
init() {}
}
/// A multi-producer single-consumer channel.
///
/// The ``MultiProducerSingleConsumerAsyncChannel`` provides a ``MultiProducerSingleConsumerAsyncChannel/Source`` to
/// send values to the channel. The channel supports different back pressure strategies to control the
/// buffering and demand. The channel will buffer values until its backpressure strategy decides that the
/// producer have to wait.
///
/// This channel is also suitable for the single-producer single-consumer use-case
///
/// ## Using a MultiProducerSingleConsumerAsyncChannel
///
/// To use a ``MultiProducerSingleConsumerAsyncChannel`` you have to create a new channel with its source first by calling
/// the ``MultiProducerSingleConsumerAsyncChannel/makeChannel(of:throwing:backpressureStrategy:)`` method.
/// Afterwards, you can pass the source to the producer and the channel to the consumer.
///
/// ```
/// let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
/// of: Int.self,
/// backpressureStrategy: .watermark(low: 2, high: 4)
/// )
///
/// // The channel and source can be extracted from the returned type
/// let channel = consume channelAndSource.channel
/// let source = consume channelAndSource.source
/// ```
///
/// ### Asynchronous producing
///
/// Values can be send to the source from asynchronous contexts using ``MultiProducerSingleConsumerAsyncChannel/Source/send(contentsOf:)-1h7t9``
/// and ``MultiProducerSingleConsumerAsyncChannel/Source/send(_:)->()``. Backpressure results in calls
/// to the `send` methods to be suspended. Once more elements should be produced the `send` methods will be resumed.
///
/// ```
/// try await withThrowingTaskGroup(of: Void.self) { group in
/// group.addTask {
/// try await source.send(1)
/// try await source.send(2)
/// try await source.send(3)
/// }
///
/// for await element in channel {
/// print(element)
/// }
/// }
/// ```
///
/// ### Synchronous produceing
///
/// Values can also be send to the source from synchronous context. Backpressure is also exposed on the synchronous contexts; however,
/// it is up to the caller to decide how to properly translate the backpressure to underlying producer e.g. by blocking the thread.
///
/// ```swift
/// do {
/// let sendResult = try source.send(contentsOf: sequence)
///
/// switch sendResult {
/// case .produceMore:
/// // Trigger more production in the underlying system
///
/// case .enqueueCallback(let callbackHandle):
/// // There are enough values in the channel already. We need to enqueue
/// // a callback to get notified when we should produce more.
/// callbackHandle.enqueueCallback(onProduceMore: { result in
/// switch result {
/// case .success:
/// // Trigger more production in the underlying system
/// case .failure(let error):
/// // Terminate the underlying producer
/// }
/// })
/// }
/// } catch {
/// // `send(contentsOf:)` throws if the channel already terminated
/// }
/// ```
///
/// ### Multiple producers
///
/// To support multiple producers the source offers a ``Source/makeAdditionalSource()`` method to produce a new source.
///
/// ### Terminating the production of values
///
/// The consumer can be terminated through multiple ways:
/// - Calling ``Source/finish(throwing:)``.
/// - Deiniting all sources.
///
/// In both cases, if there are still elements buffered by the channel, then the consumer will receive
/// all buffered elements. Afterwards it will be terminated.
///
/// ### Observing termination of the consumer
///
/// When the consumer stops consumption by either deiniting the channel or the task calling ``next(isolation:)``
/// getting cancelled, the source will get notified about the termination if a termination callback has been set
/// before by calling ``Source/setOnTerminationCallback(_:)``.
@available(AsyncAlgorithms 1.1, *)
public struct MultiProducerSingleConsumerAsyncChannel<Element, Failure: Error>: ~Copyable {
/// The backing storage.
@usableFromInline
let storage: _Storage
/// A struct containing the initialized channel and source.
///
/// This struct can be deconstructed by consuming the individual
/// components from it.
///
/// ```swift
/// let channelAndSource = MultiProducerSingleConsumerAsyncChannel.makeChannel(
/// of: Int.self,
/// backpressureStrategy: .watermark(low: 5, high: 10)
/// )
/// var channel = consume channelAndSource.channel
/// var source = consume channelAndSource.source
/// ```
@frozen
public struct ChannelAndStream: ~Copyable {
/// The channel.
@usableFromInline
var channel: Disconnected<MultiProducerSingleConsumerAsyncChannel?>
/// Takes and returns the channel.
///
/// - Important: If this is called more than once it will result in a runtime crash.
@inlinable
public mutating func takeChannel() -> sending MultiProducerSingleConsumerAsyncChannel {
var channel = self.channel.swap(newValue: nil)
return channel.take()!
}
/// The source.
public var source: Source
init(
channel: consuming sending MultiProducerSingleConsumerAsyncChannel,
source: consuming Source
) {
self.channel = Disconnected(value: channel)
self.source = source
}
}
/// Initializes a new ``MultiProducerSingleConsumerAsyncChannel`` and an ``MultiProducerSingleConsumerAsyncChannel/Source``.
///
/// - Parameters:
/// - elementType: The element type of the channel.
/// - failureType: The failure type of the channel.
/// - backpressureStrategy: The backpressure strategy that the channel should use.
/// - Returns: A struct containing the channel and its source. The source should be passed to the
/// producer while the channel should be passed to the consumer.
public static func makeChannel(
of elementType: Element.Type = Element.self,
throwing failureType: Failure.Type = Never.self,
backpressureStrategy: Source.BackpressureStrategy
) -> sending ChannelAndStream {
let storage = _Storage(
backpressureStrategy: backpressureStrategy.internalBackpressureStrategy
)
let source = Source(storage: storage)
return .init(channel: .init(storage: storage), source: source)
}
init(storage: _Storage) {
self.storage = storage
}
deinit {
self.storage.channelDeinitialized()
}
/// Returns the next element.
///
/// If this method returns `nil` it indicates that no further values can ever
/// be returned. The channel automatically closes when all sources have been deinited.
///
/// If there are no elements and the channel has not been finished yet, this method will
/// suspend until an element is send to the channel.
///
/// If the task calling this method is cancelled this method will return `nil`.
///
/// - Parameter isolation: The callers isolation.
/// - Returns: The next buffered element.
@inlinable
public mutating func next(
isolation: isolated (any Actor)? = #isolation
) async throws(Failure) -> Element? {
do {
return try await self.storage.next()
} catch {
// This force-cast is safe since we only allow closing the source with this failure
// We only need this force cast since continuations don't support typed throws yet.
throw error as! Failure
}
}
}
@available(AsyncAlgorithms 1.1, *)
extension MultiProducerSingleConsumerAsyncChannel {
/// A struct to send values to the channel.
///
/// Use this source to provide elements to the channel by calling one of the `send` methods.
public struct Source: ~Copyable {
/// A struct representing the backpressure of the channel.
public struct BackpressureStrategy: Sendable {
var internalBackpressureStrategy: _InternalBackpressureStrategy
/// A backpressure strategy using a high and low watermark to suspend and resume production respectively.
///
/// - Parameters:
/// - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
/// - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
public static func watermark(low: Int, high: Int) -> BackpressureStrategy {
.init(
internalBackpressureStrategy: .watermark(
.init(low: low, high: high, waterLevelForElement: nil)
)
)
}
/// A backpressure strategy using a high and low watermark to suspend and resume production respectively.
///
/// - Parameters:
/// - low: When the number of buffered elements drops below the low watermark, producers will be resumed.
/// - high: When the number of buffered elements rises above the high watermark, producers will be suspended.
/// - waterLevelForElement: A closure used to compute the contribution of each buffered element to the current water level.
///
/// - Important: `waterLevelForElement` will be called during a lock on each element when it is written into the source and when
/// it is consumed from the channel, so it must be side-effect free and at best constant in time.
public static func watermark(
low: Int,
high: Int,
waterLevelForElement: @escaping @Sendable (borrowing Element) -> Int
) -> BackpressureStrategy {
.init(
internalBackpressureStrategy: .watermark(
.init(low: low, high: high, waterLevelForElement: waterLevelForElement)
)
)
}
}
/// A type that indicates the result of sending elements to the source.
public enum SendResult: ~Copyable, Sendable {
/// A handle that is returned when the channel's backpressure strategy indicated that production should
/// be suspended. Use this handle to enqueue a callback by calling the ``CallbackHandle/enqueueCallback(onProduceMore:)`` method.
///
/// - Important: ``CallbackHandle/enqueueCallback(onProduceMore:)`` and ``CallbackHandle/cancelCallback()`` must
/// only be called once.
public struct CallbackHandle: Sendable, Hashable {
@usableFromInline
let _id: UInt64
@usableFromInline
let _storage: _Storage
@usableFromInline
init(id: UInt64, storage: _Storage) {
self._id = id
self._storage = storage
}
/// Enqueues a callback that will be invoked once more elements should be produced.
///
/// - Important: Calling enqueue more than once is **not allowed**.
///
/// - Parameters:
/// - onProduceMore: The callback which gets invoked once more elements should be produced.
@inlinable
public mutating func enqueueCallback(
onProduceMore: sending @escaping (Result<Void, Error>) -> Void
) {
self._storage.enqueueProducer(callbackToken: self._id, onProduceMore: onProduceMore)
}
/// Cancel an enqueued callback.
///
/// - Note: This methods supports being called before ``enqueueCallback(onProduceMore:)`` is called.
///
/// - Important: Calling enqueue more than once is **not allowed**.
@inlinable
public mutating func cancelCallback() {
self._storage.cancelProducer(callbackToken: self._id)
}
@inlinable
public static func == (lhs: Self, rhs: Self) -> Bool {
lhs._id == rhs._id
}
@inlinable
public func hash(into hasher: inout Hasher) {
hasher.combine(self._id)
}
}
/// Indicates that more elements should be produced and send to the source.
case produceMore
/// Indicates that a callback should be enqueued.
case enqueueCallback(CallbackHandle)
}
@usableFromInline
let _storage: _Storage
@usableFromInline
let _id: UInt64
internal init(
storage: _Storage
) {
self._storage = storage
self._id = self._storage.sourceInitialized()
}
deinit {
self._storage.sourceDeinitialized()
}
/// Sets a callback to invoke when the channel terminated.
///
/// This is called after the last element has been consumed by the channel.
/// If the channel has already terminated this callback is called immediately.
///
/// - Important: Only one termination callback can be set per source. Setting a callback if
/// a previous one has been set will override the previous one.
public func setOnTerminationCallback(_ callback: (@Sendable () -> Void)?) {
self._storage.setOnTerminationCallback(sourceID: self._id, callback: callback)
}
/// Creates a new source which can be used to send elements to the channel concurrently.
///
/// The channel will only automatically be finished if all existing sources have been deinited.
///
/// - Returns: A new source for sending elements to the channel.
public mutating func makeAdditionalSource() -> sending Self {
.init(storage: self._storage)
}
/// Sends new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// - Parameter sequence: The elements to send to the channel.
/// - Returns: The result that indicates if more elements should be produced at this time.
@inlinable
public mutating func send<S>(
contentsOf sequence: consuming sending S
) throws -> SendResult where Element == S.Element, S: Sequence, Element: Copyable {
try self._storage.send(contentsOf: sequence)
}
/// Send the element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// - Parameter element: The element to send to the channel.
/// - Returns: The result that indicates if more elements should be produced at this time.
@inlinable
public mutating func send(_ element: consuming sending Element) throws -> SendResult {
try self._storage.send(contentsOf: CollectionOfOne(element))
}
/// Send new elements to the channel and provide a callback which will be invoked once more elements should be produced.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then `onProduceMore` will be invoked with
/// a `Result.failure`.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
/// - onProduceMore: The callback which gets invoked once more elements should be produced. This callback might be
/// invoked during the call to ``send(contentsOf:onProduceMore:)``.
@inlinable
public mutating func send<S>(
contentsOf sequence: consuming sending S,
onProduceMore: @escaping @Sendable (Result<Void, Error>) -> Void
) where Element == S.Element, S: Sequence, Element: Copyable {
do {
let sendResult = try self.send(contentsOf: sequence)
switch consume sendResult {
case .produceMore:
onProduceMore(Result<Void, Error>.success(()))
case .enqueueCallback(var callbackHandle):
callbackHandle.enqueueCallback(onProduceMore: onProduceMore)
}
} catch {
onProduceMore(.failure(error))
}
}
/// Sends the element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then `onProduceMore` will be invoked with
/// a `Result.failure`.
///
/// - Parameters:
/// - element: The element to send to the channel.
/// - onProduceMore: The callback which gets invoked once more elements should be produced. This callback might be
/// invoked during the call to ``send(_:onProduceMore:)``.
@inlinable
public mutating func send(
_ element: consuming sending Element,
onProduceMore: @escaping @Sendable (Result<Void, Error>) -> Void
) {
do {
let sendResult = try self.send(element)
switch consume sendResult {
case .produceMore:
onProduceMore(Result<Void, Error>.success(()))
case .enqueueCallback(var callbackHandle):
callbackHandle.enqueueCallback(onProduceMore: onProduceMore)
}
} catch {
onProduceMore(.failure(error))
}
}
#if compiler(>=6.2)
/// Send new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
@inlinable
public mutating nonisolated(nonsending) func send<S>(
contentsOf sequence: consuming sending S
) async throws where Element == S.Element, S: Sequence, Element: Copyable {
let syncSend: (sending S, inout Self) throws -> SendResult = { try $1.send(contentsOf: $0) }
let sendResult = try syncSend(sequence, &self)
switch consume sendResult {
case .produceMore:
return ()
case .enqueueCallback(let callbackToken):
let id = callbackToken._id
let storage = self._storage
try await withTaskCancellationHandler {
try await withUnsafeThrowingContinuation { continuation in
self._storage.enqueueProducer(
callbackToken: id,
continuation: continuation
)
}
} onCancel: {
storage.cancelProducer(callbackToken: id)
}
}
}
/// Send new element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - element: The element to send to the channel.
@inlinable
public mutating nonisolated(nonsending) func send(_ element: consuming sending Element) async throws {
let syncSend: (consuming sending Element, inout Self) throws -> SendResult = { try $1.send($0) }
let sendResult = try syncSend(element, &self)
switch consume sendResult {
case .produceMore:
return ()
case .enqueueCallback(let callbackHandle):
let id = callbackHandle._id
let storage = self._storage
try await withTaskCancellationHandler {
try await withUnsafeThrowingContinuation { continuation in
self._storage.enqueueProducer(
callbackToken: id,
continuation: continuation
)
}
} onCancel: {
storage.cancelProducer(callbackToken: id)
}
}
}
/// Send the elements of the asynchronous sequence to the channel.
///
/// This method returns once the provided asynchronous sequence or the channel finished.
///
/// - Important: This method does not finish the source if consuming the upstream sequence terminated.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
@inlinable
public mutating nonisolated(nonsending) func send<S>(contentsOf sequence: consuming sending S) async throws
where Element == S.Element, S: AsyncSequence, Element: Copyable, S: Sendable, Element: Sendable {
for try await element in sequence {
try await self.send(contentsOf: CollectionOfOne(element))
}
}
#else
/// Send new elements to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// first element of the provided sequence. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
@inlinable
public mutating func send<S>(
contentsOf sequence: consuming sending S
) async throws where Element == S.Element, S: Sequence, Element: Copyable {
let syncSend: (sending S, inout Self) throws -> SendResult = { try $1.send(contentsOf: $0) }
let sendResult = try syncSend(sequence, &self)
switch consume sendResult {
case .produceMore:
return ()
case .enqueueCallback(let callbackToken):
let id = callbackToken._id
let storage = self._storage
try await withTaskCancellationHandler {
try await withUnsafeThrowingContinuation { continuation in
self._storage.enqueueProducer(
callbackToken: id,
continuation: continuation
)
}
} onCancel: {
storage.cancelProducer(callbackToken: id)
}
}
}
/// Send new element to the channel.
///
/// If there is a task consuming the channel and awaiting the next element then the task will get resumed with the
/// provided element. If the channel already terminated then this method will throw an error
/// indicating the failure.
///
/// This method returns once more elements should be produced.
///
/// - Parameters:
/// - element: The element to send to the channel.
@inlinable
public mutating func send(_ element: consuming sending Element) async throws {
let syncSend: (consuming sending Element, inout Self) throws -> SendResult = { try $1.send($0) }
let sendResult = try syncSend(element, &self)
switch consume sendResult {
case .produceMore:
return ()
case .enqueueCallback(let callbackHandle):
let id = callbackHandle._id
let storage = self._storage
try await withTaskCancellationHandler {
try await withUnsafeThrowingContinuation { continuation in
self._storage.enqueueProducer(
callbackToken: id,
continuation: continuation
)
}
} onCancel: {
storage.cancelProducer(callbackToken: id)
}
}
}
/// Send the elements of the asynchronous sequence to the channel.
///
/// This method returns once the provided asynchronous sequence or the channel finished.
///
/// - Important: This method does not finish the source if consuming the upstream sequence terminated.
///
/// - Parameters:
/// - sequence: The elements to send to the channel.
@inlinable
public mutating func send<S>(contentsOf sequence: consuming sending S) async throws
where Element == S.Element, S: AsyncSequence, Element: Copyable, S: Sendable, Element: Sendable {
for try await element in sequence {
try await self.send(contentsOf: CollectionOfOne(element))
}
}
#endif
/// Indicates that the production terminated.
///
/// After all buffered elements are consumed the subsequent call to ``MultiProducerSingleConsumerAsyncChannel/next(isolation:)`` will return
/// `nil` or throw an error.
///
/// Calling this function more than once has no effect. After calling finish, the channel enters a terminal state and doesn't accept
/// new elements.
///
/// - Parameters:
/// - error: The error to throw, or `nil`, to finish normally.
@inlinable
public consuming func finish(throwing error: Failure? = nil) {
self._storage.finish(error)
}
}
}
@available(AsyncAlgorithms 1.1, *)
extension MultiProducerSingleConsumerAsyncChannel where Element: Copyable {
struct ChannelAsyncSequence: AsyncSequence {
@usableFromInline
final class _Backing: Sendable {
@usableFromInline
let storage: MultiProducerSingleConsumerAsyncChannel._Storage
init(storage: MultiProducerSingleConsumerAsyncChannel._Storage) {
self.storage = storage
self.storage.sequenceInitialized()
}
deinit {
self.storage.sequenceDeinitialized()
}
}
@usableFromInline
let _backing: _Backing
public func makeAsyncIterator() -> Self.Iterator {
.init(storage: self._backing.storage)
}
}
/// Converts the channel to an asynchronous sequence for consumption.
///
/// - Important: The returned asynchronous sequence only supports a single iterator to be created and
/// will fatal error at runtime on subsequent calls to `makeAsyncIterator`.
public consuming func elements() -> some (AsyncSequence<Element, Failure> & Sendable) {
ChannelAsyncSequence(_backing: .init(storage: self.storage))
}
}
@available(AsyncAlgorithms 1.1, *)
extension MultiProducerSingleConsumerAsyncChannel.ChannelAsyncSequence where Element: Copyable {
struct Iterator: AsyncIteratorProtocol {
@usableFromInline
final class _Backing {
@usableFromInline
let storage: MultiProducerSingleConsumerAsyncChannel._Storage
init(storage: MultiProducerSingleConsumerAsyncChannel._Storage) {
self.storage = storage
self.storage.iteratorInitialized()
}
deinit {
self.storage.iteratorDeinitialized()
}
}
@usableFromInline
let _backing: _Backing
init(storage: MultiProducerSingleConsumerAsyncChannel._Storage) {
self._backing = .init(storage: storage)
}
@inlinable
mutating func next() async throws -> Element? {
do {
return try await self._backing.storage.next(isolation: nil)
} catch {
throw error as! Failure
}
}
@inlinable
mutating func next(
isolation actor: isolated (any Actor)? = #isolation
) async throws(Failure) -> Element? {
do {
return try await self._backing.storage.next(isolation: actor)
} catch {
throw error as! Failure
}
}
}
}
@available(AsyncAlgorithms 1.1, *)
extension MultiProducerSingleConsumerAsyncChannel.ChannelAsyncSequence: Sendable {}
#endif
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension RangeReplaceableCollection {
/// Creates a new instance of a collection containing the elements of an asynchronous sequence.
///
+2 -3
View File
@@ -25,11 +25,10 @@ extension _ErrorMechanism {
_ = try _rethrowGet()
fatalError("materialized error without being in a throwing context")
}
internal func _rethrowGet() rethrows -> Output {
return try get()
}
}
extension Result: _ErrorMechanism { }
extension Result: _ErrorMechanism {}
+2 -1
View File
@@ -9,11 +9,12 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension SetAlgebra {
/// Creates a new set from an asynchronous sequence of items.
///
/// Use this initializer to create a new set from an asynchronous sequence
///
///
/// - Parameter source: The elements to use as members of the new set.
@inlinable
public init<Source: AsyncSequence>(_ source: Source) async rethrows where Source.Element == Element {
@@ -0,0 +1,19 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Async Algorithms open source project
//
// Copyright (c) 2024 Apple Inc. and the Swift project authors
// Licensed under Apache License v2.0 with Runtime Library Exception
//
// See https://swift.org/LICENSE.txt for license information
//
//===----------------------------------------------------------------------===//
/// A wrapper struct to unconditionally to transfer an non-Sendable value.
struct UnsafeTransfer<Element>: @unchecked Sendable {
let wrapped: Element
init(_ wrapped: Element) {
self.wrapped = wrapped
}
}
@@ -11,6 +11,7 @@
/// Creates an asynchronous sequence that concurrently awaits values from two `AsyncSequence` types
/// and emits a tuple of the values.
@available(AsyncAlgorithms 1.0, *)
public func zip<Base1: AsyncSequence, Base2: AsyncSequence>(
_ base1: Base1,
_ base2: Base2
@@ -20,8 +21,9 @@ public func zip<Base1: AsyncSequence, Base2: AsyncSequence>(
/// An asynchronous sequence that concurrently awaits values from two `AsyncSequence` types
/// and emits a tuple of the values.
@available(AsyncAlgorithms 1.0, *)
public struct AsyncZip2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: AsyncSequence, Sendable
where Base1: Sendable, Base1.Element: Sendable, Base2: Sendable, Base2.Element: Sendable {
where Base1: Sendable, Base1.Element: Sendable, Base2: Sendable, Base2.Element: Sendable {
public typealias Element = (Base1.Element, Base2.Element)
public typealias AsyncIterator = Iterator
@@ -71,4 +73,4 @@ public struct AsyncZip2Sequence<Base1: AsyncSequence, Base2: AsyncSequence>: Asy
}
@available(*, unavailable)
extension AsyncZip2Sequence.Iterator: Sendable { }
extension AsyncZip2Sequence.Iterator: Sendable {}
@@ -11,6 +11,7 @@
/// Creates an asynchronous sequence that concurrently awaits values from three `AsyncSequence` types
/// and emits a tuple of the values.
@available(AsyncAlgorithms 1.0, *)
public func zip<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>(
_ base1: Base1,
_ base2: Base2,
@@ -21,8 +22,17 @@ public func zip<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence
/// An asynchronous sequence that concurrently awaits values from three `AsyncSequence` types
/// and emits a tuple of the values.
public struct AsyncZip3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: AsyncSequence, Sendable
where Base1: Sendable, Base1.Element: Sendable, Base2: Sendable, Base2.Element: Sendable, Base3: Sendable, Base3.Element: Sendable {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncZip3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: AsyncSequence,
Sendable
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable
{
public typealias Element = (Base1.Element, Base2.Element, Base3.Element)
public typealias AsyncIterator = Iterator
@@ -37,7 +47,8 @@ public struct AsyncZip3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base
}
public func makeAsyncIterator() -> AsyncIterator {
Iterator(storage: .init(self.base1, self.base2, self.base3)
Iterator(
storage: .init(self.base1, self.base2, self.base3)
)
}
@@ -76,4 +87,4 @@ public struct AsyncZip3Sequence<Base1: AsyncSequence, Base2: AsyncSequence, Base
}
@available(*, unavailable)
extension AsyncZip3Sequence.Iterator: Sendable { }
extension AsyncZip3Sequence.Iterator: Sendable {}
@@ -10,22 +10,29 @@
//===----------------------------------------------------------------------===//
/// State machine for zip
@available(AsyncAlgorithms 1.0, *)
struct ZipStateMachine<
Base1: AsyncSequence,
Base2: AsyncSequence,
Base3: AsyncSequence
>: Sendable where
>: Sendable
where
Base1: Sendable,
Base2: Sendable,
Base3: Sendable,
Base1.Element: Sendable,
Base2.Element: Sendable,
Base3.Element: Sendable {
typealias DownstreamContinuation = UnsafeContinuation<Result<(
Base1.Element,
Base2.Element,
Base3.Element?
)?, Error>, Never>
Base3.Element: Sendable
{
typealias DownstreamContinuation = UnsafeContinuation<
Result<
(
Base1.Element,
Base2.Element,
Base3.Element?
)?, Error
>, Never
>
private enum State: Sendable {
/// Small wrapper for the state of an upstream sequence.
@@ -101,7 +108,9 @@ struct ZipStateMachine<
case .zipping:
// An iterator was deinitialized while we have a suspended continuation.
preconditionFailure("Internal inconsistency current state \(self.state) and received iteratorDeinitialized()")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received iteratorDeinitialized()"
)
case .waitingForDemand(let task, let upstreams):
// The iterator was dropped which signals that the consumer is finished.
@@ -110,7 +119,8 @@ struct ZipStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .finished:
@@ -159,7 +169,10 @@ struct ZipStateMachine<
)
}
mutating func childTaskSuspended(baseIndex: Int, continuation: UnsafeContinuation<Void, Error>) -> ChildTaskSuspendedAction? {
mutating func childTaskSuspended(
baseIndex: Int,
continuation: UnsafeContinuation<Void, Error>
) -> ChildTaskSuspendedAction? {
switch self.state {
case .initial:
// Child tasks are only created after we transitioned to `zipping`
@@ -179,7 +192,9 @@ struct ZipStateMachine<
upstreams.2.continuation = continuation
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)"
)
}
self.state = .waitingForDemand(
@@ -194,9 +209,7 @@ struct ZipStateMachine<
// already then we store the continuation otherwise we just go ahead and resume it
switch baseIndex {
case 0:
if upstreams.0.element == nil {
return .resumeContinuation(upstreamContinuation: continuation)
} else {
guard upstreams.0.element == nil else {
self.state = .modifying
upstreams.0.continuation = continuation
self.state = .zipping(
@@ -206,11 +219,10 @@ struct ZipStateMachine<
)
return .none
}
return .resumeContinuation(upstreamContinuation: continuation)
case 1:
if upstreams.1.element == nil {
return .resumeContinuation(upstreamContinuation: continuation)
} else {
guard upstreams.1.element == nil else {
self.state = .modifying
upstreams.1.continuation = continuation
self.state = .zipping(
@@ -220,11 +232,10 @@ struct ZipStateMachine<
)
return .none
}
return .resumeContinuation(upstreamContinuation: continuation)
case 2:
if upstreams.2.element == nil {
return .resumeContinuation(upstreamContinuation: continuation)
} else {
guard upstreams.2.element == nil else {
self.state = .modifying
upstreams.2.continuation = continuation
self.state = .zipping(
@@ -234,9 +245,12 @@ struct ZipStateMachine<
)
return .none
}
return .resumeContinuation(upstreamContinuation: continuation)
default:
preconditionFailure("Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)")
preconditionFailure(
"Internal inconsistency current state \(self.state) and received childTaskSuspended() with base index \(baseIndex)"
)
}
case .finished:
@@ -295,8 +309,9 @@ struct ZipStateMachine<
// Implementing this for the two arities without variadic generics is a bit awkward sadly.
if let first = upstreams.0.element,
let second = upstreams.1.element,
let third = upstreams.2.element {
let second = upstreams.1.element,
let third = upstreams.2.element
{
// We got an element from each upstream so we can resume the downstream now
self.state = .waitingForDemand(
task: task,
@@ -313,8 +328,9 @@ struct ZipStateMachine<
)
} else if let first = upstreams.0.element,
let second = upstreams.1.element,
self.numberOfUpstreamSequences == 2 {
let second = upstreams.1.element,
self.numberOfUpstreamSequences == 2
{
// We got an element from each upstream so we can resume the downstream now
self.state = .waitingForDemand(
task: task,
@@ -385,7 +401,8 @@ struct ZipStateMachine<
return .resumeContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .finished:
@@ -429,7 +446,8 @@ struct ZipStateMachine<
downstreamContinuation: downstreamContinuation,
error: error,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .finished:
@@ -460,9 +478,9 @@ struct ZipStateMachine<
mutating func cancelled() -> CancelledAction? {
switch self.state {
case .initial:
state = .finished
state = .finished
return .none
return .none
case .waitingForDemand(let task, let upstreams):
// The downstream task got cancelled so we need to cancel our upstream Task
@@ -471,7 +489,8 @@ struct ZipStateMachine<
return .cancelTaskAndUpstreamContinuations(
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .zipping(let task, let upstreams, let downstreamContinuation):
@@ -482,7 +501,8 @@ struct ZipStateMachine<
return .resumeDownstreamContinuationWithNilAndCancelTaskAndUpstreamContinuations(
downstreamContinuation: downstreamContinuation,
task: task,
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
upstreamContinuations: [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
)
case .finished:
@@ -524,7 +544,8 @@ struct ZipStateMachine<
// We also need to resume all upstream continuations now
self.state = .modifying
let upstreamContinuations = [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation].compactMap { $0 }
let upstreamContinuations = [upstreams.0.continuation, upstreams.1.continuation, upstreams.2.continuation]
.compactMap { $0 }
upstreams.0.continuation = nil
upstreams.1.continuation = nil
upstreams.2.continuation = nil
+35 -27
View File
@@ -9,8 +9,16 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
final class ZipStorage<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncSequence>: Sendable
where Base1: Sendable, Base1.Element: Sendable, Base2: Sendable, Base2.Element: Sendable, Base3: Sendable, Base3.Element: Sendable {
where
Base1: Sendable,
Base1.Element: Sendable,
Base2: Sendable,
Base2.Element: Sendable,
Base3: Sendable,
Base3.Element: Sendable
{
typealias StateMachine = ZipStateMachine<Base1, Base2, Base3>
private let stateMachine: ManagedCriticalState<StateMachine>
@@ -63,9 +71,9 @@ final class ZipStorage<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncS
switch action {
case .startTask:
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
// We are handling the startTask in the lock already because we want to avoid
// other inputs interleaving while starting the task
fatalError("Internal inconsistency")
case .resumeUpstreamContinuations(let upstreamContinuations):
// bases can be iterated over for 1 iteration so their next value can be retrieved
@@ -76,7 +84,7 @@ final class ZipStorage<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncS
continuation.resume(returning: .success(nil))
case .none:
break
break
}
}
@@ -304,30 +312,30 @@ final class ZipStorage<Base1: AsyncSequence, Base2: AsyncSequence, Base3: AsyncS
}
while !group.isEmpty {
do {
try await group.next()
} catch {
do {
try await group.next()
} catch {
// One of the upstream sequences threw an error
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let error,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .none:
break
}
group.cancelAll()
let action = self.stateMachine.withCriticalRegion { stateMachine in
stateMachine.upstreamThrew(error)
}
switch action {
case .resumeContinuationWithErrorAndCancelTaskAndUpstreamContinuations(
let downstreamContinuation,
let error,
let task,
let upstreamContinuations
):
upstreamContinuations.forEach { $0.resume(throwing: CancellationError()) }
task.cancel()
downstreamContinuation.resume(returning: .failure(error))
case .none:
break
}
group.cancelAll()
}
}
}
}
@@ -15,16 +15,32 @@ import AsyncSequenceValidation
extension XCTestCase {
func recordFailure(_ description: String, system: Bool = false, at location: AsyncSequenceValidation.SourceLocation) {
#if canImport(Darwin)
let context = XCTSourceCodeContext(location: XCTSourceCodeLocation(filePath: location.file.description, lineNumber: Int(location.line)))
let issue = XCTIssue(type: system ? .system : .assertionFailure, compactDescription: description, detailedDescription: nil, sourceCodeContext: context, associatedError: nil, attachments: [])
#if canImport(Darwin)
let context = XCTSourceCodeContext(
location: XCTSourceCodeLocation(filePath: location.file.description, lineNumber: Int(location.line))
)
let issue = XCTIssue(
type: system ? .system : .assertionFailure,
compactDescription: description,
detailedDescription: nil,
sourceCodeContext: context,
associatedError: nil,
attachments: []
)
record(issue)
#else
#else
XCTFail(description, file: location.file, line: location.line)
#endif
#endif
}
func validate<Test: AsyncSequenceValidationTest, Theme: AsyncSequenceValidationTheme>(theme: Theme, expectedFailures: Set<String>, @AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test, file: StaticString = #file, line: UInt = #line) {
@available(AsyncAlgorithms 1.0, *)
func validate<Test: AsyncSequenceValidationTest, Theme: AsyncSequenceValidationTheme>(
theme: Theme,
expectedFailures: Set<String>,
@AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test,
file: StaticString = #file,
line: UInt = #line
) {
var expectations = expectedFailures
var result: AsyncSequenceValidationDiagram.ExpectationResult?
var failures = [AsyncSequenceValidationDiagram.ExpectationFailure]()
@@ -61,16 +77,33 @@ extension XCTestCase {
XCTFail("Expected failure: \(expectation) did not occur.", file: file, line: line)
}
}
func validate<Test: AsyncSequenceValidationTest>(expectedFailures: Set<String>, @AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test, file: StaticString = #file, line: UInt = #line) {
@available(AsyncAlgorithms 1.0, *)
func validate<Test: AsyncSequenceValidationTest>(
expectedFailures: Set<String>,
@AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test,
file: StaticString = #file,
line: UInt = #line
) {
validate(theme: .ascii, expectedFailures: expectedFailures, build, file: file, line: line)
}
public func validate<Test: AsyncSequenceValidationTest, Theme: AsyncSequenceValidationTheme>(theme: Theme, @AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test, file: StaticString = #file, line: UInt = #line) {
@available(AsyncAlgorithms 1.0, *)
public func validate<Test: AsyncSequenceValidationTest, Theme: AsyncSequenceValidationTheme>(
theme: Theme,
@AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test,
file: StaticString = #file,
line: UInt = #line
) {
validate(theme: theme, expectedFailures: [], build, file: file, line: line)
}
public func validate<Test: AsyncSequenceValidationTest>(@AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test, file: StaticString = #file, line: UInt = #line) {
@available(AsyncAlgorithms 1.0, *)
public func validate<Test: AsyncSequenceValidationTest>(
@AsyncSequenceValidationDiagram _ build: (AsyncSequenceValidationDiagram) -> Test,
file: StaticString = #file,
line: UInt = #line
) {
validate(theme: .ascii, expectedFailures: [], build, file: file, line: line)
}
}
@@ -12,71 +12,115 @@
import _CAsyncSequenceValidationSupport
@resultBuilder
public struct AsyncSequenceValidationDiagram : Sendable {
@available(AsyncAlgorithms 1.0, *)
public struct AsyncSequenceValidationDiagram: Sendable {
public struct Component<T> {
var component: T
var location: SourceLocation
}
public struct AccumulatedInputs {
var inputs: [Specification] = []
}
public struct AccumulatedInputsWithOperation<Operation: AsyncSequence> where Operation.Element == String {
var inputs: [Specification]
var operation: Operation
}
public static func buildExpression(_ expr: String, file: StaticString = #file, line: UInt = #line) -> Component<String> {
public static func buildExpression(
_ expr: String,
file: StaticString = #file,
line: UInt = #line
) -> Component<String> {
Component(component: expr, location: SourceLocation(file: file, line: line))
}
public static func buildExpression<S: AsyncSequence>(_ expr: S, file: StaticString = #file, line: UInt = #line) -> Component<S> {
public static func buildExpression<S: AsyncSequence>(
_ expr: S,
file: StaticString = #file,
line: UInt = #line
) -> Component<S> {
Component(component: expr, location: SourceLocation(file: file, line: line))
}
public static func buildPartialBlock(first input: Component<String>) -> AccumulatedInputs {
return AccumulatedInputs(inputs: [Specification(specification: input.component, location: input.location)])
}
public static func buildPartialBlock<Operation: AsyncSequence>(first operation: Component<Operation>) -> AccumulatedInputsWithOperation<Operation> where Operation.Element == String {
public static func buildPartialBlock<Operation: AsyncSequence>(
first operation: Component<Operation>
) -> AccumulatedInputsWithOperation<Operation> where Operation.Element == String {
return AccumulatedInputsWithOperation(inputs: [], operation: operation.component)
}
public static func buildPartialBlock(accumulated: AccumulatedInputs, next input: Component<String>) -> AccumulatedInputs {
return AccumulatedInputs(inputs: accumulated.inputs + [Specification(specification: input.component, location: input.location)])
public static func buildPartialBlock(
accumulated: AccumulatedInputs,
next input: Component<String>
) -> AccumulatedInputs {
return AccumulatedInputs(
inputs: accumulated.inputs + [Specification(specification: input.component, location: input.location)]
)
}
public static func buildPartialBlock<Operation: AsyncSequence>(accumulated: AccumulatedInputs, next operation: Component<Operation>) -> AccumulatedInputsWithOperation<Operation> {
public static func buildPartialBlock<Operation: AsyncSequence>(
accumulated: AccumulatedInputs,
next operation: Component<Operation>
) -> AccumulatedInputsWithOperation<Operation> {
return AccumulatedInputsWithOperation(inputs: accumulated.inputs, operation: operation.component)
}
public static func buildPartialBlock<Operation: AsyncSequence>(accumulated: AccumulatedInputsWithOperation<Operation>, next output: Component<String>) -> some AsyncSequenceValidationTest {
return Test(inputs: accumulated.inputs, sequence: accumulated.operation, output: Specification(specification: output.component, location: output.location))
public static func buildPartialBlock<Operation: AsyncSequence>(
accumulated: AccumulatedInputsWithOperation<Operation>,
next output: Component<String>
) -> some AsyncSequenceValidationTest {
return Test(
inputs: accumulated.inputs,
sequence: accumulated.operation,
output: Specification(specification: output.component, location: output.location)
)
}
public static func buildBlock<Operation: AsyncSequence>(_ sequence: Component<Operation>, _ output: Component<String>) -> some AsyncSequenceValidationTest where Operation.Element == String {
public static func buildBlock<Operation: AsyncSequence>(
_ sequence: Component<Operation>,
_ output: Component<String>
) -> some AsyncSequenceValidationTest where Operation.Element == String {
let part1 = buildPartialBlock(first: sequence)
let part2 = buildPartialBlock(accumulated: part1, next: output)
return part2
}
public static func buildBlock<Operation: AsyncSequence>(_ input1: Component<String>, _ sequence: Component<Operation>, _ output: Component<String>) -> some AsyncSequenceValidationTest where Operation.Element == String {
public static func buildBlock<Operation: AsyncSequence>(
_ input1: Component<String>,
_ sequence: Component<Operation>,
_ output: Component<String>
) -> some AsyncSequenceValidationTest where Operation.Element == String {
let part1 = buildPartialBlock(first: input1)
let part2 = buildPartialBlock(accumulated: part1, next: sequence)
let part3 = buildPartialBlock(accumulated: part2, next: output)
return part3
}
public static func buildBlock<Operation: AsyncSequence>(_ input1: Component<String>, _ input2: Component<String>, _ sequence: Component<Operation>, _ output: Component<String>) -> some AsyncSequenceValidationTest where Operation.Element == String {
public static func buildBlock<Operation: AsyncSequence>(
_ input1: Component<String>,
_ input2: Component<String>,
_ sequence: Component<Operation>,
_ output: Component<String>
) -> some AsyncSequenceValidationTest where Operation.Element == String {
let part1 = buildPartialBlock(first: input1)
let part2 = buildPartialBlock(accumulated: part1, next: input2)
let part3 = buildPartialBlock(accumulated: part2, next: sequence)
let part4 = buildPartialBlock(accumulated: part3, next: output)
return part4
}
public static func buildBlock<Operation: AsyncSequence>(_ input1: Component<String>, _ input2: Component<String>, _ input3: Component<String>, _ sequence: Component<Operation>, _ output: Component<String>) -> some AsyncSequenceValidationTest where Operation.Element == String {
public static func buildBlock<Operation: AsyncSequence>(
_ input1: Component<String>,
_ input2: Component<String>,
_ input3: Component<String>,
_ sequence: Component<Operation>,
_ output: Component<String>
) -> some AsyncSequenceValidationTest where Operation.Element == String {
let part1 = buildPartialBlock(first: input1)
let part2 = buildPartialBlock(accumulated: part1, next: input2)
let part3 = buildPartialBlock(accumulated: part2, next: input3)
@@ -84,8 +128,15 @@ public struct AsyncSequenceValidationDiagram : Sendable {
let part5 = buildPartialBlock(accumulated: part4, next: output)
return part5
}
public static func buildBlock<Operation: AsyncSequence>(_ input1: Component<String>, _ input2: Component<String>, _ input3: Component<String>, _ input4: Component<String>, _ sequence: Component<Operation>, _ output: Component<String>) -> some AsyncSequenceValidationTest where Operation.Element == String {
public static func buildBlock<Operation: AsyncSequence>(
_ input1: Component<String>,
_ input2: Component<String>,
_ input3: Component<String>,
_ input4: Component<String>,
_ sequence: Component<Operation>,
_ output: Component<String>
) -> some AsyncSequenceValidationTest where Operation.Element == String {
let part1 = buildPartialBlock(first: input1)
let part2 = buildPartialBlock(accumulated: part1, next: input2)
let part3 = buildPartialBlock(accumulated: part2, next: input3)
@@ -94,17 +145,17 @@ public struct AsyncSequenceValidationDiagram : Sendable {
let part6 = buildPartialBlock(accumulated: part5, next: output)
return part6
}
let queue: WorkQueue
let _clock: Clock
public var inputs: InputList
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
public var clock: Clock {
_clock
}
internal init() {
let queue = WorkQueue()
self.queue = queue
@@ -112,4 +163,3 @@ public struct AsyncSequenceValidationDiagram : Sendable {
self._clock = Clock(queue: queue)
}
}
+41 -31
View File
@@ -11,22 +11,22 @@
import AsyncAlgorithms
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram {
public struct Clock {
public struct Clock: Sendable {
let queue: WorkQueue
init(queue: WorkQueue) {
self.queue = queue
}
}
}
public protocol TestClock: Sendable {
associatedtype Instant: TestInstant
var now: Instant { get }
func sleep(until deadline: Self.Instant, tolerance: Self.Instant.Duration?) async throws
}
@@ -34,80 +34,81 @@ public protocol TestInstant: Equatable {
associatedtype Duration
}
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram.Clock {
public struct Step: DurationProtocol, Hashable, CustomStringConvertible {
internal var rawValue: Int
internal init(_ rawValue: Int) {
self.rawValue = rawValue
}
public static func + (lhs: Step, rhs: Step) -> Step {
return .init(lhs.rawValue + rhs.rawValue)
}
public static func - (lhs: Step, rhs: Step) -> Step {
.init(lhs.rawValue - rhs.rawValue)
}
public static func / (lhs: Step, rhs: Int) -> Step {
.init(lhs.rawValue / rhs)
}
public static func * (lhs: Step, rhs: Int) -> Step {
.init(lhs.rawValue * rhs)
}
public static func / (lhs: Step, rhs: Step) -> Double {
Double(lhs.rawValue) / Double(rhs.rawValue)
}
public static func < (lhs: Step, rhs: Step) -> Bool {
lhs.rawValue < rhs.rawValue
}
public static var zero: Step { .init(0) }
public static func steps(_ amount: Int) -> Step {
return Step(amount)
}
public var description: String {
return "step \(rawValue)"
}
}
public struct Instant: CustomStringConvertible {
public typealias Duration = Step
let when: Step
public func advanced(by duration: Step) -> Instant {
Instant(when: when + duration)
}
public func duration(to other: Instant) -> Step {
other.when - when
}
public static func < (lhs: Instant, rhs: Instant) -> Bool {
lhs.when < rhs.when
}
public var description: String {
// the raw value is 1 indexed in execution but we should report it as 0 indexed
return "tick \(when.rawValue - 1)"
}
}
public var now: Instant {
queue.now
}
public var minimumResolution: Step {
.steps(1)
}
public func sleep(
until deadline: Instant,
tolerance: Step? = nil
@@ -115,7 +116,12 @@ extension AsyncSequenceValidationDiagram.Clock {
let token = queue.prepare()
try await withTaskCancellationHandler {
try await withUnsafeThrowingContinuation { continuation in
queue.enqueue(AsyncSequenceValidationDiagram.Context.currentJob, deadline: deadline, continuation: continuation, token: token)
queue.enqueue(
AsyncSequenceValidationDiagram.Context.currentJob,
deadline: deadline,
continuation: continuation,
token: token
)
}
} onCancel: {
queue.cancel(token)
@@ -123,16 +129,20 @@ extension AsyncSequenceValidationDiagram.Clock {
}
}
extension AsyncSequenceValidationDiagram.Clock.Instant: TestInstant { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram.Clock.Instant: TestInstant {}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncSequenceValidationDiagram.Clock.Instant: InstantProtocol { }
extension AsyncSequenceValidationDiagram.Clock.Instant: InstantProtocol {}
extension AsyncSequenceValidationDiagram.Clock: TestClock { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram.Clock: TestClock {}
@available(macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0, *)
extension AsyncSequenceValidationDiagram.Clock: Clock { }
extension AsyncSequenceValidationDiagram.Clock: Clock {}
// placeholders to avoid warnings
extension AsyncSequenceValidationDiagram.Clock.Instant: Hashable { }
extension AsyncSequenceValidationDiagram.Clock.Instant: Comparable { }
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram.Clock.Instant: Hashable {}
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram.Clock.Instant: Comparable {}
+19 -16
View File
@@ -9,14 +9,15 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram {
struct Failure: Error, Equatable { }
struct Failure: Error, Equatable {}
enum ParseFailure: Error, CustomStringConvertible, SourceFailure {
case stepInGroup(String, String.Index, SourceLocation)
case nestedGroup(String, String.Index, SourceLocation)
case unbalancedNesting(String, String.Index, SourceLocation)
var location: SourceLocation {
switch self {
case .stepInGroup(_, _, let location): return location
@@ -24,7 +25,7 @@ extension AsyncSequenceValidationDiagram {
case .unbalancedNesting(_, _, let location): return location
}
}
var description: String {
switch self {
case .stepInGroup:
@@ -36,14 +37,14 @@ extension AsyncSequenceValidationDiagram {
}
}
}
enum Event {
case value(String, String.Index)
case failure(Error, String.Index)
case finish(String.Index)
case delayNext(String.Index)
case cancel(String.Index)
var results: [Result<String?, Error>] {
switch self {
case .value(let value, _): return [.success(value)]
@@ -53,7 +54,7 @@ extension AsyncSequenceValidationDiagram {
case .cancel: return []
}
}
var index: String.Index {
switch self {
case .value(_, let index): return index
@@ -63,23 +64,26 @@ extension AsyncSequenceValidationDiagram {
case .cancel(let index): return index
}
}
static func parse<Theme: AsyncSequenceValidationTheme>(_ dsl: String, theme: Theme, location: SourceLocation) throws -> [(Clock.Instant, Event)] {
static func parse<Theme: AsyncSequenceValidationTheme>(
_ dsl: String,
theme: Theme,
location: SourceLocation
) throws -> [(Clock.Instant, Event)] {
var emissions = [(Clock.Instant, Event)]()
var when = Clock.Instant(when: .steps(0))
var string: String?
var grouping = 0
for index in dsl.indices {
let ch = dsl[index]
switch theme.token(dsl[index], inValue: string != nil) {
case .step:
if string == nil {
if grouping == 0 {
when = when.advanced(by: .steps(1))
} else {
guard grouping == 0 else {
throw ParseFailure.stepInGroup(dsl, index, location)
}
when = when.advanced(by: .steps(1))
} else {
string?.append(ch)
}
@@ -130,11 +134,10 @@ extension AsyncSequenceValidationDiagram {
emissions.append((when, .value(value, index)))
}
case .beginGroup:
if grouping == 0 {
when = when.advanced(by: .steps(1))
} else {
guard grouping == 0 else {
throw ParseFailure.nestedGroup(dsl, index, location)
}
when = when.advanced(by: .steps(1))
grouping += 1
case .endGroup:
grouping -= 1
@@ -9,6 +9,7 @@
//
//===----------------------------------------------------------------------===//
@available(AsyncAlgorithms 1.0, *)
extension AsyncSequenceValidationDiagram {
public struct ExpectationResult: Sendable {
public struct Event: Sendable {
@@ -18,7 +19,7 @@ extension AsyncSequenceValidationDiagram {
}
public var expected: [Event]
public var actual: [(Clock.Instant, Result<String?, Error>)]
func reconstitute<Theme: AsyncSequenceValidationTheme>(_ result: Result<String?, Error>, theme: Theme) -> String {
var reconstituted = ""
switch result {
@@ -39,9 +40,13 @@ extension AsyncSequenceValidationDiagram {
}
return reconstituted
}
func reconstitute<Theme: AsyncSequenceValidationTheme>(_ events: [Clock.Instant : [Result<String?, Error>]], theme: Theme, end: Clock.Instant) -> String {
var now = Clock.Instant(when: .steps(1)) // adjust for the offset index
func reconstitute<Theme: AsyncSequenceValidationTheme>(
_ events: [Clock.Instant: [Result<String?, Error>]],
theme: Theme,
end: Clock.Instant
) -> String {
var now = Clock.Instant(when: .steps(1)) // adjust for the offset index
var reconstituted = ""
while now <= end {
if let results = events[now] {
@@ -61,11 +66,11 @@ extension AsyncSequenceValidationDiagram {
}
return reconstituted
}
public func reconstituteExpected<Theme: AsyncSequenceValidationTheme>(theme: Theme) -> String {
var events = [Clock.Instant : [Result<String?, Error>]]()
var events = [Clock.Instant: [Result<String?, Error>]]()
var end: Clock.Instant = Clock.Instant(when: .zero)
for expectation in expected {
let when = expectation.when
let result = expectation.result
@@ -74,25 +79,25 @@ extension AsyncSequenceValidationDiagram {
end = when
}
}
return reconstitute(events, theme: theme, end: end)
}
public func reconstituteActual<Theme: AsyncSequenceValidationTheme>(theme: Theme) -> String {
var events = [Clock.Instant : [Result<String?, Error>]]()
var events = [Clock.Instant: [Result<String?, Error>]]()
var end: Clock.Instant = Clock.Instant(when: .zero)
for (when, result) in actual {
events[when, default: []].append(result)
if when > end {
end = when
}
}
return reconstitute(events, theme: theme, end: end)
}
}
public struct ExpectationFailure: Sendable, CustomStringConvertible {
public enum Kind: Sendable {
case expectedFinishButGotValue(String)
@@ -108,23 +113,23 @@ extension AsyncSequenceValidationDiagram {
case unexpectedValue(String)
case unexpectedFinish
case unexpectedFailure(Error)
case specificationViolationGotValueAfterIteration(String)
case specificationViolationGotFailureAfterIteration(Error)
}
public var when: Clock.Instant
public var kind: Kind
public var specification: Specification?
public var index: String.Index?
init(when: Clock.Instant, kind: Kind, specification: Specification? = nil, index: String.Index? = nil) {
self.when = when
self.kind = kind
self.specification = specification
self.index = index
}
var reason: String {
switch kind {
case .expectedFinishButGotValue(let actual):
@@ -159,7 +164,7 @@ extension AsyncSequenceValidationDiagram {
return "specification violation got failure after iteration terminated"
}
}
public var description: String {
return reason + " at tick \(when.when.rawValue - 1)"
}
Loaded 100 of 172 files, more files were not shown because too many files have changed in this diff. Show more