Motivation:
Swift 6.0 is no longer supported, we should bump the tools version and
remove it from our CI.
Modifications:
* Bump the Swift tools version to Swift 6.1
* Remove Swift 6.0 jobs where appropriate in main.yml, pull_request.yml
Result:
Code reflects our support window.
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
## Motivation
- Users need to be able to create HPACKHeaders to pass into library such
as GRPC v1
- To create them, without relying on SPM weirdness, users should be
importing NIOHPACK
- To import NIOHPACK without relying on SPM weirdness, users should be
depending on NIOHPACK in Package.swift
- Users currently cannot depend on NIOHPACK in Package.swift, because
NIOHPACK is not a product
## Changes
Make NIOHPACK a product
Motivation:
Swift 5.9 is no longer supported, we should bump the tools version and
remove it from our CI.
Modifications:
* Bump the Swift tools version to Swift 5.10
* Remove Swift 5.9 jobs where appropriate in main.yml, pull_request.yml
Result:
Code reflects our support window.
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
We added a couple of shims which were upstreamed to swift-nio and were
released in 2.82.0.
Modifications:
- Bump minimum verison of swift-nio
- Remove shims
Result:
Less code
Motivation:
The HTTP2StreamMultiplexer isn't `Sendable`, but it has public API for
creating streams which must operate from off of the `EventLoop`.
Modifications:
- Add a `SendableView` over the multiplexer which only provides the
methods for creating streams. These are safe to call from any isolation
domain.
Result:
Can create streams from off of the EL.
Motivation:
To increase concurrency safety the NIOPHACK module should compile
without warning under strict concurrency checking.
Modifications:
- Add explicit Sendable annotations
Result:
No concurrency warnings.
Enable MemberImportVisibility check on all targets. Use a standard
string header and footer to bracket the new block for ease of updating
in the future with scripts.
Motivation:
Standardised formatting makes the project cleaner to look at and work
on.
Modifications:
Setup swift-format with the same settings as Swift-NIO
Result:
Common formatting throughout
Motivation:
Only the last 3 versions of Swift are supported.
Modifications:
Update the Swift tools version in Package.swift.
Remove code which can now never be compiled.
Remove legacy docker configuration.
Result:
Only Swift 5.9+ supported.
Motivation:
Want to move away from legacy CI.
Modifications:
Add configuration and update code to use GitHub actions.
Result:
GitHub Actions can be used for basic CI.
Motivation:
Building swift-nio-http2 with Swift 5.8 fails due to a compiler error
when building swift-docc-plugin
(https://github.com/swiftlang/swift-docc-plugin/issues/94).
Modifications:
- Adjust the dependency version of swift-docc-plugin from `from:
"1.0.0"` to `"1.0.0"..<"1.4.0"`.
Result:
Building swift-nio-http2 with Swift 5.8 will now be successful.
We previously developed an async API for NIO HTTP/2 which was guarded under SPI. Now that the swift-nio async API is released we can reintroduce this code promoted to SPI.
This change introduces:
* `AsyncStreamMultiplexer` - an async variant of the HTTP/2 stream multiplexer which can be used to create outbound streams and provide access to an async sequence (`NIOHTTP2AsyncSequence`) of inbound streams
* New pipeline configuration functions (e.g. `configureAsyncHTTP2Pipeline`) to support the new async mode
* Bump minimum Swift version to 5.7
Motivation:
Now that Swift 5.9 is GM we should update the supported versions and
remove 5.6
Modifications:
* Update `Package.swift`
* Remove `#if swift(>=5.7)` guards
* Delete the 5.6 docker compose file and make a 5.10 one
* Update 5.10 alloc limits
* Update docs
Result:
Remove support for Swift 5.6, add 5.10
* Update to latest NIO with updated protocol negotiation API
Signed-off-by: Si Beaumont <beaumont@apple.com>
* fixup: Also replace deprecated get().waitForFinalResult()
Signed-off-by: Si Beaumont <beaumont@apple.com>
---------
Signed-off-by: Si Beaumont <beaumont@apple.com>
# Motivation
In my latest PR in NIO, I removed a bunch of API from the bootstrap methods to make the API easier to digest and reduce a number of duplicated code issues.
# Modification
This PR adopts the latest NIO changes and adapts the APIs in H2. More specifically, it removes the `NIOAsyncChannel` convenience methods and makes sure that everything aligns nicely with the proposes bootstrap APIs.
# Result
We now have the APIs in place to bootstrap a protocol negotiated H1/H2 server.
* Store HTTP/2 inbound streams as async sequence
Motivation:
In preparation for exposing APIs which surface HTTP/2 connections and
streams using structured concurrency this PR introduces a store for
inbound HTTP/2 streams.
Modifications:
Define types and methods for storing generic types which wrap `Channels`
corresponding to inbound HTTP/2 streams in an async sequence.
Result:
The new types are not yet exposed, this work introduces part of the
framework for future functionality.
* Expose (as spi) helpers for H2 async pipelines
Motivation:
This PR is the first step in exposing APIs which surface HTTP/2 connections and
streams using structured concurrency. This PR exposes the most abstract
spelling of this concept, assuming no particular types/forms for the
types involved in the stream channel types.
Modifications:
* Store a generic `streamInitializer` on the common inbound initializer
which is used to initialize inbound streams before yielding them to
the continuation of streams.
* Expose pipeline configuration functions which assume HTTP/2 but
nothing else about sream channel types.
* Provide internal functions for creating streams without configuring
them. Configuring leads to activation so it can be helpful to allow
that step to be performed manually once any provided initialization
closures have beenn executed.
Result:
Adopters of the new SPI should be able to create outbound and deal with
inbound HTTP/2 stream channels using async streams.
Outside of SPI there should be no changes.
Motivation:
Less expensive stream multiplexing. To implement direct stream multiplexing (with an incomplete public API) to break up delivery and allow testing.
Modifications:
* Factor out common inbound stream multiplexer functionality from `HTTP2StreamMultiplexer` into `CommonInboundStreamMultiplexer` (first commit).
* Direct stream multiplexing is implemented as `InlineStreamMultiplexer`. It implements both the `HTTP2InboundStreamMultiplexer` and `HTTP2OutboundStreamMultiplexer` protocols.
* We expose `NIOHTTP2Hander.StreamMultiplexer` as a `public` view over the new multiplexer to allow stream creation.
* Adds various `propagate...` functions to relay signals such as read complete to the child channels held by the inline stream multiplexer.
* Adds a stub for `HTTP2StreamDelegate` which will be completed in an upcoming PR. It will be responsible for surfacing stream open/close information.
* Adds conveniences to `NIOHTTP2Handler` for setting up a handler using the inline multiplexer
* Duplicate existing tests and modify to use the inline stream multiplexer where appropriate. `SimpleClientServerFramePayloadStreamTests.swift` and `HTTP2FramePayloadStreamMultiplexerTests.swift` were copied and modified
Result:
Default behavior should be unchanged at this time.
Motivation
Documentation is nice, and we can help support users by providing useful
clear docs.
Modifications
Add Docc to 5.6 and later builds
Make sure symbol references work
Add overview docs
Result
Nice rendering docs
As outlined in a [Swift forums post in November ’21](https://forums.swift.org/t/swiftnio-swift-version-support/53232), SwiftNIO will only support the latest non-patch Swift release and the 2 immediately prior non-patch versions.
- drop support for Swift 5.2 and 5.3.
- update CI for Swift 5.4 to run on bionic instead of focal to ensure that we still test bionic.
Motivation
The HTTP2FrameDecoder is a complex object that was written early in the
development of swift-nio-http2. Its logical flow is complex, and it
hasn't been meaningfully rewritten in quite some time, so it's difficult
to work with and understand.
Annoyingly, some bugs have crept in over the years. Because of the
structure of the code it can be quite difficult to understand how the
parser actually works, and fixing a given issue can be difficult.
This patch aims to produce a substantial change to the HTTP2FrameDecoder
to make it easier to understand and maintain in the long run.
Modifications
This patch provides a complete rewrite of HTTP2FrameDecoder. It doesn't
do this by having a ground-up rewrite: instead, it's more like a
renovation, with the general scaffolding kept. The rewrite was performed
incrementally, keeping the existing test suite passing and writing new
tests when necessary.
The following major changes were made:
1. New states and edges were added to the state machine to handle
padding.
Prior to this change, padding was handled as part of frame payload
decoding. This is not totally unreasonable, but it dispersed padding
management widely and made it easy to have bugs slip in. This patch
replaces this with a smaller set of locations.
Padding is now handled in two distinct ways. For HEADERS and
PUSH_PROMISE frames, trailing padding is still stripped as part of
frame payload decode, but it's done so generically, and the padding
bytes are never exposed to the individual frame parser. For DATA,
there is a new state to handle trailing padding removal, which
simplifies the highly complex logic around synthesised data frames.
For all frames, the leading padding byte is handled by a new
dedicated state which is used unconditionally, instead of attempting
to opportunistically strip it. This simplifies the code flow.
As a side benefit, this change means we can now accurately report the
padding used on HEADERS and PUSH_PROMISE frames, even when they are
part of a CONTINUATION sequence.
2. The synthesised DATA frame logic has been substantially reworked.
With the removal of the padding logic from the state, we now know
that so long as we have either got a byte of data to emit _or_ the
DATA frame is zero length, we will always emit a frame. This has made
it simpler to understand the control flow when synthesising DATA
frames.
3. The monolithic state switch has been refactored into per-state
methods.
This helps manage the amount of state that each method can see, as
well as to logically split them up. In addition, it allows us to
recast state transformations as (fairly) pure functions.
Additionally, this allowed the larger methods to be refactored with
smaller helpers that are more obviously correct.
4. The frame payload parsers have been rewritten.
The main goal here was to remove preflight length checks and unsafe
code. The preflight length checks cause trouble when they disagree
with the parsing code, so we now rely on the parsing code being
correct with regard to length.
Relatedly, we previously had two separate places where we
communicated length: a frame header length and a ByteBuffer length.
This was unnecessary duplication of information, so we instead use a
ByteBuffer slice to manage the length. This ensures that we cannot
over-parse a message.
Finally, in places that used unsafe code or separate integer reads,
we have refactored to stop using that unsafe code and to use combined
integer reads.
5. Extraneous serialization code has been extracted.
The HTTP2FrameEncoder was unnecessarily in this file, which took a
large file and made it larger. I moved this out.
Result
The resulting parser is clearer and safer. Complex logic has been broken
out into smaller methods with less access to global data. The code
should be generally clearer.
Motivation:
With NIO 2.32.0 we broke the core NIO module up into modules that split
apart the POSIX layer and the core abstractions. As a result, this
package no longer needs to express a hard dependency on the POSIX layer.
Modifications:
- Rewrote imports of NIO to NIOCore.
- Added NIOEmbedded imports where necessary in tests.
- Note that the main modules still depend on NIO, which is necessary
for backwards-compatibility reasons. This dependency is unused.
Result:
No need to use NIOPosix.
### Motivation:
`removeHandlers(channel:)` was deprecated in NIO 2.32.0.
### Modifications:
- Raise minimum required `NIO` version to 2.32.0
- Use `removeHandlers(pipeline:)`
### Result:
We don't use deprecated API.
Motivation:
SwiftNIO 2.27.0 added support for synchronous channel options allowing
callers to get options synchronously -- saving a future allocation -- if
they know they're on the correct event loop. 'HTTP2StreamChannel' should
support this.
Modifications:
- Add 'HTTP2StreamChannel.SynchronousOptions' with conformance to
'NIOSynchronousChannelOptions'
Result:
Callers can get and set options synchronously on 'HTTP2StreamChannel'
Motivation:
Performance benchmarking wants accurate numbers, and a wide range of
possible benchmarks. One way to get somewhat more accurate numbers is to
use a tool like Cachegrind to measure the instruction counts of a
program. Cachegrind allows us to isolate a number of noise-introducing
effects, as "number of executed instructions" is a much more stable
number than how long a program takes to run.
When running under cachegrind we want to change the way our performance
benchmarks work. Instead of running lots of times to try to average out
noise, we only want to run once to save ourselves time, as the benchmark
itself should be drastically more stable.
While we're here, let's add some much more isolated benchmarks than our
current "make lots of streams" benchmark. This is basically the same
logic, but it does less of NIO and so makes the impact of NIOHTTP2
substantially more prominent.
Modifications:
- Added a cachegrind build mode to the benchmarks.
- Added new benchmarks and alloc counters.
Result:
More info
* Updated to use SwiftNIO 1.10+ to gain access to HTTPHeaders internals.
* Add pure-Swift HTTP/2 protocol implementation.
Motivation:
Currently we use nghttp2 for our basic protocol implementation. This is
plenty workable, but provides us a number of limitations. It adds
complexity to our build system, provides us with external dependencies we
have little control over, limits our users' ability to express low-level
protocol semantics if they so wish, causes us to heap-allocate like crazy over
the lifetime of a connection, and brings a world of pain around reentrancy.
This set of limitations has motivated a desire to move away from using
nghttp2. This is a part of that dream.
Modifications:
- Add pure Swift protocol implementations.
- Test them.
Results:
Another step towards protocol independence.
* Initial basic parser code.
* Adding unit tests.
* Add NIOHTTP2FlowControlHandler
Motivation:
One of the things that nghttp2 used to do for us was to handle buffering
DATA frames until flow control allowed them to go out. For users that
want that behaviour, we need to replace the nghttp2 code with a new
ChannelHandler that can do this work.
Modifications:
- Added NIOHTTP2FlowControlHandler.
Result:
Users will have the same feature set as they did with nghttp2.
* Generate Linux test information.
* Documentation comment.
* Updated new tests from master to use new payload types.
* Updated following review.
Notable changes:
- Implemented unit tests for the ALTSVC (0xa) and ORIGIN (0xc) frame types.
- Additional DATA frames are synthesized to wrap individual payload chunks as they arrive. If the payload of a DATA frame with 4MB data arrives in 4 separate 1MB chunks, the decoder will emit 4 DATA frames, with only the last containing the END_STREAM flag (if that was set on the incoming frame).
- HEADERS / PUSH_PROMISE frames and any following CONTINUATION frames are now handled properly: header fragments from all frames are gathered up into a single buffer. Once a frame with END_HEADERS is encountered, the resultant accumulated buffer is sent for decoding along with the flags and stream ID of the original HEADERS / PUSH_PROMISE frame.
- HPACK encoders/decoders now live in the frame encoder/decoder types, rather than being attached to a stream.
* Updated Linux tests.
* Headers placed in relevant files.
* Add pure-Swift HTTP/2 protocol implementation.
Motivation:
Currently we use nghttp2 for our basic protocol implementation. This is
plenty workable, but provides us a number of limitations. It adds
complexity to our build system, provides us with external dependencies we
have little control over, limits our users' ability to express low-level
protocol semantics if they so wish, causes us to heap-allocate like crazy over
the lifetime of a connection, and brings a world of pain around reentrancy.
This set of limitations has motivated a desire to move away from using
nghttp2. This is a part of that dream.
Modifications:
- Add pure Swift protocol implementations.
- Test them.
Results:
Another step towards protocol independence.
* Revamped frame decoder state machine and API. Now uses separate input and output methods to manage multiple frames in a single buffer.
* Updated Linux tests.
* Bumped swift-nio version to 1.12.0 and updated `HTTPHeaders` -> `HPACKHeaders` conversion initializer to directly copy the buffer and map the header name/value indices.
* Set up HTTP2Parser in handlerAdded if the channel had been previously activated. (#36)
By setting up the handler in the handlerAdded hook, HTTP2Parser supports
being added dynamically in a channelRead event, after the channel has sent
the channelActive event.
(cherry picked from commit 88bd6d369a)
* Minor tweaks to get the tests passing
* Initial integration here
* Minor clearing up of the handler
* Get the tests compiling
* Fix header issues.
We now pass the frame parser to the various setting operations to let it
update the state. We also fixed some issues with specific settings.
* Remove ngthttp2-based impl, clean up stream ID
* Tests with new HTTP2StreamID
* Simple test cleanup
* Get remaining non-core tests passing
* Allow GOAWAY even when fully quiesced
* We don't buffer outbound frames any more
* Emit StreamClosed events on GoAway frame
* Notify stream creation and closure
* This test started failing
* Goodbye, C code! 🎉
* Send PING ACK automatically
* Updated based on PR feedback.
* Correctly handle 1XX responses
* Add SETTINGS_MAX_FRAME_SIZE support to flow controller
* Correctly account for flow control
* Define ConcurrentStreamsHandler
* Prepare for signalling connection state changes
* Better state change message propagation
* Refactor to use result with effect everywhere
* Don't lose flow control info on stream error
* Add basic flow control management to stream multiplexer
* Better pipeline in sample server
* Clean up some reentrancy concerns
* Move to Swift 5 and NIO 2
* Move to new Swift 5 for CI
* Allow wider range of choices of years
* Catch up to NIO 2 again
Motivation:
We never supported Swift 4.0 in this repository and Swift 4.0 is quite a
pain mostly because of the Unsafe*Pointer API differences.
Changes:
Require 4.1
Result:
fewer lies, fixes#17
Motivation:
This is the beginning of removing the dependency on `nghttp2`.
Modifications:
I've added a new library target, `NIOHPACK`, which contains all the necessary pieces to implement HPACK encoding as specified in RFC 7541. There are unit tests exercising all the code and some additional edge cases around integer encoding, along with some performance tests for the Huffman encode/decode process. The latter currently takes longer to decode than to encode, which is worrying, but this is only really noticeable when encoding > 16KB strings as a rule; therefore I've decided not to pull out the optimization hammer just yet. Included in the unit tests are the inputs and outputs from all the examples in RFC 7541 and RFC 7540.
I've tried to match the coding style from the repo, but some of my own idioms may have crept in here & there. Please let me know if you see anything wrong in that regard.
Currently the static header table and Huffman encoding tables are implemented as Swift `let`s. For the static header table that's ok (only 62 entries), but the Huffman table is huge. I'm debating whether it's worth having that in a plain C file somewhere, but I'm not sure what the tradeoffs are vs. performance. Right now it seems to only affect compilation time, but I may be wrong.
Result:
In general, nothing will change: nothing in NIOHTTP2 is using this library yet. That part is what I'm working on now.
Motivation:
When closing a stream channel, it must remove its channel handlers.
Modifications:
Call ChannelCore.removeHandlers on shutdown.
Result:
Handlers are removed on channel shutdown.