Motivation:
Complete enablement of all standard GitHub Actions workflows.
Modifications:
Describe the modifications you've done.
Result:
After your change, what will change.
Motivation:
Want to move away from legacy CI.
Modifications:
Add configuration and update code to use GitHub actions.
Result:
GitHub Actions can be used for basic CI.
Motivation:
We're no longer maintaining this list as it can be trivially obtained
from git history.
Modifications:
Remove script for generating contributors list.
Result:
One less script to maintain
In scenarios, in which we want to send lots of small HTTP/2 requests, we want to reuse a single HPACKHeaders object as often as we can. To support this we should have a removeAll(keepingCapacity:) method.
Motivation:
Now Swift 5.8 is released, we are dropping support for 5.5, in line with the usual support
Modifications:
- Remove `Package@swift-5.5.swift`.
- Remove `LinuxMain.swift` and `generate_linux_tests.rb` script.
- Remove use of `generate_linux_tests.rb` from CI check.
- Update README.
Result:
No functional change, but Swift 5.5 is no longer supported.
Motivation:
Soundness check currently fails for copyright statements including 2023
as the year
Modifications:
add '3' to the checking regex
Result:
Soundness checks should not fail for documents created this year
Co-authored-by: Cory Benfield <lukasa@apple.com>
Motivation:
Docs are generated by and hosted on the Swift Package Index. We no
longer need the script to generate docs via Jazzy.
Modifications:
- Remove the generate_docs script
- Remove Jazzy from the Dockerfile but keep Ruby; it's used for
generating test manifests.
- Remove SwiftFormat from the Dockerfile; we don't use it.
Result:
Fewer unused things.
Motivation:
SwiftNIO periodically drops support for older Swift versions. Now that 5.7 has been released, 5.4 will be dropped.
Modifications:
- Remove 5.4 specific Package.swift and docker-compose
- Update the 5.7 docker-compose to use the released 5.7 and move from focal (2004) to jammy (2204)
- Fix tools version in Package@swift-5.5.swift
- Update docs
Results:
Minimum Swift version is 5.5
Motivation
Documentation is nice, and we can help support users by providing useful
clear docs.
Modifications
Add Docc to 5.6 and later builds
Make sure symbol references work
Add overview docs
Result
Nice rendering docs
As outlined in a [Swift forums post in November ’21](https://forums.swift.org/t/swiftnio-swift-version-support/53232), SwiftNIO will only support the latest non-patch Swift release and the 2 immediately prior non-patch versions.
- drop support for Swift 5.2 and 5.3.
- update CI for Swift 5.4 to run on bionic instead of focal to ensure that we still test bionic.
### Motivation:
SPM has built in functionality to check the API of modules against a target git treeish. We can use this to simplify our `check_no_api_breakages.sh` script. Closes https://github.com/apple/swift-nio/issues/1239
### Modifications:
This PR, exchanges the direct calls to Swift's API checker with the new SPM `diagnose-api-breaking-changes` tool. This allows us to get rid of the manual module parsing, build invocations and result comparisons.
### Result:
We are now using SPMs `diagnose-api-breaking-changes` to check for breaking changes.
* Lower allocation limits
Our recent fixes reduced our allocation count, which is good.
* fix code soundness check
motivation: code should be sound!
changes:
* update one case of inappropriate langauge
* update header years check regex
(cherry picked from commit a89e8ae4e0)
Co-authored-by: tom doron <tomer@apple.com>
Motivation:
The docs haven't been generated for a while.
See also: https://github.com/apple/swift-nio/pull/2013
Modifications:
- Pass `--spm` flag to sourcekitten
- Update source location of sourcekitten
- Fix link to NIO docs
Result:
Docs build.
Motivation:
With NIO 2.32.0 we broke the core NIO module up into modules that split
apart the POSIX layer and the core abstractions. As a result, this
package no longer needs to express a hard dependency on the POSIX layer.
Modifications:
- Rewrote imports of NIO to NIOCore.
- Added NIOEmbedded imports where necessary in tests.
- Note that the main modules still depend on NIO, which is necessary
for backwards-compatibility reasons. This dependency is unused.
Result:
No need to use NIOPosix.
From: https://github.com/apple/swift-nio-extras/pull/141
Motivation:
The combination of Ruby and Jazzy are throwing up issues in CI on older
Ubuntu versions.
Modifications:
- Only install Jazzy on focal
- Fix doc generation script to use more recent source-kitten syntax
Results:
- CI doesn't complain
- Docs get built
Motivation:
SwiftNIO recently dropped support for Swift 5.0 and Swift 5.1 but our
build_podspec.sh script doesn't reflect this change. Additionally the
list_topsorted_dependencies.sh script always assumed named depenencies
when listing module depenencies. Since our tools version has now been
bumped to 5.2 that's no longer the case and the script produces a bunch
of nulls which throws the build_podspec.sh script off course.
Modifications:
- Bump the Swift version in build_podspec.sh to 5.2
- Include "product" dependencies when listing module dependencies in
list_topsorted_dependencies.sh
Result:
Podspecs are created as expected
Motivation:
Run the tests on Android.
Modifications:
- Enable the tests on Android.
- Replace '/tmp' with 'FileManager.default.temporaryDirectory'.
Result:
All the tests pass on Android AArch64 and x86_64.
Motivation:
Fuzzing is great, we should have a good sensible place to centralise a
harness.
Modifications:
- Add a fuzzing harness.
Result:
There's a common way to do fuzzing.
Motivation:
The OutboundFlowControlBuffer is responsible for managing DATA and
HEADERS frames for all streams in order to ensure that we obey HTTP/2
flow control rules, even when users don't. This object is therefore on
the data path for the vast majority of frames, and is extremely
performance sensitive.
Unfortunately, the current algorithm has some nasty performance cliffs.
Right now it has the awkward property of making flushes take an amount
of time linear in the number of streams on the connection, not the
number of streams that had had frames sent on them. The result is that
connections with highly parallel numbers of streams see extremely poor
performance when flushing, a problem that is exacerbated if other parts
of the HTTP/2 stack fail to coalesce those flushes.
We can clean this up in a few ways, but the easiest way is to just keep
track of which streams have done I/O since the last flush. While we're
there, we can make a few other optimisations to ensure we only flush
data on streams we are actually processing, to further reduce the risk
of wasting our time processing unnecessary work.
Modifications:
- Add a Set to keep track of where we've done writes since the last
flush.
- Streams whose flow control window goes to zero stop being writable.
- Cheaper and more accurate calculation of zero-length writes, which
avoids the possibility of awkward stalls with zero-length DATA frames
and END_STREAM.
- Stop doing math about the number of flushed bytes, we don't use that
information any more.
Results:
An approximately 10% performance gain on the 100 concurrent streams
server-only benchmark. The relative performance gain shifts with the
number of active streams, but it should be positive in all cases.
Motivation:
We support watchOS 6+ with SwiftNIO Transport Services; as such we should
include watchOS as a deployment target for our CocoaPods.
Modifications:
- Add a watchOS deployment target to `build_podspecs.sh`
Result:
Users can deploy to watchOS 6+ with CocoaPods.
Motivation:
I have suspicions that the allocation counter tests in NIOHTTP2 don't
fail properly, so let's bring the scaffolding up to date.
Modifications:
Update the few local files to the most recent NIO versions.
Result:
Smaller diffs to NIO, maybe better infrastructure?
Motivation:
As part of #214 we need to introduce new pipeline helpers to configure
the channel pipeline with the new handlers. This PR adds those helpers
and deprecates the old paths.
Modifications:
- Add pipeline helpers with the new streamID-less initializer
- Switch old pipeline helpers which provided codecs to use the new path
and codecs
- Duplicate SimpleClientServerTests and ConfiguringPipelineTests to add
frame payload based counterparts
- Deprecate older helpers and multiplexer init, delete tests which don't
hit deprecated paths (note: these still exist in the payload based
counterparts)
- Update Bench1Conn10kRequests to be payload based
Result:
- More 'FramePayload' based Tests
- Paths for creating an 'HTTP2Frame' based stream via
'HTTP2StreamMultiplexer' are now deprecated
Motivation:
To test deprecated functionality without a bunch of warnings, tests must
be marked as deprecated. This also needs to be bubbled up to anywhere
calling that test, such as in the generated linux test manifests.
Our script to generate the manifests currently doesn't support this and
as part of #214 we'll need to deprecate a few things while keeping their
tests.
Modifications:
- Pull in the latest version of the `generate_linux_tests.rb` script
from SwiftNIO.
- Re-run the script, since it includes some formatting changes too.
Result:
- When we deprecate functionality, we can deprecate the tests without
the generate manifests emitting warnings.
Motivation:
Thread Sanitizer will find bugs which might be missed otherwise.
Modifications:
When running docker image for 1604/5.1 tests and h2spec are both compiled with --sanitize=thread
Result:
Tests are run with thread sanitizer turned on.
Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
Motivation:
The build_podspec.sh script generates a podspec which requires exact
versions of its dependencies. This very quickly turns into unresolvable
dependency graphs.
Modifications:
- NIO version passed to script must be in the format `MAJOR.MINOR`
- Podspec dependencies are now '>= MAJOR.MINOR', '< MAJOR+1'
Result:
Looser version requirements for podspecs
Motivation:
When a window update event is received in the stream channel we notify
the inbound window manager. Doing so may emit a WINDOW_UPDATE frame
to indicate the remote may send us more DATA. However, this happens even
if there are inbound frames waiting to be delivered to the channel. This
may lead to the remote sending us more DATA which we have to buffer
until it is read.
Modifications:
- Update the inbound window manager to track the number of bytes
buffered in the stream channel
- Account for buffered bytes when receiving a new window size event
- Possibly emit WINDOW_UPDATE after unbuffering inbound frames into the
stream channel's pipeline
Result:
The timing of WINDOW_UPDATE frames is better aligned with when frames
are actually read in a stream.
motivation: docs scripts is broken on selinux due to restriction on ssh-agent
changes:
* change the jazzy workspace to a directory accessible by the CI node so we can push from the agent instead of from the docker container
* dont remove older docs when pushing new ones
* add version information and source links
motivation: prepare to run performance tests in ci
changes:
add performance-test docker-compose task
add script that can parse the performance test suite results and compare between two result-sets
add generic shell docker-compose task
Motivation:
Allocation tests are good :).
Modifications:
Add two allocations tests and NIO's integration test framework.
Result:
Hopefully fewer allocations in the future. And better ways to check what
allocations are happening.
Motivation:
Allowing readers of the API documentation to drill into the implementation of documented declarations can be educational and helpful for debugging, among several other reasons.
Modifications:
Add --github-file-prefix appending the current version in order to keep stable links. This means that documentation generated from any non-release revision may resolve incorrectly. This can be refined in the doc generation script later if it is deemed problematic by resolving to the commit sha if it doesn't exactly align with the version.
Result:
Users will be able to click a 'Show on GitHub' for all documented API declarations.
Motivation:
RFC 7540 requires that compliant specifications emit errors if content-length has been incorrectly
set. We should have this behaviour by default, though we should allow it to be disabled if necessary.
Modifications:
- Added content-length checking.
- Added flag to disable content-length checking.
Result:
Better RFC 7540 compliance.
motivation: publish API docs
changes:
* amend docker file to include jazzy
* add doc generation script based on what we have in swift-nio and swift-log
Motivation:
When we get around to it we should aim for our example server to pass
h2spec with flying colours. We aren't there yet, but it's helpful to
be able to run h2spec easily.
Modifications:
Add a docker-compose recipe for running h2spec.
Result:
It'll be easier to run h2spec.
* Updated to use SwiftNIO 1.10+ to gain access to HTTPHeaders internals.
* Add pure-Swift HTTP/2 protocol implementation.
Motivation:
Currently we use nghttp2 for our basic protocol implementation. This is
plenty workable, but provides us a number of limitations. It adds
complexity to our build system, provides us with external dependencies we
have little control over, limits our users' ability to express low-level
protocol semantics if they so wish, causes us to heap-allocate like crazy over
the lifetime of a connection, and brings a world of pain around reentrancy.
This set of limitations has motivated a desire to move away from using
nghttp2. This is a part of that dream.
Modifications:
- Add pure Swift protocol implementations.
- Test them.
Results:
Another step towards protocol independence.
* Initial basic parser code.
* Adding unit tests.
* Add NIOHTTP2FlowControlHandler
Motivation:
One of the things that nghttp2 used to do for us was to handle buffering
DATA frames until flow control allowed them to go out. For users that
want that behaviour, we need to replace the nghttp2 code with a new
ChannelHandler that can do this work.
Modifications:
- Added NIOHTTP2FlowControlHandler.
Result:
Users will have the same feature set as they did with nghttp2.
* Generate Linux test information.
* Documentation comment.
* Updated new tests from master to use new payload types.
* Updated following review.
Notable changes:
- Implemented unit tests for the ALTSVC (0xa) and ORIGIN (0xc) frame types.
- Additional DATA frames are synthesized to wrap individual payload chunks as they arrive. If the payload of a DATA frame with 4MB data arrives in 4 separate 1MB chunks, the decoder will emit 4 DATA frames, with only the last containing the END_STREAM flag (if that was set on the incoming frame).
- HEADERS / PUSH_PROMISE frames and any following CONTINUATION frames are now handled properly: header fragments from all frames are gathered up into a single buffer. Once a frame with END_HEADERS is encountered, the resultant accumulated buffer is sent for decoding along with the flags and stream ID of the original HEADERS / PUSH_PROMISE frame.
- HPACK encoders/decoders now live in the frame encoder/decoder types, rather than being attached to a stream.
* Updated Linux tests.
* Headers placed in relevant files.
* Add pure-Swift HTTP/2 protocol implementation.
Motivation:
Currently we use nghttp2 for our basic protocol implementation. This is
plenty workable, but provides us a number of limitations. It adds
complexity to our build system, provides us with external dependencies we
have little control over, limits our users' ability to express low-level
protocol semantics if they so wish, causes us to heap-allocate like crazy over
the lifetime of a connection, and brings a world of pain around reentrancy.
This set of limitations has motivated a desire to move away from using
nghttp2. This is a part of that dream.
Modifications:
- Add pure Swift protocol implementations.
- Test them.
Results:
Another step towards protocol independence.
* Revamped frame decoder state machine and API. Now uses separate input and output methods to manage multiple frames in a single buffer.
* Updated Linux tests.
* Bumped swift-nio version to 1.12.0 and updated `HTTPHeaders` -> `HPACKHeaders` conversion initializer to directly copy the buffer and map the header name/value indices.
* Set up HTTP2Parser in handlerAdded if the channel had been previously activated. (#36)
By setting up the handler in the handlerAdded hook, HTTP2Parser supports
being added dynamically in a channelRead event, after the channel has sent
the channelActive event.
(cherry picked from commit 88bd6d369a)
* Minor tweaks to get the tests passing
* Initial integration here
* Minor clearing up of the handler
* Get the tests compiling
* Fix header issues.
We now pass the frame parser to the various setting operations to let it
update the state. We also fixed some issues with specific settings.
* Remove ngthttp2-based impl, clean up stream ID
* Tests with new HTTP2StreamID
* Simple test cleanup
* Get remaining non-core tests passing
* Allow GOAWAY even when fully quiesced
* We don't buffer outbound frames any more
* Emit StreamClosed events on GoAway frame
* Notify stream creation and closure
* This test started failing
* Goodbye, C code! 🎉
* Send PING ACK automatically
* Updated based on PR feedback.
* Correctly handle 1XX responses
* Add SETTINGS_MAX_FRAME_SIZE support to flow controller
* Correctly account for flow control
* Define ConcurrentStreamsHandler
* Prepare for signalling connection state changes
* Better state change message propagation
* Refactor to use result with effect everywhere
* Don't lose flow control info on stream error
* Add basic flow control management to stream multiplexer
* Better pipeline in sample server
* Clean up some reentrancy concerns
* Move to Swift 5 and NIO 2
* Move to new Swift 5 for CI
* Allow wider range of choices of years
* Catch up to NIO 2 again