53 Commits
Author SHA1 Message Date
Peter Adams 4bce87f3cf Enable ShellCheck CI (#478)
Motivation:

Complete enablement of all standard GitHub Actions workflows.

Modifications:

Describe the modifications you've done.

Result:

After your change, what will change.
2024-10-30 15:22:05 +00:00
Peter Adams bc93af21b0 GitHub Actions to parity with existing CI (#469)
Motivation:

Want to move away from legacy CI.

Modifications:

Add configuration and update code to use GitHub actions.

Result:

GitHub Actions can be used for basic CI.
2024-10-23 14:02:30 +00:00
Peter Adams 482f425a94 Remove script for generating contributors list (#468)
Motivation:

We're no longer maintaining this list as it can be trivially obtained
from git history.

Modifications:

Remove script for generating contributors list.

Result:

One less script to maintain
2024-10-23 13:55:02 +01:00
George Barnett 0a3fcea35d Update soundness, it is 2024 now (#442) 2024-06-21 09:36:39 -07:00
Fabian Fett 145eec4e4f Add HPACKHeader.removeAll(keepingCapacity:) (#393)
In scenarios, in which we want to send lots of small HTTP/2 requests, we want to reuse a single HPACKHeaders object as often as we can. To support this we should have a removeAll(keepingCapacity:) method.
2023-05-17 02:09:06 -07:00
Si Beaumont 428b0f3551 Drop support for Swift 5.5 (#392)
Motivation:

Now Swift 5.8 is released, we are dropping support for 5.5, in line with the usual support 

Modifications:

- Remove `Package@swift-5.5.swift`.
- Remove `LinuxMain.swift` and `generate_linux_tests.rb` script.
- Remove use of `generate_linux_tests.rb` from CI check.
- Update README.

Result:

No functional change, but Swift 5.5 is no longer supported.
2023-04-17 13:27:31 +01:00
Cory Benfield 81b001dd07 Add regression testing for docs and fix the issues. (#389) 2023-04-11 11:04:31 +02:00
Rick Newton-RogersandCory Benfield 312fb6813f make soundness check aware of 2023 (#378)
Motivation:

Soundness check currently fails for copyright statements including 2023
as the year

Modifications:

add '3' to the checking regex

Result:

Soundness checks should not fail for documents created this year

Co-authored-by: Cory Benfield <lukasa@apple.com>
2023-03-03 05:53:01 -08:00
George Barnett 1e64bb7cf2 Remove Jazzy doc generation (#368)
Motivation:

Docs are generated by and hosted on the Swift Package Index. We no
longer need the script to generate docs via Jazzy.

Modifications:

- Remove the generate_docs script
- Remove Jazzy from the Dockerfile but keep Ruby; it's used for
  generating test manifests.
- Remove SwiftFormat from the Dockerfile; we don't use it.

Result:

Fewer unused things.
2023-01-23 07:47:05 -08:00
George Barnett 00576e6f1e Raise minimum supported Swift version from 5.4 to 5.5 (#355)
Motivation:

SwiftNIO periodically drops support for older Swift versions. Now that 5.7 has been released, 5.4 will be dropped.

Modifications:

- Remove 5.4 specific Package.swift and docker-compose
- Update the 5.7 docker-compose to use the released 5.7 and move from focal (2004) to jammy (2204)
- Fix tools version in Package@swift-5.5.swift
- Update docs

Results:

Minimum Swift version is 5.5
2022-09-29 09:21:42 +01:00
Cory Benfield cb6f87d9e0 Use Docc for documentation (#351)
Motivation

Documentation is nice, and we can help support users by providing useful
clear docs.

Modifications

Add Docc to 5.6 and later builds
Make sure symbol references work
Add overview docs

Result

Nice rendering docs
2022-08-09 06:44:21 +01:00
George Barnett eb1d696793 Remove build_podspec.sh (#341)
Motivation:

We no longer support Cocoapods.

Modifications:

- Remove build_podspec.sh and associated scripts

Result:

Less unused code.
2022-05-04 11:02:50 +01:00
George Barnett ef83310fe9 Add script to list transitive dependencies (#340)
See https://github.com/apple/swift-nio/pull/2082
2022-05-03 10:16:09 +01:00
Fabian Fett dbfca164c1 Drop support for Swift 5.2 and 5.3 (#337)
As outlined in a [Swift forums post in November ’21](https://forums.swift.org/t/swiftnio-swift-version-support/53232), SwiftNIO will only support the latest non-patch Swift release and the 2 immediately prior non-patch versions.

- drop support for Swift 5.2 and 5.3. 
- update CI for Swift 5.4 to run on bionic instead of focal to ensure that we still test bionic.
2022-04-20 08:57:26 +02:00
Franz Busch c897189177 Use SPM API diff checker (#334)
### Motivation:

SPM has built in functionality to check the API of modules against a target git treeish. We can use this to simplify our `check_no_api_breakages.sh` script. Closes https://github.com/apple/swift-nio/issues/1239

### Modifications:

This PR, exchanges the direct calls to Swift's API checker with the new SPM `diagnose-api-breaking-changes` tool. This allows us to get rid of the manual module parsing, build invocations and result comparisons.

### Result:

We are now using SPMs `diagnose-api-breaking-changes` to check for breaking changes.
2022-03-23 09:56:16 -07:00
Cory Benfieldandtom doron 2d5d718d90 Update alloc limits and clean up soundness (#331)
* Lower allocation limits

Our recent fixes reduced our allocation count, which is good.

* fix code soundness check

motivation: code should be sound!

changes:
* update one case of inappropriate langauge
* update header years check regex

(cherry picked from commit a89e8ae4e0)

Co-authored-by: tom doron <tomer@apple.com>
2022-03-17 09:08:42 +00:00
George Barnett bec04abc90 Update doc generation script (#322)
Motivation:

The docs haven't been generated for a while.

See also: https://github.com/apple/swift-nio/pull/2013

Modifications:

- Pass `--spm` flag to sourcekitten
- Update source location of sourcekitten
- Fix link to NIO docs

Result:

Docs build.
2021-12-17 11:39:37 +00:00
Cory Benfield dbfc8fcd1a Clean up imports and dependencies. (#300)
Motivation:

With NIO 2.32.0 we broke the core NIO module up into modules that split
apart the POSIX layer and the core abstractions. As a result, this
package no longer needs to express a hard dependency on the POSIX layer.

Modifications:

- Rewrote imports of NIO to NIOCore.
- Added NIOEmbedded imports where necessary in tests.
- Note that the main modules still depend on NIO, which is necessary
    for backwards-compatibility reasons. This dependency is unused.

Result:

No need to use NIOPosix.
2021-09-14 11:30:04 +01:00
George Barnett dbe7fb05fe Fix doc generation and jazzy version (#298)
From: https://github.com/apple/swift-nio-extras/pull/141

Motivation:

The combination of Ruby and Jazzy are throwing up issues in CI on older
Ubuntu versions.

Modifications:

- Only install Jazzy on focal
- Fix doc generation script to use more recent source-kitten syntax

Results:

- CI doesn't complain
- Docs get built
2021-09-03 14:38:04 +01:00
George Barnett f6cde03ef8 Bump Swift version in build_podspec.sh (#294)
Motivation:

SwiftNIO recently dropped support for Swift 5.0 and Swift 5.1 but our
build_podspec.sh script doesn't reflect this change. Additionally the
list_topsorted_dependencies.sh script always assumed named depenencies
when listing module depenencies. Since our tools version has now been
bumped to 5.2 that's no longer the case and the script produces a bunch
of nulls which throws the build_podspec.sh script off course.

Modifications:

- Bump the Swift version in build_podspec.sh to 5.2
- Include "product" dependencies when listing module dependencies in
  list_topsorted_dependencies.sh

Result:

Podspecs are created as expected
2021-08-17 10:11:26 +01:00
buttaface 94d3652e37 Enable tests for Android (#296)
Motivation:

Run the tests on Android.

Modifications:

- Enable the tests on Android.
- Replace '/tmp' with 'FileManager.default.temporaryDirectory'.

Result:

All the tests pass on Android AArch64 and x86_64.
2021-08-16 19:05:52 +01:00
David Evans 488444b14c Remove support for Swift 5.0 and 5.1 (#292)
* Update package

* Update CI

* Add to readme
2021-06-23 08:37:07 +01:00
Cory Benfield 1ef8bac96a Add fuzzing harness to swift-nio-http2. (#288)
Motivation:

Fuzzing is great, we should have a good sensible place to centralise a
harness.

Modifications:

- Add a fuzzing harness.

Result:

There's a common way to do fuzzing.
2021-05-27 16:33:43 +01:00
Johannes Weiss 6ed44fd465 update code of conduct to version 1.4 (#278) 2021-02-17 14:17:34 +00:00
Fabian Fett 16e49c6246 Use welcoming language (#274) 2021-01-22 08:55:07 +00:00
Cory Benfield 37919ba1b7 Reduce the cost of flushing (#265)
Motivation:

The OutboundFlowControlBuffer is responsible for managing DATA and
HEADERS frames for all streams in order to ensure that we obey HTTP/2
flow control rules, even when users don't. This object is therefore on
the data path for the vast majority of frames, and is extremely
performance sensitive.

Unfortunately, the current algorithm has some nasty performance cliffs.
Right now it has the awkward property of making flushes take an amount
of time linear in the number of streams on the connection, not the
number of streams that had had frames sent on them. The result is that
connections with highly parallel numbers of streams see extremely poor
performance when flushing, a problem that is exacerbated if other parts
of the HTTP/2 stack fail to coalesce those flushes.

We can clean this up in a few ways, but the easiest way is to just keep
track of which streams have done I/O since the last flush. While we're
there, we can make a few other optimisations to ensure we only flush
data on streams we are actually processing, to further reduce the risk
of wasting our time processing unnecessary work.

Modifications:

- Add a Set to keep track of where we've done writes since the last
   flush.
- Streams whose flow control window goes to zero stop being writable.
- Cheaper and more accurate calculation of zero-length writes, which
   avoids the possibility of awkward stalls with zero-length DATA frames
   and END_STREAM.
- Stop doing math about the number of flushed bytes, we don't use that
   information any more.

Results:

An approximately 10% performance gain on the 100 concurrent streams
server-only benchmark. The relative performance gain shifts with the
number of active streams, but it should be positive in all cases.
2020-12-03 11:00:25 +00:00
George Barnett b6efd3f0eb Add watchOS deployment to PodSpec build script (#251)
Motivation:

We support watchOS 6+ with SwiftNIO Transport Services; as such we should
include watchOS as a deployment target for our CocoaPods.

Modifications:

- Add a watchOS deployment target to `build_podspecs.sh`

Result:

Users can deploy to watchOS 6+ with CocoaPods.
2020-10-19 10:08:17 +01:00
Cory Benfield 78ddbdfca1 We now use the main branch. (#250) 2020-09-24 16:26:39 +01:00
Cory Benfield 28d94c489d Update alloc counter scaffolding. (#249)
Motivation:

I have suspicions that the allocation counter tests in NIOHTTP2 don't
fail properly, so let's bring the scaffolding up to date.

Modifications:

Update the few local files to the most recent NIO versions.

Result:

Smaller diffs to NIO, maybe better infrastructure?
2020-09-24 13:34:21 +01:00
Daniel Alm 7026b3bbea Fix the -f option in build_podspec.sh. (#229)
* Fix the -f option in build_podspec.sh.

* Another minor tweak.
2020-08-06 18:23:15 +01:00
George Barnett 177c50a566 Add new pipeline helpers and deprecate old paths (#226)
Motivation:

As part of #214 we need to introduce new pipeline helpers to configure
the channel pipeline with the new handlers. This PR adds those helpers
and deprecates the old paths.

Modifications:

- Add pipeline helpers with the new streamID-less initializer
- Switch old pipeline helpers which provided codecs to use the new path
  and codecs
- Duplicate SimpleClientServerTests and ConfiguringPipelineTests to add
  frame payload based counterparts
- Deprecate older helpers and multiplexer init, delete tests which don't
  hit deprecated paths (note: these still exist in the payload based
  counterparts)
- Update Bench1Conn10kRequests to be payload based

Result:

- More 'FramePayload' based Tests
- Paths for creating an 'HTTP2Frame' based stream via
  'HTTP2StreamMultiplexer' are now deprecated
2020-08-04 12:52:00 +01:00
George Barnett 48082eecc1 Update the test generation script to allow for deprecated tests (#223)
Motivation:

To test deprecated functionality without a bunch of warnings, tests must
be marked as deprecated. This also needs to be bubbled up to anywhere
calling that test, such as in the generated linux test manifests.

Our script to generate the manifests currently doesn't support this and
as part of #214 we'll need to deprecate a few things while keeping their
tests.

Modifications:

- Pull in the latest version of the `generate_linux_tests.rb` script
  from SwiftNIO.
- Re-run the script, since it includes some formatting changes too.

Result:

- When we deprecate functionality, we can deprecate the tests without
  the generate manifests emitting warnings.
2020-07-30 16:13:12 +01:00
Peter AdamsandJohannes Weiss 59e864c1a2 Enable thread sanitizer in CI (#212)
Motivation:

Thread Sanitizer will find bugs which might be missed otherwise.

Modifications:

When running docker image for 1604/5.1 tests and h2spec are both compiled with --sanitize=thread

Result:

Tests are run with thread sanitizer turned on.

Co-authored-by: Johannes Weiss <johannesweiss@apple.com>
2020-07-23 14:08:48 +01:00
George Barnett c5d10f4165 Add synchronous option to trunk push (#207)
See https://github.com/apple/swift-nio/pull/1551
2020-06-10 15:18:29 +01:00
George Barnett 5be29ae313 Update build_podspec.sh to not use exact dependency versions (#206)
Motivation:

The build_podspec.sh script generates a podspec which requires exact
versions of its dependencies. This very quickly turns into unresolvable
dependency graphs.

Modifications:

- NIO version passed to script must be in the format `MAJOR.MINOR`
- Podspec dependencies are now '>= MAJOR.MINOR', '< MAJOR+1'

Result:

Looser version requirements for podspecs
2020-06-10 13:50:40 +01:00
Johannes WeissandCory Benfield cdf3c48b15 link swift-nio-extras in docs (#195)
Co-authored-by: Cory Benfield <lukasa@apple.com>
2020-05-11 12:53:34 +01:00
George Barnett fbfe24ccee Exert backpressure via stream flow control windows (#191)
Motivation:

When a window update event is received in the stream channel we notify
the inbound window manager. Doing so may emit a WINDOW_UPDATE frame
to indicate the remote may send us more DATA. However, this happens even
if there are inbound frames waiting to be delivered to the channel. This
may lead to the remote sending us more DATA which we have to buffer
until it is read.

Modifications:

- Update the inbound window manager to track the number of bytes
  buffered in the stream channel
- Account for buffered bytes when receiving a new window size event
- Possibly emit WINDOW_UPDATE after unbuffering inbound frames into the
  stream channel's pipeline

Result:

The timing of WINDOW_UPDATE frames is better aligned with when frames
are actually read in a stream.
2020-03-12 07:45:00 +00:00
Jake PrickettandJake Prickett 0841f641fc Update Build Podspec & Dependencies Script with weissi's Updates (#189)
Co-authored-by: Jake Prickett <jpricke1@ford.com>
2020-03-03 10:33:04 +00:00
tomer doron 780e3a290b update docs generation script to work better with selinux (#187)
motivation: docs scripts is broken on selinux due to restriction on ssh-agent

changes:
* change the jazzy workspace to a directory accessible by the CI node so we can push from the agent instead of from the docker container
* dont remove older docs when pushing new ones
* add version information and source links
2020-02-28 12:26:01 -08:00
Jake PrickettandJake Prickett 0c7d697b5f Update build_podspec and added SwiftNIOHPACK Dependency (#185)
Co-authored-by: Jake Prickett <jpricke1@ford.com>
2020-02-18 20:16:26 +00:00
Jake PrickettandJake Prickett a8550db895 Added Build Podspec Script (#184)
Co-authored-by: Jake Prickett <jpricke1@ford.com>
2020-02-14 14:49:05 +00:00
tomer doron 97e8bbbcf8 api breakage detection (#163)
motivation: test for api breakage on PR

changes:
* add script to detect api breakage
* amend docker setup to include jq
2019-10-07 15:00:10 -07:00
tomer doron 3d0b38268e add performance analysis docker-compose setup and script (#141)
motivation: prepare to run performance tests in ci

changes:

add performance-test docker-compose task
add script that can parse the performance test suite results and compare between two result-sets
add generic shell docker-compose task
2019-07-16 11:04:26 +01:00
Johannes Weiss 16fae8a3d1 add the first allocation tests for nio-http2 (#122)
Motivation:

Allocation tests are good :).

Modifications:

Add two allocations tests and NIO's integration test framework.

Result:

Hopefully fewer allocations in the future. And better ways to check what
allocations are happening.
2019-05-17 14:36:46 +01:00
Johannes Weiss 8c991df0c0 Add 'Show on GitHub' link to API docs (#108)
Motivation:

Allowing readers of the API documentation to drill into the implementation of documented declarations can be educational and helpful for debugging, among several other reasons.

Modifications:

Add --github-file-prefix appending the current version in order to keep stable links. This means that documentation generated from any non-release revision may resolve incorrectly. This can be refined in the doc generation script later if it is deemed problematic by resolving to the commit sha if it doesn't exactly align with the version.

Result:

Users will be able to click a 'Show on GitHub' for all documented API declarations.
2019-04-29 10:42:20 +01:00
Cory Benfield aaaeca77f9 Police correct content-length headers. (#98)
Motivation:

RFC 7540 requires that compliant specifications emit errors if content-length has been incorrectly
set. We should have this behaviour by default, though we should allow it to be disabled if necessary.

Modifications:

- Added content-length checking.
- Added flag to disable content-length checking.

Result:

Better RFC 7540 compliance.
2019-04-15 16:57:16 +01:00
Johannes Weiss 1a5c69c7f9 API docs: repo switcher (#83) 2019-03-28 16:03:38 +00:00
tomer doron 13d9b4852e add doc generation script (#77)
motivation: publish API docs

changes:
* amend docker file to include jazzy
* add doc generation script based on what we have in swift-nio and swift-log
2019-03-27 09:33:19 +00:00
Cory Benfield 42087764b1 Add Dockerfile to run h2spec. (#41)
Motivation:

When we get around to it we should aim for our example server to pass
h2spec with flying colours. We aren't there yet, but it's helpful to
be able to run h2spec easily.

Modifications:

Add a docker-compose recipe for running h2spec.

Result:

It'll be easier to run h2spec.
2019-02-21 16:32:29 +00:00
Cory Benfield 3c90345c36 Pure-Swift HTTP/2 implementation (#40)
* Updated to use SwiftNIO 1.10+ to gain access to HTTPHeaders internals.

* Add pure-Swift HTTP/2 protocol implementation.

Motivation:

Currently we use nghttp2 for our basic protocol implementation. This is
plenty workable, but provides us a number of limitations. It adds
complexity to our build system, provides us with external dependencies we
have little control over, limits our users' ability to express low-level
protocol semantics if they so wish, causes us to heap-allocate like crazy over
the lifetime of a connection, and brings a world of pain around reentrancy.

This set of limitations has motivated a desire to move away from using
nghttp2. This is a part of that dream.

Modifications:

- Add pure Swift protocol implementations.
- Test them.

Results:

Another step towards protocol independence.

* Initial basic parser code.

* Adding unit tests.

* Add NIOHTTP2FlowControlHandler

Motivation:

One of the things that nghttp2 used to do for us was to handle buffering
DATA frames until flow control allowed them to go out. For users that
want that behaviour, we need to replace the nghttp2 code with a new
ChannelHandler that can do this work.

Modifications:

- Added NIOHTTP2FlowControlHandler.

Result:

Users will have the same feature set as they did with nghttp2.

* Generate Linux test information.

* Documentation comment.

* Updated new tests from master to use new payload types.

* Updated following review.

Notable changes:

- Implemented unit tests for the ALTSVC (0xa) and ORIGIN (0xc) frame types.
- Additional DATA frames are synthesized to wrap individual payload chunks as they arrive. If the payload of a DATA frame with 4MB data arrives in 4 separate 1MB chunks, the decoder will emit 4 DATA frames, with only the last containing the END_STREAM flag (if that was set on the incoming frame).
- HEADERS / PUSH_PROMISE frames and any following CONTINUATION frames are now handled properly: header fragments from all frames are gathered up into a single buffer. Once a frame with END_HEADERS is encountered, the resultant accumulated buffer is sent for decoding along with the flags and stream ID of the original HEADERS / PUSH_PROMISE frame.
- HPACK encoders/decoders now live in the frame encoder/decoder types, rather than being attached to a stream.

* Updated Linux tests.

* Headers placed in relevant files.

* Add pure-Swift HTTP/2 protocol implementation.

Motivation:

Currently we use nghttp2 for our basic protocol implementation. This is
plenty workable, but provides us a number of limitations. It adds
complexity to our build system, provides us with external dependencies we
have little control over, limits our users' ability to express low-level
protocol semantics if they so wish, causes us to heap-allocate like crazy over
the lifetime of a connection, and brings a world of pain around reentrancy.

This set of limitations has motivated a desire to move away from using
nghttp2. This is a part of that dream.

Modifications:

- Add pure Swift protocol implementations.
- Test them.

Results:

Another step towards protocol independence.

* Revamped frame decoder state machine and API. Now uses separate input and output methods to manage multiple frames in a single buffer.

* Updated Linux tests.

* Bumped swift-nio version to 1.12.0 and updated `HTTPHeaders` -> `HPACKHeaders` conversion initializer to directly copy the buffer and map the header name/value indices.

* Set up HTTP2Parser in handlerAdded if the channel had been previously activated. (#36)

By setting up the handler in the handlerAdded hook, HTTP2Parser supports
being added dynamically in a channelRead event, after the channel has sent
the channelActive event.

(cherry picked from commit 88bd6d369a)

* Minor tweaks to get the tests passing

* Initial integration here

* Minor clearing up of the handler

* Get the tests compiling

* Fix header issues.

We now pass the frame parser to the various setting operations to let it
update the state. We also fixed some issues with specific settings.

* Remove ngthttp2-based impl, clean up stream ID

* Tests with new HTTP2StreamID

* Simple test cleanup

* Get remaining non-core tests passing

* Allow GOAWAY even when fully quiesced

* We don't buffer outbound frames any more

* Emit StreamClosed events on GoAway frame

* Notify stream creation and closure

* This test started failing

* Goodbye, C code! 🎉

* Send PING ACK automatically

* Updated based on PR feedback.

* Correctly handle 1XX responses

* Add SETTINGS_MAX_FRAME_SIZE support to flow controller

* Correctly account for flow control

* Define ConcurrentStreamsHandler

* Prepare for signalling connection state changes

* Better state change message propagation

* Refactor to use result with effect everywhere

* Don't lose flow control info on stream error

* Add basic flow control management to stream multiplexer

* Better pipeline in sample server

* Clean up some reentrancy concerns

* Move to Swift 5 and NIO 2

* Move to new Swift 5 for CI

* Allow wider range of choices of years

* Catch up to NIO 2 again
2019-02-21 11:52:49 +00:00