Motivation:
Swift 6.0 is no longer supported, we should bump the tools version and
remove it from our CI.
Modifications:
* Bump the Swift tools version to Swift 6.1
* Remove Swift 6.0 jobs where appropriate in main.yml, pull_request.yml
Result:
Code reflects our support window.
---------
Co-authored-by: George Barnett <gbarnett@apple.com>
Motivation:
Complete enablement of all standard GitHub Actions workflows.
Modifications:
Describe the modifications you've done.
Result:
After your change, what will change.
Motivation:
Standardised formatting makes the project cleaner to look at and work
on.
Modifications:
Setup swift-format with the same settings as Swift-NIO
Result:
Common formatting throughout
Motivation
The HTTP2FrameDecoder is a complex object that was written early in the
development of swift-nio-http2. Its logical flow is complex, and it
hasn't been meaningfully rewritten in quite some time, so it's difficult
to work with and understand.
Annoyingly, some bugs have crept in over the years. Because of the
structure of the code it can be quite difficult to understand how the
parser actually works, and fixing a given issue can be difficult.
This patch aims to produce a substantial change to the HTTP2FrameDecoder
to make it easier to understand and maintain in the long run.
Modifications
This patch provides a complete rewrite of HTTP2FrameDecoder. It doesn't
do this by having a ground-up rewrite: instead, it's more like a
renovation, with the general scaffolding kept. The rewrite was performed
incrementally, keeping the existing test suite passing and writing new
tests when necessary.
The following major changes were made:
1. New states and edges were added to the state machine to handle
padding.
Prior to this change, padding was handled as part of frame payload
decoding. This is not totally unreasonable, but it dispersed padding
management widely and made it easy to have bugs slip in. This patch
replaces this with a smaller set of locations.
Padding is now handled in two distinct ways. For HEADERS and
PUSH_PROMISE frames, trailing padding is still stripped as part of
frame payload decode, but it's done so generically, and the padding
bytes are never exposed to the individual frame parser. For DATA,
there is a new state to handle trailing padding removal, which
simplifies the highly complex logic around synthesised data frames.
For all frames, the leading padding byte is handled by a new
dedicated state which is used unconditionally, instead of attempting
to opportunistically strip it. This simplifies the code flow.
As a side benefit, this change means we can now accurately report the
padding used on HEADERS and PUSH_PROMISE frames, even when they are
part of a CONTINUATION sequence.
2. The synthesised DATA frame logic has been substantially reworked.
With the removal of the padding logic from the state, we now know
that so long as we have either got a byte of data to emit _or_ the
DATA frame is zero length, we will always emit a frame. This has made
it simpler to understand the control flow when synthesising DATA
frames.
3. The monolithic state switch has been refactored into per-state
methods.
This helps manage the amount of state that each method can see, as
well as to logically split them up. In addition, it allows us to
recast state transformations as (fairly) pure functions.
Additionally, this allowed the larger methods to be refactored with
smaller helpers that are more obviously correct.
4. The frame payload parsers have been rewritten.
The main goal here was to remove preflight length checks and unsafe
code. The preflight length checks cause trouble when they disagree
with the parsing code, so we now rely on the parsing code being
correct with regard to length.
Relatedly, we previously had two separate places where we
communicated length: a frame header length and a ByteBuffer length.
This was unnecessary duplication of information, so we instead use a
ByteBuffer slice to manage the length. This ensures that we cannot
over-parse a message.
Finally, in places that used unsafe code or separate integer reads,
we have refactored to stop using that unsafe code and to use combined
integer reads.
5. Extraneous serialization code has been extracted.
The HTTP2FrameEncoder was unnecessarily in this file, which took a
large file and made it larger. I moved this out.
Result
The resulting parser is clearer and safer. Complex logic has been broken
out into smaller methods with less access to global data. The code
should be generally clearer.
Motivation:
We don't currently support ALTSVC and ORIGIN frames. At the moment when
receiving or attempting to write these frames we trap. Trapping when
receiving an unsupported frame can lead to DoS attacks. Moreover these
frames are non critical so can safely be ignored.
Modifications:
- Forward inbound and outbound ALTSVC and ORIGIN frames to the
connection state machine
- The connection state machine now ignores inbound frames of this type
- The connection state machine traps on outbound frames of this type (as
per the current behaviour)
- Document this behaviour on `HTTP2Frame.FramePayload`
- Tests and fuzz testing failure case
Result:
We don't trap when receiving unsupported frames.
* Refactor HPACK integer decoding
Motivation:
The HPACK integer decoding used a number of unchecked operations which
can trap on some inputs. Since it is network reachable code, it should
throw errors if the operations overflow.
Modifications:
- Add a failure case to the fuzz testing
- Refactor the integer decoding to check for overflow on the arithmetic
- This throws a new 'HPACKErrors.UnrepresentableInteger' error on
overflow
- Add a missing bounds check
- Remove an unnecessary and incorrect path
- Remove the default argument from the function driving the decoding,
the default was not valid and would cause an assertion to fail if used
- Return the decoded value as an `Int` rather than a `UInt`
- More tests
Result:
Integer decoding is safer.
* Use unchecked shifting
* Use truncatingIfNeeded
* make error internal
* Validate HEADERS frame length accounts for priority data
Motivation:
When parsing a HEADERS frame payload which includes priority data the
expected length of the frame is not validated to ensure that it is large
enough to account for the stream data. This can lead to the expected
payload length being negative.
Modifications:
- Validate that when stream priority data is present that the frame
length is at least that size
- Add fuzz testing failure case
Result:
The frame decoder will throw a protocol error when parsing a HEADERS
frame if the length of the frame is less than the number of bytes
required for stream priority data and the priority flag is set.
* Add additional tests
* Fix weirdo formatting
Motivation:
With NIO 2.32.0 we broke the core NIO module up into modules that split
apart the POSIX layer and the core abstractions. As a result, this
package no longer needs to express a hard dependency on the POSIX layer.
Modifications:
- Rewrote imports of NIO to NIOCore.
- Added NIOEmbedded imports where necessary in tests.
- Note that the main modules still depend on NIO, which is necessary
for backwards-compatibility reasons. This dependency is unused.
Result:
No need to use NIOPosix.
Motivation:
Fuzzing is great, we should have a good sensible place to centralise a
harness.
Modifications:
- Add a fuzzing harness.
Result:
There's a common way to do fuzzing.