Fix multiple issues in the BoringSSL vendoring script and update
BoringSSL to 0226f30467f540a3f62ef48d453f93927da199b6
### Checklist
- [x] I've run tests to see all new and existing tests pass
- [x] I've followed the code style of the rest of the project
- [x] I've read the [Contribution Guidelines](CONTRIBUTING.md)
- [x] I've updated the documentation if necessary
### Motivation:
The `vendor-boringssl.sh` script is currently broken in a number of
ways, both in general and relative to the latest BoringSSL version:
- `--enable-test-discovery` is still in use
- The Linux Swift SDKs being generated are out of date (5.10-jammy)
- The script does not prefix exported C++ symbols
- Invoking the script without a `BORINGSSL_REVISION` results in an
unconditional error
- The `PATTERNS` list is out of date for the latest BoringSSL
- The latest BoringSSL contains a bug which fails to apply `extern "C"`
to two exported symbols, allowing for collisions.
In addition, the latest BoringSSL has made `BN_MONT_CTX` opaque,
necessitating that it be referenced with `OpaquePointer` rather than
`UnsafePointer<>`.
### Modifications:
The following changes are included:
- `--enable-test-discovery` has been removed
- `vendor-boringssl.sh` and `generate-linux-sdks.sh` have been updated
to use Swift 6.1.2 SDKs built for Ubuntu Noble
- The `mangle_cpp_structures` function from `swift-nio-ssl`'s version
of `vendor-boringssl.sh` has been copied over (thanks @Lukasa!)
- `BORINGSSL_REVISION` is now allowed to be an empty string
- The `PATTERNS` array has been updated
- A patch has been added which applies `extern "C"` to the functions
which should have it
- `s/UnsafePointer<BN_MONT_CTX>/OpaquePointer/g`
- Updated `CMakeLists.txt` per `update-cmake-lists.sh`
- Update BoringSSL to
`0226f30467f540a3f62ef48d453f93927da199b6`
### Result:
Vendoring the latest BoringSSL will work again.
The latest BoringSSL is vendored.
Swift SDK solution for adding armv7 symbols for swift-crypto using the
[swift-sdk-generator](https://github.com/swiftlang/swift-sdk-generator).
### Motivation:
As outlined in #223, the necessary symbols for armv7 are missing from
swift-crypto, and we'd like to add them in some way that could be
accepted by the team. It was discussed that using Swift SDKs generated
from the swift-sdk-generator could be a solution that could be easily
vetted, since the swift-sdk-generator is a component maintained by Apple
and the swiftlang organization.
### Modifications:
- Added a new `./scripts/generate-linux-sdks.sh` script to clone & build
the swift-sdk-generator, and create Swift SDKs for x86_64, aarch64, and
armv7.
- Modified the `./scripts/vendor-boringssl.sh` script to use these
installed Swift SDKs to cross-compile for Linux architectures in the
script.
- Ran the scripts against the exact revision of BoringSSL to avoid
source updates, which adds the missing symbols for armv7.
### Result:
Swift SDKs are used to generate symbols for Linux architectures instead
of Docker, and missing symbols for armv7 are added as well.
I have a few pros and cons for this solution:
- Pros
- Swift SDKs are used to cross-compile for Linux instead of Docker,
which performs better than Rosetta 2 emulation. Docker does not have to
be used at all for this solution.
- Switching to using Swift SDKs is the more "correct" way to
cross-compile for Linux targets, and much of the ecosystem for
cross-compilation of Swift packages is moving to using Swift SDKs.
- Cons
- Lots of downloads need to happen since the swift-sdk-generator needs
to download the host macOS toolchain for Swift 5.10, plus Ubuntu
packages and target toolchains for each architecture (x86_64, aarch64,
armv7). This results in ~15GB of artifacts in the /tmp directory for
just the generator artifacts. However, this only needs to be run once
before running the vendoring script.
Let me know what you think @Lukasa.
Updates for the `scripts/vendor-boringssl.h` script that make it a bit
easier to use and ensure you don't accidentally get an update of
BoringSSL you don't want.
### Motivation:
I was working on investigating building a Swift SDK for #223 and ran
into a couple of things that I saw could be improved. Not all need to be
done, but at the very least I feel the boringssl revision lock should be
considered. @Lukasa
### Modifications:
- Add missing invocation of `$sed` in the script- `gsed` was being
invoked instead, although everywhere else `$sed` was used (typo).
- Support locking `BORINGSSL_REVISION` to a specific revision number
through an argument to the script. This way the script can be re-run
without unintentionally upgrading boringssl.
- Remove the non-existent `boringssl_prefix_symbols_nasm.inc` exclude
from the `CCryptoBoringSSL` target. I'm guessing it was there for an
older version of boringssl, but no longer there with the current
revision?
### Result:
The `scripts/vendor_boringssl.sh` script behaves a bit better and can be
run without upgrading the boringssl version unless explicitly desired.
This patch updates BoringSSL to
035e720641f385e82c72b7b0a9e1d89e58cb5ed5. As with all of our patch
updates to BoringSSL, this comes in multiple commits. The first contains
the changes to the vendoring script, and the last contains the tweaks to
product code. The middle commit can be ignored for review purposes.
Add support for ML-KEM post-quantum key encapsulation mechanism inside
`_CryptoExtras`.
<!-- Thanks for contributing to Swift Crypto! Before you submit your
issue, please make sure you followed our checklist and check the
appropriate boxes by putting an x in the [ ]: [x] -->
### Checklist
- [X] I've run tests to see all new and existing tests pass
- [X] I've followed the code style of the rest of the project
- [X] I've read the [Contribution Guidelines](CONTRIBUTING.md)
- [X] I've updated the documentation if necessary
#### If you've made changes to `gyb` files
- [x] I've run `.script/generate_boilerplate_files_with_gyb` and
included updated generated files in a commit of this pull request
### Motivation:
With the advent of quantum computing, the mathematical foundations on
which the cryptographic protocols in use today are based have been
questioned, as they can easily be circumvented and violated by quantum
computers.
While waiting for the creation of quantum computers that work at full
capacity, and to protect network communications from "[Harvest Now,
Decrypt
Later](https://en.wikipedia.org/wiki/Harvest_now,_decrypt_later)"
attacks, the cryptographic community is working on post-quantum
cryptography algorithms, which work on the traditional computers we use
today, but are resistant to future attacks by quantum computers.
One of these algorithms is ML-KEM (AKA Kyber), a module lattice-based
key encapsulation mechanism standardized by NIST in [FIPS
203](https://csrc.nist.gov/pubs/fips/203/final), that is available
inside BoringSSL.
By including ML-KEM inside Swift Crypto, we can get closer to
normalizing quantum secure algorithms and start implementing them into
our apps and libraries to make them quantum-proof.
### Modifications:
Added `MLKEM768` and `MLKEM1024` enums (generated with `gyb`) inside the
`_CryptoExtras` module with corresponding `PrivateKey` and `PublicKey`
structs that use BoringSSL methods to decapsulate and encapsulate
ML-KEM-768 and ML-KEM-1024 ciphertexts, with the code style of other
KEMs in the library (they also conform to `KEMPrivateKey` and
`KEMPublicKey` protocols).
### Result:
ML-KEM key agreements can be handled with Swift Crypto.
Add support for ML-DSA post-quantum digital signatures inside
`_CryptoExtras`.
### Motivation:
With the advent of quantum computing, the mathematical foundations on
which the cryptographic protocols in use today are based have been
questioned, as they can easily be circumvented and violated by quantum
computers.
While waiting for the creation of quantum computers that work at full
capacity, and to protect network communications from "[Harvest Now,
Decrypt
Later](https://en.wikipedia.org/wiki/Harvest_now,_decrypt_later)"
attacks, the cryptographic community is working on post-quantum
cryptography algorithms, which work on the traditional computers we use
today, but are resistant to future attacks by quantum computers.
One of these algorithms is ML-DSA (AKA Dilithium), a module
lattice-based signature scheme standardized by NIST in [FIPS
204](https://csrc.nist.gov/pubs/fips/204/final), that is available
inside BoringSSL.
By including ML-DSA inside Swift Crypto, we can get closer to
normalizing quantum secure algorithms and start implementing them into
our apps and libraries to make them quantum-proof.
### Modifications:
Added a `MLDSA65` enum inside the `_CryptoExtras` module with
corresponding `PrivateKey`, `PublicKey` and `Signature` structs that use
BoringSSL methods to produce and verify ML-DSA-65 digital signatures,
with the code style of other signature schemes in the library.
Added tests that cover use cases of the ML-DSA scheme, including [test
vectors taken from the BoringSSL
repo](https://boringssl.googlesource.com/boringssl/+/refs/heads/master/crypto/mldsa/mldsa_nist_keygen_tests.txt)
(extracted from a `.txt` file and encoded in JSON).
### Result:
ML-DSA-65 digital signatures can be created and verified with Swift
Crypto.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Use centralized CMake checks
### Motivation:
We now have centrally-hosted CMake checks in the NIO repository which we
should use to reduce maintenance. The new checks also increase coverage
by performing a build with CMake as part of the checks.
### Modifications:
* Use the new workflow
* Document how to update the CMakeLists files
* Remove the obsolete scripts
### Result:
Better test coverage.
This change prepares swift-crypto for use on Windows with CMake and also
the Package.swift for use in the Swift toolchain CI system.
### Checklist
- [X] I've run tests to see all new and existing tests pass
- [X] I've followed the code style of the rest of the project
- [X] I've read the [Contribution Guidelines](CONTRIBUTING.md)
- [X] I've updated the documentation if necessary
#### If you've made changes to `gyb` files
- [X] I've run `.script/generate_boilerplate_files_with_gyb` and
included updated generated files in a commit of this pull request
### Motivation:
Upgrading swift-crypto from an older version requires that Windows works
properly with CMake, and also SwiftPM.
### Modifications:
Modify the CMakeLists.txt files and the generator script so that it
conditionally adds assembly files for non-Windows platforms. Prepare the
Package.swift file for use in Swift CI pipelines, by consulting an
environment to either use the absolute URL, or a prepared version of the
swift-asn1 dependency.
### Result:
After this change it should be possible to build with the newest
swift-crypto in the Swift CI system.
Update both the CMakeLists.txt for CCryptoBoringSSL and the
CMakeLists.txt generator so that it can add platform-independent
assembly files.
### Checklist
- [X] I've run tests to see all new and existing tests pass
- [X] I've followed the code style of the rest of the project
- [X] I've read the [Contribution Guidelines](CONTRIBUTING.md)
- [X] I've updated the documentation if necessary
#### If you've made changes to `gyb` files
- [X] I've run `./scripts/generate_boilerplate_files_with_gyb` and
included updated generated files in a commit of this pull request
### Motivation:
There are downstream linker failures due to missing symbols from certain
platform neutral assembly files.
### Modifications:
The `scripts/update-cmake-lists.sh` has an updated find expression to
discover platform-independent assembly files and includes them in the
`CMakeLists.txt` files.
### Result:
There will be more symbols included from third-parties such as
BoringSSL.
## Motivation
As part of the Github Actions CI migration we held off enabling the
format check and moving to the same formatting rules as Swift NIO, but
we'd like to do that now.
## Modifications
- Add a script to generate (and update) the `.swiftformatignore` file
since this repo only wants to format a specific set of files matching a
pattern.
- Generate and commit the `.swiftformatignore` file.
- Run the fomatter and commit the results.
- Additional commits to satisfy the remaining linter rules that weren't
auto-fixed by swift-format.
- Enable the format check in CI.
## Result
- Code that's not shared is formatted with a style consistent with the
other repos this team maintain.
- CI is now checking code format again.
---------
Co-authored-by: Cory Benfield <lukasa@apple.com>
Co-authored-by: Rick Newton-Rogers <rnro@apple.com>
* Add cxx interop build pipeline
* Update docker-compose.2204.59.yaml
* Change cxx interop CI step to build separate project instead of using cxx compiler flag
* Update docker-compose for 5.10
* Fix package name in docker
* Change code root dirname and remove test target
---------
Co-authored-by: Yim Lee <yim_lee@apple.com>
* Exclude AES from CMake
SwiftPM doesn't need it and this would allow removal of CCryptoBoringSSL dependency on non-Darwin platforms, thus reducing SwiftPM size.
* Update update_cmakelists.sh to exclude AES source files
In some cases developers have need of AES modes other than what we
support today, or need the ability to implement algorithms that require
access to the core AES permutation function. As a starting point, we can
provide access to the AES block function in CryptoExtras.
# Motivation
BoringSSL exposes AES-GCM-SIV algorithms which are in general useful to have and provide a nonce-misuse resistant mode of AES-GCM. Since, `CryptoKit` is not exposing AES-GCM-SIV we need to add this to `_CryptoExtras`
# Modification
Exposes `AES-GCM-SIV` through `_CryptoExtras`.
# Result
We can now use `AES-GCM-SIV` through `_CryptoExtras`.
Motivation:
When using only the CryptoKit API on Apple platforms we have always
thunked through to the CryptoKit interface and implementation. However,
we didn't do a thorough job of preventing the BoringSSL target from
getting compiled and linked.
We can do a better job now, which will save compile times and binary
sizes in many cases.
Modifications:
- Change Package.swift to express a target specific dependency in most
cases.
- Preserve a development mode which overrides that target specific
dependency.
- Add the missing compile guards.
Results:
Smaller binaries and faster compiles on Apple platforms.
motivation: source-kitten has changed how its invoked
changes:
* update docker file to install jazzy correctly
* invoke source-kitten with the correct flags
* make location of source-kitten source checkout more robust
Motivation:
The various Apple SoS projects are currently dropping Swift 5.1 support.
Swift Crypto should do the same. While we're here, let's add CI for 5.4,
5.5 nightlies, and main nightlies.
Modifications:
- Remove 5.1 dockerfile.
- Stop claiming we support 5.1.
- Add 5.4, 5.5, and main dockerfiles.
- Updated Swift Format to 5.2.
- Associated formatting changes.
Result:
Better test matrix and support matrix.
* Update vendor-boringssl to save modulemap
Motivation:
The vendor-boringssl script rewrites `include`, which leads to deletion of `module.modulemap`.
Modification:
Save a copy of modulemap to tmp dir and copy it back at the end.
Result:
`module.modulemap` remains after running the script.
* Recreate modulemap each time instead of copying
Add support for Windows for vending BoringSSL. This is required in
order to build SwiftCrypto for Windows.
Co-authored-by: Cory Benfield <lukasa@apple.com>