17 KiB
Cleanup
- Schemas and attached databases: can we enhance our pragma handling from https://sqlite.org/forum/forumpost/27f85a4634
- SQLCipher: sqlite3_rekey is discouraged (https://github.com/ccgus/fmdb/issues/547#issuecomment-259219320)
- Write regression tests for #156 and #157
- deprecate ScopeAdapter(base, scopes), because base.addingScopes has a better implementation
- https://github.com/groue/GRDB.swift/issues/514
- Test NOT TESTED methods
- Get inspiration from the templates at https://github.com/woocommerce/woocommerce-ios/tree/develop/.github (https://github.com/woocommerce/woocommerce-ios/pull/3523)
Documentation
- Enhance the introduction to SQLRequest, based on the feedback in https://github.com/groue/GRDB.swift/issues/617
- Association: document how to use aggregates with inner join (testAnnotatedWithHasManyDefaultMaxJoiningRequired)
- Should we document that
PRAGMA locking_mode = EXCLUSIVEimproves performances? https://sqlite.org/forum/forumpost/866bf3407a
Features
-
Detect when
ColumnExpression.noOverwriteexcludes a non-existing column. Code is in DOA.upsertStatement()`. We should not silently ignore non-existing columns, as demonstrated by https://github.com/groue/GRDB.swift/issues/1539. -
Can Swift 5.5 help us with
select(.all)(request of RowDecoder),select(.id)(request of RowDecoder.ID),select(.rowid)(request of Int64)? -
Direct access to statement for bindings
-
Property wrapper that decodes dictionaries (but how to tell the key column?)
-
See if SQLITE_FCNTL_DATA_VERSION could help working around the lack of snapshots in order to avoid double initial fetch of ValueObservation. Result: no, it does not look it returns values that are comparable between two distinct SQLite connections (from the initial reader, and from the writer thhat starts the observation)
-
Grab all FTS tokens in a string
-
[NO] Can we generate EXISTS with association?
Team.annotated(with: Team.players.exists)No. We already haveTeam.annotated(with: Team.players.isEmpty == false). It does not use an EXISTS expression, but a JOIN, and this is better for the internal consistency of the query interface, and the rules that deal with association keys. -
GRDB 6: have DatabaseRegionObservation produce DatabaseCancellable just as ValueObservation.
-
RangeReplaceableCollection should have append(contentsOf: cursor)
-
GRDB 6: choose persistence table
-
GRDB 6: decoding errors
- throwing FetchableRecord initializer FIRST
- throwing Decodable FetchableRecord initializer SECOND
- deal with as much value decoding error as possible
- [?] expose throwing row accessors
-
GRDB 6: Batch insert & Batch insert RETURNING - https://stackoverflow.com/questions/1609637/is-it-possible-to-insert-multiple-rows-at-a-time-in-an-sqlite-database
-
GRDB 6: INSERT or UPDATE columns to their default value
-
GRDB 6: afterNextTransactionCommit -> afterNextTransaction(onCommit:onRollback:)
-
GRDB 6: encoding errors for record (
EncodableRecord.encode(to:))- throwing EncodableRecord.encode FIRST
-
[?] GRDB 6: protocol-based record container? This could avoid computing & encoding values we do not need.
-
GRDB 6: encoding & statement binding errors for database values (conversion to DatabaseValue, statement binding, etc)
- Prevent Date > 9999 from being encoded
-
GRDB 6: Swift 5.7
-
GRDB 6: any / some
-
GRDB 6: primary associated types (cursor, requests, ...)
-
GRDB 6: remove existential/generic duplicated methods
-
GRDB 6: remove useless AnyXXX Type erasers
-
GRDB 6: conflict resolution in persistence methods
-
GRDB 6: UPSERT
-
GRDB 6: support for RETURNING
- Support for default values:
Player.insert(db, as: FullPlayer.self)
- Support for default values:
-
[?] GRDB 6: allow mutating
update(for timestamps) -
[?] GRDB 6: let record choose persistence table (insert(into:) ?)
-
[?] GRDB 6: Support opaque return types (macOS Catalina, iOS 13, tvOS 13, watchOS 6 and later: https://stackoverflow.com/questions/56518406)
-
Long run edition. Use case: user edits the database (CRUD) but the application wants to commit and the end of the editing session.
- Create an edition SQLite connection with an open transaction (a new kind of DatabaseWriter with a save() method)
- All other writes will fail with SQLITE_BUSY. Unless they are schedules in a target dispatch queue which is paused during the edition.
-
Can we use generated columns to makes it convenient to index on inserted JSON objects? https://github.com/apple/swift-package-manager/pull/3090#issuecomment-740091760
-
Look at @FetchRequest: managed object context is stored in the environment, and error processing happens somewhere else (where?).
-
Handle SQLITE_LIMIT_VARIABLE_NUMBER in deleteAll(_:keys:) and similar APIs. https://www.sqlite.org/limits.html
-
Subqueries:
request.isEmpty/ [X] request.exists() -
Subqueries: request.count
-
Extract one row from a hasMany association (the one with the maximum date, the one with a flag set, etc.) https://stackoverflow.com/questions/43188771/sqlite-join-query-most-recent-posts-by-each-user (failed PR: https://github.com/groue/GRDB.swift/pull/767)
-
Turn a hasMany to hasOne without first/last : hasMany(Book.self).filter(Column("isBest") /* assume a single book is flagged best */).asOne()
-
Support for more kinds of joins: https://github.com/groue/GRDB.swift/issues/740
-
HasAndBelongsToMany: https://github.com/groue/GRDB.swift/issues/711
-
Support UNION https://github.com/groue/GRDB.swift/issues/671 (https://www.sqlite.org/lang_select.html#compound)
-
Measure the duration of transactions
-
Improve SQL generation for
Player.....fetchCount(db), especially with distinct. Try to avoidSELECT COUNT(*) FROM (SELECT DISTINCT player.* ...) -
Alternative technique for custom SQLite builds: see the Podfile at https://github.com/CocoaPods/CocoaPods/issues/9104, and https://github.com/clemensg/sqlite3pod
-
Attach databases. Interesting question: what happens when one attaches a non-WAL db to a databasePool?
-
SQL Generation
- date functions
- NOW/CURRENT_TIMESTAMP
- ROUND() http://marc.info/?l=sqlite-users&m=130419182719263
- RANDOM() https://www.sqlite.org/lang_corefunc.html
- GLOB https://www.sqlite.org/lang_expr.html
- REGEXP https://www.sqlite.org/lang_expr.html
- CASE x WHEN w1 THEN r1 WHEN w2 THEN r2 ELSE r3 END https://www.sqlite.org/lang_expr.html
-
Allow concurrent reads from a snapshot
-
More schema alterations
-
Database.clearSchemaCache() is fine, but what about dbPool readers? Can we invalidate the cache for a whole pool?
-
What can we do with
cross-module-optimization? See https://github.com/apple/swift-homomorphic-encryption -
GRDB7/BREAKING: insertAndFetch, saveAndFetch, and updateAndFetch no longer return optionals (
32f41472) -
GRDB7/BREAKING: AsyncValueObservation does not need any scheduler (
83c0e643) -
GRDB7/BREAKING: Stop exporting SQLite (
679d6463) -
GRDB7/BREAKING: Remove Configuration.defaultTransactionKind (
2661ff46) -
GRDB7: Replace LockedBox with Mutex (
00ccab06) -
GRDB7: Sendable: BusyCallback (
e0d8e20b) -
GRDB7: Sendable: BusyMode (
e0d8e20b) -
GRDB7: Sendable: TransactionClock (
f7dc72a5) -
GRDB7: Sendable: Configuration (
54ffb21f) -
GRDB7: Sendable: DatabaseDataEncodingStrategy (
264d7fb5) -
GRDB7: Sendable: DatabaseDateEncodingStrategy (
264d7fb5) -
GRDB7: Sendable: DatabaseColumnEncodingStrategy (
264d7fb5) -
GRDB7: Sendable: DatabaseDataDecodingStrategy (
264d7fb5) -
GRDB7: Sendable: DatabaseDateDecodingStrategy (
264d7fb5) -
GRDB7: Sendable: DatabaseColumnDecodingStrategy (
264d7fb5) -
GRDB7/BREAKING: Remove DatabaseFuture and concurrentRead (
05f7d3c8) -
GRDB7: Sendable: DatabaseFunction (
6e691fe7) -
GRDB7: Sendable: DatabaseMigrator (
22114ad4) -
GRDB7: Not Sendable: FilterCursor (
b26e9709) -
GRDB7: Sendable: RowAdapter (
d138af26) -
GRDB7: Sendable: ValueObservationScheduler (
8429eb68) -
GRDB7: Sendable: DatabaseCollation (
4d9d67dd) -
GRDB7: Sendable: LogErrorFunction (
f362518d) -
GRDB7: Sendable: ReadWriteBox (
57a86a0e) -
GRDB7: Sendable: Pool (
f13b2d2e) -
GRDB7: Sendable: OnDemandFuture fulfill (
2aabc4c1) -
GRDB7: Sendable: WALSnapshotTransaction (
7fd34012) -
[-] GRDB7: sending closures for SerializedDatabase
-
[-] GRDB7: sending closures for ValueObservationScheduler
-
GRDB7: Sendable closures for ValueObservation.handleEvents
-
GRDB7: Not Sendable: Record (make it explicit if subclasses can be made sendable)
-
GRDB7: Not Sendable: databasepublishers/databaseregion, migrate, read, value, write
-
GRDB7: Sendable closures for writePublisher
-
GRDB7: Sendable closures for readPublisher
-
[-] GRDB7: Not Sendable: fts5customtokenizer, fts5tokenizer, fts5wrappertokenizer
-
GRDB7: Sendable: TableAlias (
f2b0b186) -
GRDB7: Sendable: SQLRelation (
9545bf70) -
GRDB7: Sendable: SQL (
ac33856f) -
GRDB7: Split Row.swift (
2ce8a619) -
GRDB7: Cleanup ValueReducer (
6c73b1c5) -
GRDB7: DatabaseCursor has a primary associated type (
b11c5dd2) -
GRDB7: Enable Strict Concurrency Checks (
6aa43ded) -
GRDB7: Sendable: OrderedDictionary (
e022c35b) -
GRDB7: Rename ReadWriteBox to ReadWriteLock (
7f5205ef) -
GRDB7: Sendable: DatabaseRegionConvertible (
b4677ded) -
GRDB7: Sendable: ValueConcurrentObserver (
87b9db65,5465d056) -
GRDB7: Sendable: ValueWriteOnlyObserver (
ff2a7548) -
GRDB7: ValueObservation closures
-
[?] GRDB7: DatabasePublishers.ValueSubscription
-
GRDB7: Sendable: ValueObservation (
93f6f982) -
[?] GRDB7: Not Sendable: SharedValueObservation
-
GRDB7: doc (
c0838cf9) -
GRDB7/BREAKING: PersistenceContainer is Sendable (
50eefa8c) -
GRDB7: TableRecord.databaseSelection must be declared as a computed property (
24d232aa)- Doc
- Migration Guide
-
GRDB7: Sendable: Association (
b06aaee4) -
GRDB7/Tests: Sendable: ValueObservationRecorder (
2947b3d7) -
GRDB7: ValueObservation.print cautiously uses its stream argument (
5f8b39b7) -
GRDB7/Tests: use a single and Sendable test TextOutputStream (
bbb1a736) -
GRDB7: ValueObservation needs a ValueReducer, not a
_ValueReducer(08733108) -
GRDB7: Database support for cancellation (
4ddf4bca) -
GRDB7: SerializedDatabase support for async db access with support for Task cancellation (
737cb149) -
GRDB7: DatabaseWriter async methods support Task cancellation (
a5226501) -
GRDB7: DatabaseReader async methods support Task cancellation (
10c9d311) -
GRDB7: Document that async methods can throw CancellationError (
8df18fb8) -
[-] GRDB7: Sendable: AssociationAggregate (
48ad10ae) -
GRDB7: Sendable: AsyncValueObservation (necessary for async algorithm) (
ce63cdfa) -
GRDB7: Sendable: DatabaseRegionObservation (
b4ff52fb) -
[-] GRDB7: DispatchQueue.asyncSending (
7b075e6b) -
GRDB7: Replace sequences with collection (e.g. https://github.com/tidal-music/tidal-sdk-ios/pull/39)
-
GRDB7: Replace
someDatabaseReader/Writer withanywhere possible, in order to avoid issues with accessing DatabaseContext from GRDBQuery (if the problem exists in Xcode 16) -
GRDB7: bump to iOS 13, macOS 10.15, tvOS 13 (for ValueObservation support for MainActor)
-
GRDB7: DatabasePublishers.Value should carry the type of the Scheduler, so that we can rely on main-actor-isolated callbacks.
-
GRDB7: Remove warning about "products" in Package.swift
-
GRDB7: Fixits
- defaultTransactionKind
- concurrentRead
-
GRDB7: Swift Concurrency recommendations
- Record classe(s)
- InferSendableFromCaptures
-
GRDB7: stop fostering the Record class
- Remove all mentions from the README
- Warn about it in the documentation of the class.
-
GRDB7: Breaking changes documentation
- [BREAKING] Xcode 16+, Swift 6+
- [BREAKING] iOS 13+
- [BREAKING] macOS 10.15+
- [BREAKING] tvOS 13+
- [BREAKING] watchOS 7+
- [-] insertAndFetch, updateAndFetch, saveAndFetch
- CSQLite renamed to GRDBCSQLite
- CSQLite is not exported
- defaultTransactionKind
- concurrentRead
- record column strategies:
- databaseDataEncodingStrategy
- databaseDateEncodingStrategy
- databaseUUIDEncodingStrategy
- databaseDataDecodingStrategy
- databaseDateDecodingStrategy
- PersistenceContainer subscript no longer returns its input value
- cancellation of async database access
- Async sequences built from ValueObservation schedule values and errors on the cooperative thread pool by default.
TableRecord.databaseSelectionshould be declared as a computed static property- [-] databaseDecodingUserInfo and databaseEncodingUserInfo must be declared as a computed property
-
GRDB7: Review experimental apis
-
[?] GRDB7: Change ValueObservation callback argument so that it could expose snapshots? https://github.com/groue/GRDB.swift/discussions/1523#discussioncomment-9092500
Unsure if necessary
- Have TableAlias embed the record type
- Have SQLPrimaryKeyExpression embed the record type
- Remove support for suspended databases - https://inessential.com/2020/02/13/how_we_fixed_the_dreaded_0xdead10cc_cras
- https://sqlite.org/pragma.html#pragma_index_xinfo
- filter(rowid:), filter(rowids:)
- https://github.com/apple/swift-evolution/blob/master/proposals/0075-import-test.md
- https://forums.swift.org/t/how-to-encode-objects-of-unknown-type/12253/6
- Configuration.crashOnError = true
- Glossary (Database Access Methods, etc.)
- Not sure: type safety for SQL expressions
- Introduce some record protocol with an associated primary key type. Restrict filter(key:) methods to this type. Allow distinguishing FooId from BarId types.
- Replace Column with TypedColumn. How to avoid code duplication (repeated types)? Keypaths?
- Remove prefix from association keys when association name is namespaced: https://github.com/groue/GRDB.swift/issues/584#issuecomment-517658122
- Alternative support for custom SQLite builds, with CocoaPods: https://github.com/CocoaPods/CocoaPods/issues/9103
Unsure how
-
Association limits:
Author.including(optional: Author.books.order(date.desc).first) -
Joins and full-text tables
-
Support for "INSERT INTO ... SELECT ...".
-
Look at the jazzy configuration of https://github.com/bignerdranch/Deferred
-
Predicates, so that a filter can be evaluated both on the database, and on a record instance.
After investigation, we can't do it reliably without knowing the collation used by a column. And SQLite does not provide this information elsewhere than in the full CREATE TABLE statement stored in sqlite_master.
-
ValueObservation erasure
// Do better than this observation.mapReducer { _, reducer in AnyValueReducer(reducer) } -
new.updateChanges(from: old) vs. old.updateChanges(with: { old.a = new.a }). This is confusing.
Reading list
- Documentation generation: https://twitter.com/jckarter/status/987525569196650496: cmark is the implementation the Swift compiler uses for doc comments etc.
- VACUUM (https://blogs.gnome.org/jnelson/)
- http://www.sqlite.org/intern-v-extern-blob.html
- https://sqlite.org/sharedcache.html
- Undo: https://www.sqlite.org/undoredo.html
- Undo: https://sqlite.org/sessionintro.html
- Swift, Xcode:List of Swift documentation keywords: https://swift.org/documentation/api-design-guidelines.html#special-instructions
- Swift, Xcode:Amazing tip from Xcode labs: add a EXCLUDED_SOURCE_FILE_NAMES build setting to conditionally exclude sources for different configuration: https://twitter.com/zats/status/74386298602026496
- SQLITE_ENABLE_SQLLOG: http://mjtsai.com/blog/2016/07/19/sqlite_enable_sqllog/
- Swift, Xcode: https://github.com/apple/swift/blob/master/docs/OptimizationTips.rst
- Associations: http://docs.diesel.rs/diesel/associations/index.html
- FTS: http://cocoamine.net/blog/2015/09/07/contentless-fts4-for-large-immutable-documents/
- pinyin: http://hustlzp.com/post/2016/02/ios-full-text-search-using-sqlite-fts4
- Realm oddity: http://stackoverflow.com/questions/41721769/realm-update-object-without-updating-lists
- File protection: https://github.com/ccgus/fmdb/issues/262
- File protection: https://lists.apple.com/archives/cocoa-dev/2012/Aug/msg00527.html
- iOS apps are terminated every time they enter the background if they share an encrypted database with an app extension
- Cross-Process notifications with CFNotificationCenterGetDarwinNotifyCenter