1 Report on the Conclusions of the QUIC DDD Process 2 ================================================= 3 4 The [QUIC Demo-Driven Design process](README.md) was undertaken to meet the OMC 5 requirement to develop a QUIC API that required only minimal changes to existing 6 applications to be able to adapt their code to use QUIC. The demo-driven design 7 process developed a set of representative demos modelling a variety of common 8 OpenSSL usage patterns based on analysis of a broad spectrum of open source 9 software projects using OpenSSL. 10 11 As part of this process, a set of proposed diffs were produced. These proposed 12 diffs were the expected changes which would be needed to the baseline demos to 13 support QUIC based on theoretical analysis of the minimum requirements to be 14 able to support QUIC. This analysis concluded that the changes needed to 15 applications could be kept very small in many circumstances, with only minimal 16 diff sizes to the baseline demos. 17 18 Following the development of QUIC MVP, these demos have been revisited and the 19 correspondence of our actual final API and usage patterns with the planned diffs 20 have been reviewed. 21 22 This document discusses the planned changes and the actual changes for each demo 23 and draws conclusions on the level of disparity. 24 25 Since tracking a set of diffs separately is unwieldy, both the planned and 26 unplanned changes have been folded into the original baseline demo files guarded 27 with `#ifdef USE_QUIC`. Viewing these files therefore is informative to 28 application writers as it provides a clear view of what is different when using 29 QUIC. (The originally planned changes, and the final changes, are added in 30 separate, clearly-labelled commits; to view the originally planned changes only, 31 view the commit history for a given demo file.) 32 33 ddd-01-conn-blocking 34 -------------------- 35 36 This demo exists to demonstrate the simplest possible usage of OpenSSL, whether 37 with TLS or QUIC. 38 39 ### Originally planned changes 40 41 The originally planned change to enable applications for QUIC amounted to just a 42 single line: 43 44 ```diff 45 + ctx = SSL_CTX_new(QUIC_client_method()); 46 - ctx = SSL_CTX_new(TLS_client_method()); 47 ``` 48 49 ### Actual changes 50 51 The following additional changes needed to be made: 52 53 - `QUIC_client_method` was renamed to `OSSL_QUIC_client_method` for namespacing 54 reasons. 55 56 - A call to `SSL_set_alpn_protos` to configure ALPN was added. This is necessary 57 because QUIC mandates the use of ALPN, and this was not noted during the 58 DDD process. 59 60 ddd-02-conn-nonblocking 61 ----------------------- 62 63 This demo exists to demonstrate simple non-blocking usage. As with 64 ddd-01-conn-blocking, the name resolution process is managed by `BIO_s_connect`. 65 66 It also arbitrarily adds a `BIO_f_buffer` pushed onto the BIO stack 67 as this is a common application usage pattern. 68 69 ### Originally planned changes 70 71 The originally planned changes to enable applications for QUIC amounted to: 72 73 - Change of method (as for ddd-01-conn-blocking); 74 75 - Use of a `BIO_f_dgram_buffer` BIO method instead of a `BIO_f_buffer`; 76 77 - Use of a `BIO_get_poll_fd` function to get the FD to poll rather than 78 `BIO_get_fd`; 79 80 - A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2) 81 need to be determined. 82 83 - Additional functions in application code to determine event handling 84 timeouts related to QUIC (`get_conn_pump_timeout`) and to pump 85 the QUIC event loop (`pump`). 86 87 - Timeout computation code which involves merging and comparing different 88 timeouts and calling `pump` as needed, based on deadlines reported 89 by libssl. 90 91 Note that some of these changes are unnecessary when using the thread assisted 92 mode (see the variant ddd-02-conn-nonblocking-threads below). 93 94 ### Actual changes 95 96 The following additional changes needed to be made: 97 98 - Change of method name (as for ddd-01-conn-blocking); 99 100 - Use of ALPN (as for ddd-01-conn-blocking); 101 102 - The strategy for how to expose pollable OS resource handles 103 to applications to determine I/O readiness has changed substantially since the 104 original DDD process. As such, applications now use `BIO_get_rpoll_descriptor` 105 and `BIO_get_wpoll_descriptor` to determine I/O readiness, rather than the 106 originally hypothesised `SSL_get_poll_fd`. 107 108 - The strategy for how to determine when to poll for `POLLIN`, when to 109 poll for `POLLOUT`, etc. has changed since the original DDD process. 110 This information is now exposed via `SSL_net_read_desired` and 111 `SSL_net_write_desired`. 112 113 - The API to expose the event handling deadline for the QUIC engine 114 has evolved since the original DDD process. The new API 115 `SSL_get_event_timeout` is used, rather than the originally hypothesised 116 `BIO_get_timeout`/`SSL_get_timeout`. 117 118 - The API to perform QUIC event processing has been renamed to be 119 more descriptive. It is now called `SSL_handle_events` rather than 120 the originally hypothesised `BIO_pump`/`SSL_pump`. 121 122 The following changes were foreseen to be necessary, but turned out to actually 123 not be necessary: 124 125 - The need to change code which pushes a `BIO_f_buffer()` after an SSL BIO 126 was foreseen as use of buffering on the network side is unworkable with 127 QUIC. This turned out not to be necessary since we can just reject the 128 BIO_push() call. The buffer should still be freed eventually when the 129 SSL BIO is freed. The buffer is not used and is unnecessary, so it is 130 still desirable for applications to remove this code. 131 132 ddd-02-conn-nonblocking-threads 133 ------------------------------- 134 135 This is a variant of the ddd-02-conn-nonblocking demo. The base is the same, but 136 the changes made are different. The use of thread-assisted mode, in which an 137 internal assist thread is used to perform QUIC event handling, enables an 138 application to make fewer changes than are needed in the ddd-02-conn-nonblocking 139 demo. 140 141 ### Originally planned changes 142 143 The originally planned changes to enable applications for QUIC amounted to: 144 145 - Change of method, this time using method `QUIC_client_thread_method` rather 146 than `QUIC_client_method`; 147 148 - Use of a `BIO_get_poll_fd` function to get the FD to poll rather than 149 `BIO_get_fd`; 150 151 - A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2) 152 need to be determined. 153 154 Note that this is a substantially smaller list of changes than for 155 ddd-02-conn-nonblocking. 156 157 ### Actual changes 158 159 The following additional changes needed to be made: 160 161 - Change of method name (`QUIC_client_thread_method` was renamed to 162 `OSSL_QUIC_client_thread_method` for namespacing reasons); 163 164 - Use of ALPN (as for ddd-01-conn-blocking); 165 166 - Use of `BIO_get_rpoll_descriptor` rather than `BIO_get_poll_fd` (as for 167 ddd-02-conn-nonblocking). 168 169 - Use of `SSL_net_read_desired` and `SSL_net_write_desired` (as for 170 ddd-02-conn-nonblocking). 171 172 ddd-03-fd-blocking 173 ------------------ 174 175 This demo is similar to ddd-01-conn-blocking but uses a file descriptor passed 176 directly by the application rather than BIO_s_connect. 177 178 ### Originally planned changes 179 180 - Change of method (as for ddd-01-conn-blocking); 181 182 - The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM, 183 IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`. 184 185 ### Actual changes 186 187 The following additional changes needed to be made: 188 189 - Change of method name (as for ddd-01-conn-blocking); 190 191 - Use of ALPN (as for ddd-01-conn-blocking). 192 193 ddd-04-fd-nonblocking 194 --------------------- 195 196 This demo is similar to ddd-01-conn-nonblocking but uses a file descriptor 197 passed directly by the application rather than BIO_s_connect. 198 199 ### Originally planned changes 200 201 - Change of method (as for ddd-01-conn-blocking); 202 203 - The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM, 204 IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`; 205 206 - A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2) 207 need to be determined. 208 209 - Additional functions in application code to determine event handling 210 timeouts related to QUIC (`get_conn_pump_timeout`) and to pump 211 the QUIC event loop (`pump`). 212 213 - Timeout computation code which involves merging and comparing different 214 timeouts and calling `pump` as needed, based on deadlines reported 215 by libssl. 216 217 ### Actual changes 218 219 The following additional changes needed to be made: 220 221 - Change of method name (as for ddd-01-conn-blocking); 222 223 - Use of ALPN (as for ddd-01-conn-blocking); 224 225 - `SSL_get_timeout` replaced with `SSL_get_event_timeout` (as for 226 ddd-02-conn-nonblocking); 227 228 - `SSL_pump` renamed to `SSL_handle_events` (as for ddd-02-conn-nonblocking); 229 230 - The strategy for how to determine when to poll for `POLLIN`, when to 231 poll for `POLLOUT`, etc. has changed since the original DDD process. 232 This information is now exposed via `SSL_net_read_desired` and 233 `SSL_net_write_desired` (as for ddd-02-conn-nonblocking). 234 235 ddd-05-mem-nonblocking 236 ---------------------- 237 238 This demo is more elaborate. It uses memory buffers created and managed by an 239 application as an intermediary between libssl and the network, which is a common 240 usage pattern for applications. Managing this pattern for QUIC is more elaborate 241 since datagram semantics on the network channel need to be maintained. 242 243 ### Originally planned changes 244 245 - Change of method (as for ddd-01-conn-blocking); 246 247 - Call to `BIO_new_bio_pair` is changed to `BIO_new_dgram_pair`, which 248 provides a bidirectional memory buffer BIO with datagram semantics. 249 250 - A change to how the `POLLIN`/`POLLOUT`/`POLLERR` flags to pass to poll(2) 251 need to be determined. 252 253 - Potential changes to buffer sizes used by applications to buffer 254 datagrams, if those buffers are smaller than 1472 bytes. 255 256 - The arguments to the `socket(2)` call are changed from `(AF_INET, SOCK_STREAM, 257 IPPROTO_TCP)` to `(AF_INET, SOCK_DGRAM, IPPROTO_UDP)`; 258 259 ### Actual changes 260 261 The following additional changes needed to be made: 262 263 - Change of method name (as for ddd-01-conn-blocking); 264 265 - Use of ALPN (as for ddd-01-conn-blocking); 266 267 - The API to construct a `BIO_s_dgram_pair` ended up being named 268 `BIO_new_bio_dgram_pair` rather than `BIO_new_dgram_pair`; 269 270 - Use of `SSL_net_read_desired` and `SSL_net_write_desired` (as for 271 ddd-02-conn-nonblocking). 272 273 ddd-06-mem-uv 274 ------------- 275 276 This demo is the most elaborate of the set. It uses a real-world asynchronous 277 I/O reactor, namely libuv (the engine used by Node.js). In doing so it seeks to 278 demonstrate and prove the viability of our API design with a real-world 279 asynchronous I/O system. It operates wholly in non-blocking mode and uses memory 280 buffers on either side of the QUIC stack to feed data to and from the 281 application and the network. 282 283 ### Originally planned changes 284 285 - Change of method (as for ddd-01-conn-blocking); 286 287 - Various changes to use of libuv needed to switch to using UDP; 288 289 - Additional use of libuv to configure a timer event; 290 291 - Call to `BIO_new_bio_pair` is changed to `BIO_new_dgram_pair` 292 (as for ddd-05-mem-nonblocking); 293 294 - Some reordering of code required by the design of libuv. 295 296 ### Actual changes 297 298 The following additional changes needed to be made: 299 300 - Change of method name (as for ddd-01-conn-blocking); 301 302 - Use of ALPN (as for ddd-01-conn-blocking); 303 304 - `BIO_new_dgram_pair` renamed to `BIO_new_bio_dgram_pair` (as for 305 ddd-05-mem-nonblocking); 306 307 - `SSL_get_timeout` replaced with `SSL_get_event_timeout` (as for 308 ddd-02-conn-nonblocking); 309 310 - `SSL_pump` renamed to `SSL_handle_events` (as for ddd-02-conn-nonblocking); 311 312 - Fixes to use of libuv based on a corrected understanding 313 of its operation, and changes that necessarily ensue. 314 315 Conclusions 316 ----------- 317 318 The DDD process has successfully delivered on the objective of delivering a QUIC 319 API which can be used with only minimal API changes. The additional changes on 320 top of those originally planned which were required to successfully execute the 321 demos using QUIC were highly limited in scope and mostly constituted only minor 322 changes. The sum total of the changes required for each demo (both planned and 323 additional), as denoted in each DDD demo file under `#ifdef USE_QUIC` guards, 324 are both minimal and limited in scope. 325 326 Minimal and limited are distinct criteria. If inexorable technical 327 requirements dictate, an enormous set of changes to an application could be 328 considered minimal. The changes required to representative applications, as 329 demonstrated by the DDD demos, are not merely minimal but also limited. 330 331 For example, while the extent of these necessary changes varies by the 332 sophistication of each demo and the kind of application usage pattern it 333 represents, some demos in particular demonstrate exceptionally small changesets; 334 for example, ddd-01-conn-blocking and ddd-02-conn-nonblocking-threads, with 335 ddd-01-conn-blocking literally being enabled by a single line change assuming 336 ALPN is already configured. 337 338 This report concludes the DDD process for the single-stream QUIC client API 339 design process, which sought to validate our API design and API ease of use for 340 existing applications seeking to adopt QUIC. 341
Indexes created Sun Mar 01 05:31:48 UTC 2026