1 Design document for the unified scheme data 2 =========================================== 3 4 How are things connected? 5 ------------------------- 6 7 The unified scheme takes all its data from the `build.info` files seen 8 throughout the source tree. These files hold the minimum information 9 needed to build end product files from diverse sources. See the 10 section on `build.info` files below. 11 12 From the information in `build.info` files, `Configure` builds up an 13 information database as a hash table called `%unified_info`, which is 14 stored in configdata.pm, found at the top of the build tree (which may 15 or may not be the same as the source tree). 16 17 [`Configurations/common.tmpl`](common.tmpl) uses the data from `%unified_info` to 18 generate the rules for building end product files as well as 19 intermediary files with the help of a few functions found in the 20 build-file templates. See the section on build-file templates further 21 down for more information. 22 23 build.info files 24 ---------------- 25 26 As mentioned earlier, `build.info` files are meant to hold the minimum 27 information needed to build output files, and therefore only (with a 28 few possible exceptions [1]) have information about end products (such 29 as scripts, library files and programs) and source files (such as C 30 files, C header files, assembler files, etc). Intermediate files such 31 as object files are rarely directly referred to in `build.info` files (and 32 when they are, it's always with the file name extension `.o`), they are 33 inferred by `Configure`. By the same rule of minimalism, end product 34 file name extensions (such as `.so`, `.a`, `.exe`, etc) are never mentioned 35 in `build.info`. Their file name extensions will be inferred by the 36 build-file templates, adapted for the platform they are meant for (see 37 sections on `%unified_info` and build-file templates further down). 38 39 The variables `PROGRAMS`, `LIBS`, `MODULES` and `SCRIPTS` are used to declare 40 end products. There are variants for them with `_NO_INST` as suffix 41 (`PROGRAM_NO_INST` etc) to specify end products that shouldn't get installed. 42 43 The variables `SOURCE`, `DEPEND`, `INCLUDE` and `DEFINE` are indexed by a 44 produced file, and their values are the source used to produce that 45 particular produced file, extra dependencies, include directories 46 needed, or C macros to be defined. 47 48 All their values in all the `build.info` throughout the source tree are 49 collected together and form a set of programs, libraries, modules and 50 scripts to be produced, source files, dependencies, etc etc etc. 51 52 Let's have a pretend example, a very limited contraption of OpenSSL, 53 composed of the program `apps/openssl`, the libraries `libssl` and 54 `libcrypto`, an module `engines/ossltest` and their sources and 55 dependencies. 56 57 # build.info 58 LIBS=libcrypto libssl 59 INCLUDE[libcrypto]=include 60 INCLUDE[libssl]=include 61 DEPEND[libssl]=libcrypto 62 63 This is the top directory `build.info` file, and it tells us that two 64 libraries are to be built, the include directory `include/` shall be 65 used throughout when building anything that will end up in each 66 library, and that the library `libssl` depend on the library 67 `libcrypto` to function properly. 68 69 # apps/build.info 70 PROGRAMS=openssl 71 SOURCE[openssl]=openssl.c 72 INCLUDE[openssl]=.. ../include 73 DEPEND[openssl]=../libssl 74 75 This is the `build.info` file in `apps/`, one may notice that all file 76 paths mentioned are relative to the directory the `build.info` file is 77 located in. This one tells us that there's a program to be built 78 called `apps/openss` (the file name extension will depend on the 79 platform and is therefore not mentioned in the `build.info` file). It's 80 built from one source file, `apps/openssl.c`, and building it requires 81 the use of `.` and `include/` include directories (both are declared 82 from the point of view of the `apps/` directory), and that the program 83 depends on the library `libssl` to function properly. 84 85 # crypto/build.info 86 LIBS=../libcrypto 87 SOURCE[../libcrypto]=aes.c evp.c cversion.c 88 DEPEND[cversion.o]=buildinf.h 89 90 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" 91 DEPEND[buildinf.h]=../Makefile 92 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm 93 94 This is the `build.info` file in `crypto/`, and it tells us a little more 95 about what's needed to produce `libcrypto`. LIBS is used again to 96 declare that `libcrypto` is to be produced. This declaration is 97 really unnecessary as it's already mentioned in the top `build.info` 98 file, but can make the info file easier to understand. This is to 99 show that duplicate information isn't an issue. 100 101 This `build.info` file informs us that `libcrypto` is built from a few 102 source files, `crypto/aes.c`, `crypto/evp.c` and `crypto/cversion.c`. 103 It also shows us that building the object file inferred from 104 `crypto/cversion.c` depends on `crypto/buildinf.h`. Finally, it 105 also shows the possibility to declare how some files are generated 106 using some script, in this case a perl script, and how such scripts 107 can be declared to depend on other files, in this case a perl module. 108 109 Two things are worth an extra note: 110 111 `DEPEND[cversion.o]` mentions an object file. DEPEND indexes is the 112 only location where it's valid to mention them 113 114 # ssl/build.info 115 LIBS=../libssl 116 SOURCE[../libssl]=tls.c 117 118 This is the build.info file in `ssl/`, and it tells us that the 119 library `libssl` is built from the source file `ssl/tls.c`. 120 121 # engines/build.info 122 MODULES=dasync 123 SOURCE[dasync]=e_dasync.c 124 DEPEND[dasync]=../libcrypto 125 INCLUDE[dasync]=../include 126 127 MODULES_NO_INST=ossltest 128 SOURCE[ossltest]=e_ossltest.c 129 DEPEND[ossltest]=../libcrypto.a 130 INCLUDE[ossltest]=../include 131 132 This is the `build.info` file in `engines/`, telling us that two modules 133 called `engines/dasync` and `engines/ossltest` shall be built, that 134 `dasync`'s source is `engines/e_dasync.c` and `ossltest`'s source is 135 `engines/e_ossltest.c` and that the include directory `include/` may 136 be used when building anything that will be part of these modules. 137 Also, both modules depend on the library `libcrypto` to function 138 properly. `ossltest` is explicitly linked with the static variant of 139 the library `libcrypto`. Finally, only `dasync` is being installed, as 140 `ossltest` is only for internal testing. 141 142 When `Configure` digests these `build.info` files, the accumulated 143 information comes down to this: 144 145 LIBS=libcrypto libssl 146 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c 147 DEPEND[crypto/cversion.o]=crypto/buildinf.h 148 INCLUDE[libcrypto]=include 149 SOURCE[libssl]=ssl/tls.c 150 INCLUDE[libssl]=include 151 DEPEND[libssl]=libcrypto 152 153 PROGRAMS=apps/openssl 154 SOURCE[apps/openssl]=apps/openssl.c 155 INCLUDE[apps/openssl]=. include 156 DEPEND[apps/openssl]=libssl 157 158 MODULES=engines/dasync 159 SOURCE[engines/dasync]=engines/e_dasync.c 160 DEPEND[engines/dasync]=libcrypto 161 INCLUDE[engines/dasync]=include 162 163 MODULES_NO_INST=engines/ossltest 164 SOURCE[engines/ossltest]=engines/e_ossltest.c 165 DEPEND[engines/ossltest]=libcrypto.a 166 INCLUDE[engines/ossltest]=include 167 168 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" 169 DEPEND[crypto/buildinf.h]=Makefile 170 DEPEND[util/mkbuildinf.pl]=util/Foo.pm 171 172 A few notes worth mentioning: 173 174 `LIBS` may be used to declare routine libraries only. 175 176 `PROGRAMS` may be used to declare programs only. 177 178 `MODULES` may be used to declare modules only. 179 180 The indexes for `SOURCE` must only be end product files, such as 181 libraries, programs or modules. The values of `SOURCE` variables must 182 only be source files (possibly generated). 183 184 `INCLUDE` and `DEPEND` shows a relationship between different files 185 (usually produced files) or between files and directories, such as a 186 program depending on a library, or between an object file and some 187 extra source file. 188 189 When `Configure` processes the `build.info` files, it will take it as 190 truth without question, and will therefore perform very few checks. 191 If the build tree is separate from the source tree, it will assume 192 that all built files and up in the build directory and that all source 193 files are to be found in the source tree, if they can be found there. 194 `Configure` will assume that source files that can't be found in the 195 source tree (such as `crypto/bildinf.h` in the example above) are 196 generated and will be found in the build tree. 197 198 The `%unified_info` database 199 ---------------------------- 200 201 The information in all the `build.info` get digested by `Configure` and 202 collected into the `%unified_info` database, divided into the following 203 indexes: 204 205 depends => a hash table containing 'file' => [ 'dependency' ... ] 206 pairs. These are directly inferred from the DEPEND 207 variables in build.info files. 208 209 modules => a list of modules. These are directly inferred from 210 the MODULES variable in build.info files. 211 212 generate => a hash table containing 'file' => [ 'generator' ... ] 213 pairs. These are directly inferred from the GENERATE 214 variables in build.info files. 215 216 includes => a hash table containing 'file' => [ 'include' ... ] 217 pairs. These are directly inferred from the INCLUDE 218 variables in build.info files. 219 220 install => a hash table containing 'type' => [ 'file' ... ] pairs. 221 The types are 'programs', 'libraries', 'modules' and 222 'scripts', and the array of files list the files of 223 that type that should be installed. 224 225 libraries => a list of libraries. These are directly inferred from 226 the LIBS variable in build.info files. 227 228 programs => a list of programs. These are directly inferred from 229 the PROGRAMS variable in build.info files. 230 231 scripts => a list of scripts. There are directly inferred from 232 the SCRIPTS variable in build.info files. 233 234 sources => a hash table containing 'file' => [ 'sourcefile' ... ] 235 pairs. These are indirectly inferred from the SOURCE 236 variables in build.info files. Object files are 237 mentioned in this hash table, with source files from 238 SOURCE variables, and AS source files for programs and 239 libraries. 240 241 shared_sources => 242 a hash table just like 'sources', but only as source 243 files (object files) for building shared libraries. 244 245 As an example, here is how the `build.info` files example from the 246 section above would be digested into a `%unified_info` table: 247 248 our %unified_info = ( 249 "depends" => 250 { 251 "apps/openssl" => 252 [ 253 "libssl", 254 ], 255 "crypto/buildinf.h" => 256 [ 257 "Makefile", 258 ], 259 "crypto/cversion.o" => 260 [ 261 "crypto/buildinf.h", 262 ], 263 "engines/dasync" => 264 [ 265 "libcrypto", 266 ], 267 "engines/ossltest" => 268 [ 269 "libcrypto.a", 270 ], 271 "libssl" => 272 [ 273 "libcrypto", 274 ], 275 "util/mkbuildinf.pl" => 276 [ 277 "util/Foo.pm", 278 ], 279 }, 280 "modules" => 281 [ 282 "engines/dasync", 283 "engines/ossltest", 284 ], 285 "generate" => 286 { 287 "crypto/buildinf.h" => 288 [ 289 "util/mkbuildinf.pl", 290 "\"\$(CC)", 291 "\$(CFLAGS)\"", 292 "\"$(PLATFORM)\"", 293 ], 294 }, 295 "includes" => 296 { 297 "apps/openssl" => 298 [ 299 ".", 300 "include", 301 ], 302 "engines/ossltest" => 303 [ 304 "include" 305 ], 306 "libcrypto" => 307 [ 308 "include", 309 ], 310 "libssl" => 311 [ 312 "include", 313 ], 314 "util/mkbuildinf.pl" => 315 [ 316 "util", 317 ], 318 } 319 "install" => 320 { 321 "modules" => 322 [ 323 "engines/dasync", 324 ], 325 "libraries" => 326 [ 327 "libcrypto", 328 "libssl", 329 ], 330 "programs" => 331 [ 332 "apps/openssl", 333 ], 334 }, 335 "libraries" => 336 [ 337 "libcrypto", 338 "libssl", 339 ], 340 "programs" => 341 [ 342 "apps/openssl", 343 ], 344 "sources" => 345 { 346 "apps/openssl" => 347 [ 348 "apps/openssl.o", 349 ], 350 "apps/openssl.o" => 351 [ 352 "apps/openssl.c", 353 ], 354 "crypto/aes.o" => 355 [ 356 "crypto/aes.c", 357 ], 358 "crypto/cversion.o" => 359 [ 360 "crypto/cversion.c", 361 ], 362 "crypto/evp.o" => 363 [ 364 "crypto/evp.c", 365 ], 366 "engines/e_dasync.o" => 367 [ 368 "engines/e_dasync.c", 369 ], 370 "engines/dasync" => 371 [ 372 "engines/e_dasync.o", 373 ], 374 "engines/e_ossltest.o" => 375 [ 376 "engines/e_ossltest.c", 377 ], 378 "engines/ossltest" => 379 [ 380 "engines/e_ossltest.o", 381 ], 382 "libcrypto" => 383 [ 384 "crypto/aes.c", 385 "crypto/cversion.c", 386 "crypto/evp.c", 387 ], 388 "libssl" => 389 [ 390 "ssl/tls.c", 391 ], 392 "ssl/tls.o" => 393 [ 394 "ssl/tls.c", 395 ], 396 }, 397 ); 398 399 As can be seen, everything in `%unified_info` is fairly simple suggest 400 of information. Still, it tells us that to build all programs, we 401 must build `apps/openssl`, and to build the latter, we will need to 402 build all its sources (`apps/openssl.o` in this case) and all the 403 other things it depends on (such as `libssl`). All those dependencies 404 need to be built as well, using the same logic, so to build `libssl`, 405 we need to build `ssl/tls.o` as well as `libcrypto`, and to build the 406 latter... 407 408 Build-file templates 409 -------------------- 410 411 Build-file templates are essentially build-files (such as `Makefile` on 412 Unix) with perl code fragments mixed in. Those perl code fragment 413 will generate all the configuration dependent data, including all the 414 rules needed to build end product files and intermediary files alike. 415 At a minimum, there must be a perl code fragment that defines a set of 416 functions that are used to generates specific build-file rules, to 417 build static libraries from object files, to build shared libraries 418 from static libraries, to programs from object files and libraries, 419 etc. 420 421 generatesrc - function that produces build file lines to generate 422 a source file from some input. 423 424 It's called like this: 425 426 generatesrc(src => "PATH/TO/tobegenerated", 427 generator => [ "generatingfile", ... ] 428 generator_incs => [ "INCL/PATH", ... ] 429 generator_deps => [ "dep1", ... ] 430 incs => [ "INCL/PATH", ... ], 431 deps => [ "dep1", ... ], 432 intent => one of "libs", "dso", "bin" ); 433 434 'src' has the name of the file to be generated. 435 'generator' is the command or part of command to 436 generate the file, of which the first item is 437 expected to be the file to generate from. 438 generatesrc() is expected to analyse and figure out 439 exactly how to apply that file and how to capture 440 the result. 'generator_incs' and 'generator_deps' 441 are include directories and files that the generator 442 file itself depends on. 'incs' and 'deps' are 443 include directories and files that are used if $(CC) 444 is used as an intermediary step when generating the 445 end product (the file indicated by 'src'). 'intent' 446 indicates what the generated file is going to be 447 used for. 448 449 src2obj - function that produces build file lines to build an 450 object file from source files and associated data. 451 452 It's called like this: 453 454 src2obj(obj => "PATH/TO/objectfile", 455 srcs => [ "PATH/TO/sourcefile", ... ], 456 deps => [ "dep1", ... ], 457 incs => [ "INCL/PATH", ... ] 458 intent => one of "lib", "dso", "bin" ); 459 460 'obj' has the intended object file with `.o` 461 extension, src2obj() is expected to change it to 462 something more suitable for the platform. 463 'srcs' has the list of source files to build the 464 object file, with the first item being the source 465 file that directly corresponds to the object file. 466 'deps' is a list of explicit dependencies. 'incs' 467 is a list of include file directories. Finally, 468 'intent' indicates what this object file is going 469 to be used for. 470 471 obj2lib - function that produces build file lines to build a 472 static library file ("libfoo.a" in Unix terms) from 473 object files. 474 475 called like this: 476 477 obj2lib(lib => "PATH/TO/libfile", 478 objs => [ "PATH/TO/objectfile", ... ]); 479 480 'lib' has the intended library file name *without* 481 extension, obj2lib is expected to add that. 'objs' 482 has the list of object files to build this library. 483 484 libobj2shlib - backward compatibility function that's used the 485 same way as obj2shlib (described next), and was 486 expected to build the shared library from the 487 corresponding static library when that was suitable. 488 NOTE: building a shared library from a static 489 library is now DEPRECATED, as they no longer share 490 object files. Attempting to do this will fail. 491 492 obj2shlib - function that produces build file lines to build a 493 shareable object library file ("libfoo.so" in Unix 494 terms) from the corresponding object files. 495 496 called like this: 497 498 obj2shlib(shlib => "PATH/TO/shlibfile", 499 lib => "PATH/TO/libfile", 500 objs => [ "PATH/TO/objectfile", ... ], 501 deps => [ "PATH/TO/otherlibfile", ... ]); 502 503 'lib' has the base (static) library file name 504 *without* extension. This is useful in case 505 supporting files are needed (such as import 506 libraries on Windows). 507 'shlib' has the corresponding shared library name 508 *without* extension. 'deps' has the list of other 509 libraries (also *without* extension) this library 510 needs to be linked with. 'objs' has the list of 511 object files to build this library. 512 513 obj2dso - function that produces build file lines to build a 514 dynamic shared object file from object files. 515 516 called like this: 517 518 obj2dso(lib => "PATH/TO/libfile", 519 objs => [ "PATH/TO/objectfile", ... ], 520 deps => [ "PATH/TO/otherlibfile", 521 ... ]); 522 523 This is almost the same as obj2shlib, but the 524 intent is to build a shareable library that can be 525 loaded in runtime (a "plugin"...). 526 527 obj2bin - function that produces build file lines to build an 528 executable file from object files. 529 530 called like this: 531 532 obj2bin(bin => "PATH/TO/binfile", 533 objs => [ "PATH/TO/objectfile", ... ], 534 deps => [ "PATH/TO/libfile", ... ]); 535 536 'bin' has the intended executable file name 537 *without* extension, obj2bin is expected to add 538 that. 'objs' has the list of object files to build 539 this library. 'deps' has the list of library files 540 (also *without* extension) that the programs needs 541 to be linked with. 542 543 in2script - function that produces build file lines to build a 544 script file from some input. 545 546 called like this: 547 548 in2script(script => "PATH/TO/scriptfile", 549 sources => [ "PATH/TO/infile", ... ]); 550 551 'script' has the intended script file name. 552 'sources' has the list of source files to build the 553 resulting script from. 554 555 Along with the build-file templates is the driving template 556 [`Configurations/common.tmpl`](common.tmpl), which looks through all the 557 information in `%unified_info` and generates all the rulesets to build libraries, 558 programs and all intermediate files, using the rule generating 559 functions defined in the build-file template. 560 561 As an example with the smaller `build.info` set we've seen as an 562 example, producing the rules to build `libcrypto` would result in the 563 following calls: 564 565 # Note: obj2shlib will only be called if shared libraries are 566 # to be produced. 567 # Note 2: obj2shlib must convert the '.o' extension to whatever 568 # is suitable on the local platform. 569 obj2shlib(shlib => "libcrypto", 570 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ], 571 deps => [ ]); 572 573 obj2lib(lib => "libcrypto" 574 objs => [ "crypto/aes.o", "crypto/evp.o", "crypto/cversion.o" ]); 575 576 src2obj(obj => "crypto/aes.o" 577 srcs => [ "crypto/aes.c" ], 578 deps => [ ], 579 incs => [ "include" ], 580 intent => "lib"); 581 582 src2obj(obj => "crypto/evp.o" 583 srcs => [ "crypto/evp.c" ], 584 deps => [ ], 585 incs => [ "include" ], 586 intent => "lib"); 587 588 src2obj(obj => "crypto/cversion.o" 589 srcs => [ "crypto/cversion.c" ], 590 deps => [ "crypto/buildinf.h" ], 591 incs => [ "include" ], 592 intent => "lib"); 593 594 generatesrc(src => "crypto/buildinf.h", 595 generator => [ "util/mkbuildinf.pl", "\"$(CC)", 596 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ], 597 generator_incs => [ "util" ], 598 generator_deps => [ "util/Foo.pm" ], 599 incs => [ ], 600 deps => [ ], 601 intent => "lib"); 602 603 The returned strings from all those calls are then concatenated 604 together and written to the resulting build-file. 605
Indexes created Sun Apr 12 00:22:20 UTC 2026