1 Design document for the unified scheme data 2 =========================================== 3 4 How are things connected? 5 ------------------------- 6 7 The unified scheme takes all its data from the build.info files seen 8 throughout the source tree. These files hold the minimum information 9 needed to build end product files from diverse sources. See the 10 section on build.info files below. 11 12 From the information in build.info files, Configure builds up an 13 information database as a hash table called %unified_info, which is 14 stored in configdata.pm, found at the top of the build tree (which may 15 or may not be the same as the source tree). 16 17 Configurations/common.tmpl uses the data from %unified_info to 18 generate the rules for building end product files as well as 19 intermediary files with the help of a few functions found in the 20 build-file templates. See the section on build-file templates further 21 down for more information. 22 23 build.info files 24 ---------------- 25 26 As mentioned earlier, build.info files are meant to hold the minimum 27 information needed to build output files, and therefore only (with a 28 few possible exceptions [1]) have information about end products (such 29 as scripts, library files and programs) and source files (such as C 30 files, C header files, assembler files, etc). Intermediate files such 31 as object files are rarely directly referred to in build.info files (and 32 when they are, it's always with the file name extension .o), they are 33 inferred by Configure. By the same rule of minimalism, end product 34 file name extensions (such as .so, .a, .exe, etc) are never mentioned 35 in build.info. Their file name extensions will be inferred by the 36 build-file templates, adapted for the platform they are meant for (see 37 sections on %unified_info and build-file templates further down). 38 39 The variables PROGRAMS, LIBS, ENGINES and SCRIPTS are used to declare 40 end products. There are variants for them with '_NO_INST' as suffix 41 (PROGRAM_NO_INST etc) to specify end products that shouldn't get 42 installed. 43 44 The variables SOURCE, DEPEND and INCLUDE are indexed by a produced 45 file, and their values are the source used to produce that particular 46 produced file, extra dependencies, and include directories needed. 47 48 All their values in all the build.info throughout the source tree are 49 collected together and form a set of programs, libraries, engines and 50 scripts to be produced, source files, dependencies, etc etc etc. 51 52 Let's have a pretend example, a very limited contraption of OpenSSL, 53 composed of the program 'apps/openssl', the libraries 'libssl' and 54 'libcrypto', an engine 'engines/ossltest' and their sources and 55 dependencies. 56 57 # build.info 58 LIBS=libcrypto libssl 59 INCLUDE[libcrypto]=include 60 INCLUDE[libssl]=include 61 DEPEND[libssl]=libcrypto 62 63 This is the top directory build.info file, and it tells us that two 64 libraries are to be built, the include directory 'include/' shall be 65 used throughout when building anything that will end up in each 66 library, and that the library 'libssl' depend on the library 67 'libcrypto' to function properly. 68 69 # apps/build.info 70 PROGRAMS=openssl 71 SOURCE[openssl]=openssl.c 72 INCLUDE[openssl]=.. ../include 73 DEPEND[openssl]=../libssl 74 75 This is the build.info file in 'apps/', one may notice that all file 76 paths mentioned are relative to the directory the build.info file is 77 located in. This one tells us that there's a program to be built 78 called 'apps/openssl' (the file name extension will depend on the 79 platform and is therefore not mentioned in the build.info file). It's 80 built from one source file, 'apps/openssl.c', and building it requires 81 the use of '.' and 'include' include directories (both are declared 82 from the point of view of the 'apps/' directory), and that the program 83 depends on the library 'libssl' to function properly. 84 85 # crypto/build.info 86 LIBS=../libcrypto 87 SOURCE[../libcrypto]=aes.c evp.c cversion.c 88 DEPEND[cversion.o]=buildinf.h 89 90 GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" 91 DEPEND[buildinf.h]=../Makefile 92 DEPEND[../util/mkbuildinf.pl]=../util/Foo.pm 93 94 This is the build.info file in 'crypto', and it tells us a little more 95 about what's needed to produce 'libcrypto'. LIBS is used again to 96 declare that 'libcrypto' is to be produced. This declaration is 97 really unnecessary as it's already mentioned in the top build.info 98 file, but can make the info file easier to understand. This is to 99 show that duplicate information isn't an issue. 100 101 This build.info file informs us that 'libcrypto' is built from a few 102 source files, 'crypto/aes.c', 'crypto/evp.c' and 'crypto/cversion.c'. 103 It also shows us that building the object file inferred from 104 'crypto/cversion.c' depends on 'crypto/buildinf.h'. Finally, it 105 also shows the possibility to declare how some files are generated 106 using some script, in this case a perl script, and how such scripts 107 can be declared to depend on other files, in this case a perl module. 108 109 Two things are worth an extra note: 110 111 'DEPEND[cversion.o]' mentions an object file. DEPEND indexes is the 112 only location where it's valid to mention them 113 114 Lines in 'BEGINRAW'..'ENDRAW' sections must always mention files as 115 seen from the top directory, no exception. 116 117 # ssl/build.info 118 LIBS=../libssl 119 SOURCE[../libssl]=tls.c 120 121 This is the build.info file in 'ssl/', and it tells us that the 122 library 'libssl' is built from the source file 'ssl/tls.c'. 123 124 # engines/build.info 125 ENGINES=dasync 126 SOURCE[dasync]=e_dasync.c 127 DEPEND[dasync]=../libcrypto 128 INCLUDE[dasync]=../include 129 130 ENGINES_NO_INST=ossltest 131 SOURCE[ossltest]=e_ossltest.c 132 DEPEND[ossltest]=../libcrypto.a 133 INCLUDE[ossltest]=../include 134 135 This is the build.info file in 'engines/', telling us that two engines 136 called 'engines/dasync' and 'engines/ossltest' shall be built, that 137 dasync's source is 'engines/e_dasync.c' and ossltest's source is 138 'engines/e_ossltest.c' and that the include directory 'include/' may 139 be used when building anything that will be part of these engines. 140 Also, both engines depend on the library 'libcrypto' to function 141 properly. ossltest is explicitly linked with the static variant of 142 the library 'libcrypto'. Finally, only dasync is being installed, as 143 ossltest is only for internal testing. 144 145 When Configure digests these build.info files, the accumulated 146 information comes down to this: 147 148 LIBS=libcrypto libssl 149 SOURCE[libcrypto]=crypto/aes.c crypto/evp.c crypto/cversion.c 150 DEPEND[crypto/cversion.o]=crypto/buildinf.h 151 INCLUDE[libcrypto]=include 152 SOURCE[libssl]=ssl/tls.c 153 INCLUDE[libssl]=include 154 DEPEND[libssl]=libcrypto 155 156 PROGRAMS=apps/openssl 157 SOURCE[apps/openssl]=apps/openssl.c 158 INCLUDE[apps/openssl]=. include 159 DEPEND[apps/openssl]=libssl 160 161 ENGINES=engines/dasync 162 SOURCE[engines/dasync]=engines/e_dasync.c 163 DEPEND[engines/dasync]=libcrypto 164 INCLUDE[engines/dasync]=include 165 166 ENGINES_NO_INST=engines/ossltest 167 SOURCE[engines/ossltest]=engines/e_ossltest.c 168 DEPEND[engines/ossltest]=libcrypto.a 169 INCLUDE[engines/ossltest]=include 170 171 GENERATE[crypto/buildinf.h]=util/mkbuildinf.pl "$(CC) $(CFLAGS)" "$(PLATFORM)" 172 DEPEND[crypto/buildinf.h]=Makefile 173 DEPEND[util/mkbuildinf.pl]=util/Foo.pm 174 175 176 A few notes worth mentioning: 177 178 LIBS may be used to declare routine libraries only. 179 180 PROGRAMS may be used to declare programs only. 181 182 ENGINES may be used to declare engines only. 183 184 The indexes for SOURCE must only be end product files, such as 185 libraries, programs or engines. The values of SOURCE variables must 186 only be source files (possibly generated). 187 188 INCLUDE and DEPEND shows a relationship between different files 189 (usually produced files) or between files and directories, such as a 190 program depending on a library, or between an object file and some 191 extra source file. 192 193 When Configure processes the build.info files, it will take it as 194 truth without question, and will therefore perform very few checks. 195 If the build tree is separate from the source tree, it will assume 196 that all built files and up in the build directory and that all source 197 files are to be found in the source tree, if they can be found there. 198 Configure will assume that source files that can't be found in the 199 source tree (such as 'crypto/bildinf.h' in the example above) are 200 generated and will be found in the build tree. 201 202 203 The %unified_info database 204 -------------------------- 205 206 The information in all the build.info get digested by Configure and 207 collected into the %unified_info database, divided into the following 208 indexes: 209 210 depends => a hash table containing 'file' => [ 'dependency' ... ] 211 pairs. These are directly inferred from the DEPEND 212 variables in build.info files. 213 214 engines => a list of engines. These are directly inferred from 215 the ENGINES variable in build.info files. 216 217 generate => a hash table containing 'file' => [ 'generator' ... ] 218 pairs. These are directly inferred from the GENERATE 219 variables in build.info files. 220 221 includes => a hash table containing 'file' => [ 'include' ... ] 222 pairs. These are directly inferred from the INCLUDE 223 variables in build.info files. 224 225 install => a hash table containing 'type' => [ 'file' ... ] pairs. 226 The types are 'programs', 'libraries', 'engines' and 227 'scripts', and the array of files list the files of 228 that type that should be installed. 229 230 libraries => a list of libraries. These are directly inferred from 231 the LIBS variable in build.info files. 232 233 programs => a list of programs. These are directly inferred from 234 the PROGRAMS variable in build.info files. 235 236 rawlines => a list of build-file lines. These are a direct copy of 237 the BEGINRAW..ENDRAW lines in build.info files. Note: 238 only the BEGINRAW..ENDRAW section for the current 239 platform are copied, the rest are ignored. 240 241 scripts => a list of scripts. There are directly inferred from 242 the SCRIPTS variable in build.info files. 243 244 sources => a hash table containing 'file' => [ 'sourcefile' ... ] 245 pairs. These are indirectly inferred from the SOURCE 246 variables in build.info files. Object files are 247 mentioned in this hash table, with source files from 248 SOURCE variables, and AS source files for programs and 249 libraries. 250 251 shared_sources => 252 a hash table just like 'sources', but only as source 253 files (object files) for building shared libraries. 254 255 As an example, here is how the build.info files example from the 256 section above would be digested into a %unified_info table: 257 258 our %unified_info = ( 259 "depends" => 260 { 261 "apps/openssl" => 262 [ 263 "libssl", 264 ], 265 "crypto/buildinf.h" => 266 [ 267 "Makefile", 268 ], 269 "crypto/cversion.o" => 270 [ 271 "crypto/buildinf.h", 272 ], 273 "engines/dasync" => 274 [ 275 "libcrypto", 276 ], 277 "engines/ossltest" => 278 [ 279 "libcrypto.a", 280 ], 281 "libssl" => 282 [ 283 "libcrypto", 284 ], 285 "util/mkbuildinf.pl" => 286 [ 287 "util/Foo.pm", 288 ], 289 }, 290 "engines" => 291 [ 292 "engines/dasync", 293 "engines/ossltest", 294 ], 295 "generate" => 296 { 297 "crypto/buildinf.h" => 298 [ 299 "util/mkbuildinf.pl", 300 "\"\$(CC)", 301 "\$(CFLAGS)\"", 302 "\"$(PLATFORM)\"", 303 ], 304 }, 305 "includes" => 306 { 307 "apps/openssl" => 308 [ 309 ".", 310 "include", 311 ], 312 "engines/ossltest" => 313 [ 314 "include" 315 ], 316 "libcrypto" => 317 [ 318 "include", 319 ], 320 "libssl" => 321 [ 322 "include", 323 ], 324 "util/mkbuildinf.pl" => 325 [ 326 "util", 327 ], 328 } 329 "install" => 330 { 331 "engines" => 332 [ 333 "engines/dasync", 334 ], 335 "libraries" => 336 [ 337 "libcrypto", 338 "libssl", 339 ], 340 "programs" => 341 [ 342 "apps/openssl", 343 ], 344 }, 345 "libraries" => 346 [ 347 "libcrypto", 348 "libssl", 349 ], 350 "programs" => 351 [ 352 "apps/openssl", 353 ], 354 "rawlines" => 355 [ 356 ], 357 "sources" => 358 { 359 "apps/openssl" => 360 [ 361 "apps/openssl.o", 362 ], 363 "apps/openssl.o" => 364 [ 365 "apps/openssl.c", 366 ], 367 "crypto/aes.o" => 368 [ 369 "crypto/aes.c", 370 ], 371 "crypto/cversion.o" => 372 [ 373 "crypto/cversion.c", 374 ], 375 "crypto/evp.o" => 376 [ 377 "crypto/evp.c", 378 ], 379 "engines/e_dasync.o" => 380 [ 381 "engines/e_dasync.c", 382 ], 383 "engines/dasync" => 384 [ 385 "engines/e_dasync.o", 386 ], 387 "engines/e_ossltest.o" => 388 [ 389 "engines/e_ossltest.c", 390 ], 391 "engines/ossltest" => 392 [ 393 "engines/e_ossltest.o", 394 ], 395 "libcrypto" => 396 [ 397 "crypto/aes.c", 398 "crypto/cversion.c", 399 "crypto/evp.c", 400 ], 401 "libssl" => 402 [ 403 "ssl/tls.c", 404 ], 405 "ssl/tls.o" => 406 [ 407 "ssl/tls.c", 408 ], 409 }, 410 ); 411 412 As can be seen, everything in %unified_info is fairly simple suggest 413 of information. Still, it tells us that to build all programs, we 414 must build 'apps/openssl', and to build the latter, we will need to 415 build all its sources ('apps/openssl.o' in this case) and all the 416 other things it depends on (such as 'libssl'). All those dependencies 417 need to be built as well, using the same logic, so to build 'libssl', 418 we need to build 'ssl/tls.o' as well as 'libcrypto', and to build the 419 latter... 420 421 422 Build-file templates 423 -------------------- 424 425 Build-file templates are essentially build-files (such as Makefile on 426 Unix) with perl code fragments mixed in. Those perl code fragment 427 will generate all the configuration dependent data, including all the 428 rules needed to build end product files and intermediary files alike. 429 At a minimum, there must be a perl code fragment that defines a set of 430 functions that are used to generates specific build-file rules, to 431 build static libraries from object files, to build shared libraries 432 from static libraries, to programs from object files and libraries, 433 etc. 434 435 generatesrc - function that produces build file lines to generate 436 a source file from some input. 437 438 It's called like this: 439 440 generatesrc(src => "PATH/TO/tobegenerated", 441 generator => [ "generatingfile", ... ] 442 generator_incs => [ "INCL/PATH", ... ] 443 generator_deps => [ "dep1", ... ] 444 incs => [ "INCL/PATH", ... ], 445 deps => [ "dep1", ... ], 446 intent => one of "libs", "dso", "bin" ); 447 448 'src' has the name of the file to be generated. 449 'generator' is the command or part of command to 450 generate the file, of which the first item is 451 expected to be the file to generate from. 452 generatesrc() is expected to analyse and figure out 453 exactly how to apply that file and how to capture 454 the result. 'generator_incs' and 'generator_deps' 455 are include directories and files that the generator 456 file itself depends on. 'incs' and 'deps' are 457 include directories and files that are used if $(CC) 458 is used as an intermediary step when generating the 459 end product (the file indicated by 'src'). 'intent' 460 indicates what the generated file is going to be 461 used for. 462 463 src2obj - function that produces build file lines to build an 464 object file from source files and associated data. 465 466 It's called like this: 467 468 src2obj(obj => "PATH/TO/objectfile", 469 srcs => [ "PATH/TO/sourcefile", ... ], 470 deps => [ "dep1", ... ], 471 incs => [ "INCL/PATH", ... ] 472 intent => one of "lib", "dso", "bin" ); 473 474 'obj' has the intended object file *without* 475 extension, src2obj() is expected to add that. 476 'srcs' has the list of source files to build the 477 object file, with the first item being the source 478 file that directly corresponds to the object file. 479 'deps' is a list of explicit dependencies. 'incs' 480 is a list of include file directories. Finally, 481 'intent' indicates what this object file is going 482 to be used for. 483 484 obj2lib - function that produces build file lines to build a 485 static library file ("libfoo.a" in Unix terms) from 486 object files. 487 488 called like this: 489 490 obj2lib(lib => "PATH/TO/libfile", 491 objs => [ "PATH/TO/objectfile", ... ]); 492 493 'lib' has the intended library file name *without* 494 extension, obj2lib is expected to add that. 'objs' 495 has the list of object files (also *without* 496 extension) to build this library. 497 498 libobj2shlib - function that produces build file lines to build a 499 shareable object library file ("libfoo.so" in Unix 500 terms) from the corresponding static library file 501 or object files. 502 503 called like this: 504 505 libobj2shlib(shlib => "PATH/TO/shlibfile", 506 lib => "PATH/TO/libfile", 507 objs => [ "PATH/TO/objectfile", ... ], 508 deps => [ "PATH/TO/otherlibfile", ... ]); 509 510 'lib' has the intended library file name *without* 511 extension, libobj2shlib is expected to add that. 512 'shlib' has the corresponding shared library name 513 *without* extension. 'deps' has the list of other 514 libraries (also *without* extension) this library 515 needs to be linked with. 'objs' has the list of 516 object files (also *without* extension) to build 517 this library. 518 519 This function has a choice; it can use the 520 corresponding static library as input to make the 521 shared library, or the list of object files. 522 523 obj2dynlib - function that produces build file lines to build a 524 dynamically loadable library file ("libfoo.so" on 525 Unix) from object files. 526 527 called like this: 528 529 obj2dynlib(lib => "PATH/TO/libfile", 530 objs => [ "PATH/TO/objectfile", ... ], 531 deps => [ "PATH/TO/otherlibfile", 532 ... ]); 533 534 This is almost the same as libobj2shlib, but the 535 intent is to build a shareable library that can be 536 loaded in runtime (a "plugin"...). The differences 537 are subtle, one of the most visible ones is that the 538 resulting shareable library is produced from object 539 files only. 540 541 obj2bin - function that produces build file lines to build an 542 executable file from object files. 543 544 called like this: 545 546 obj2bin(bin => "PATH/TO/binfile", 547 objs => [ "PATH/TO/objectfile", ... ], 548 deps => [ "PATH/TO/libfile", ... ]); 549 550 'bin' has the intended executable file name 551 *without* extension, obj2bin is expected to add 552 that. 'objs' has the list of object files (also 553 *without* extension) to build this library. 'deps' 554 has the list of library files (also *without* 555 extension) that the programs needs to be linked 556 with. 557 558 in2script - function that produces build file lines to build a 559 script file from some input. 560 561 called like this: 562 563 in2script(script => "PATH/TO/scriptfile", 564 sources => [ "PATH/TO/infile", ... ]); 565 566 'script' has the intended script file name. 567 'sources' has the list of source files to build the 568 resulting script from. 569 570 Along with the build-file templates is the driving engine 571 Configurations/common.tmpl, which looks through all the information in 572 %unified_info and generates all the rulesets to build libraries, 573 programs and all intermediate files, using the rule generating 574 functions defined in the build-file template. 575 576 As an example with the smaller build.info set we've seen as an 577 example, producing the rules to build 'libcrypto' would result in the 578 following calls: 579 580 # Note: libobj2shlib will only be called if shared libraries are 581 # to be produced. 582 # Note 2: libobj2shlib gets both the name of the static library 583 # and the names of all the object files that go into it. It's up 584 # to the implementation to decide which to use as input. 585 # Note 3: common.tmpl peals off the ".o" extension from all object 586 # files, as the platform at hand may have a different one. 587 libobj2shlib(shlib => "libcrypto", 588 lib => "libcrypto", 589 objs => [ "crypto/aes", "crypto/evp", "crypto/cversion" ], 590 deps => [ ]); 591 592 obj2lib(lib => "libcrypto" 593 objs => [ "crypto/aes", "crypto/evp", "crypto/cversion" ]); 594 595 src2obj(obj => "crypto/aes" 596 srcs => [ "crypto/aes.c" ], 597 deps => [ ], 598 incs => [ "include" ], 599 intent => "lib"); 600 601 src2obj(obj => "crypto/evp" 602 srcs => [ "crypto/evp.c" ], 603 deps => [ ], 604 incs => [ "include" ], 605 intent => "lib"); 606 607 src2obj(obj => "crypto/cversion" 608 srcs => [ "crypto/cversion.c" ], 609 deps => [ "crypto/buildinf.h" ], 610 incs => [ "include" ], 611 intent => "lib"); 612 613 generatesrc(src => "crypto/buildinf.h", 614 generator => [ "util/mkbuildinf.pl", "\"$(CC)", 615 "$(CFLAGS)\"", "\"$(PLATFORM)\"" ], 616 generator_incs => [ "util" ], 617 generator_deps => [ "util/Foo.pm" ], 618 incs => [ ], 619 deps => [ ], 620 intent => "lib"); 621 622 The returned strings from all those calls are then concatenated 623 together and written to the resulting build-file. 624
Indexes created Tue Apr 21 00:23:25 UTC 2026